Will I graduate is the best opening question when choosing a college. Aggregate graduation percentages are widely available, and the percentages reveal something about overall college quality. Are these percentages reflected in college rankings?
This question occurred to me last Thursday, when Portland’s Oregonian published the editorial Getting your money’s worth out of college in Oregon isn’t a simple equation. Unobjectionably, the editors argue that students should bake their employment prospects and costs into their college selection. The editorial includes a list of Oregon colleges and universities that around in national ranking among based on an index from an organization called Educate to Career (ETC). This index includes 1222 degree-granting four year institutions in the United States.
The ordering in the Oregonian’s list is surprising, given the relative stature of various Oregon colleges:
|| Western Oregon University
|| Oregon Institute of Technology
|| Linfield College (Not included in the Oregonian article).
|| George Fox University
|| Pacific University
|| University of Oregon
|| University of Portland
|| Eastern Oregon University
|| Portland State University
|| Southern Oregon University
|| Willamette University
|| Lewis and Clark College
|| Reed College
According to ETC, their, “…college ranking system empirically determines the economic value added, by each college ranked within our system. We define economic value added as being the improvement in earnings and employability of graduates; measured against the total cost of the education.”
ETC describes the “characteristics of colleges by their rank” as follows (emphasis mine):
Colleges in the top 1/3 of the ETC Index
- A relatively high percentage of graduates will be employed in their field of study.
- A majority of students will graduate in 4 or 5 years.
- Earnings of graduates are relatively high.
- Loan default rates are very low.
Colleges in the lower 1/3 of the ETC Index
- A relatively high percentage of recent graduates are delinquent or in default on student loans.
- Most of the students will graduate in 6 years, or later, if at all.
- Recent graduates have high student loan balances
- Graduates are not employed in occupations that utilize their field of study
No one should over-interpret college listicle fodder, and usually, rankings can be entertaining (especially this one from 2009). Users of ETC’s website can assess the quality of their products for themselves. (Disclosure: I have taught at Lewis & Clark College the last two years, which ETC ranked 1076 of 1222). But do the comprehensive index-based lists reflect differences in graduation rates? I compared the ETC index to data compiled from the National Center for Education Statistics for 2010 by the The Chronicle of Higher Education.
Some of the colleges on the Oregonian’s list have relatively low rates of graduation, and I looked at ETC’s rankings in relation to four-year and six-year graduation rates.
||4 year Graduation %
||6 year Graduation %
||Western Oregon University *
||Oregon Institute of Technology *
||George Fox University
||University of Oregon *
||University of Portland
||Eastern Oregon University *
||Portland State University *
||Southern Oregon University *
||Lewis and Clark College
- Public university. ETC did not include Oregon State University in its index.
Overall, graduation rates don’t count for much in these ratings, and taken literally, the ETC claims about graduation rates are incorrect. A significant number of colleges in the top 1/3 of the ETC Index (113, or 27.9%) graduate less than half enrollees after six years, and 204, or 50.1%, of the colleges in the lower 1/3 graduate half or more. Presumably, ETC is trying to describe a tendency: they really mean that, generally, higher index values correlate to higher graduate rates.
Nevertheless, the relationship between ETC indexes and both sets of graduation rates is very weak. The relationship for six-year graduation for public colleges is somewhat stronger, but non-existent for private colleges.
- Lowess curves show 99% confidence intervals.
- R2 (SE) = -.010 (.029) R2 (SE) Private = .008 (.037) R2 (SE) Public = .319 (.043)
- Lowess curves show 99% confidence intervals.
- R2 (SE) = -.092 (.029) R2 (SE): Private = .013 (.037) R2 (SE) Public = .450 (.041)
The more serious concern is the significant number of colleges that rank very high on the ETC index with graduation rates below fifty percent. The 113 institutions noted above comprise 9.2% of all the colleges in ETC’s rankings.
Is it possible that any college with less than 50% graduation should be considered one of the upper third of a comprehensive college rating, let alone 113 of them? The top 10% of ETC’s rankings still include 29 colleges (or 24%) with 6 year graduation rates below 50%.
Graduation rates aren’t a perfect indicator of quality. The institution may be admitting students whose needs are not reflected in graduation statistics, such as part time students or those looking to take particular classes or extension programs. Or they may just be unable to compete for good students because their reputation has not caught up with their quality. In the case of the ETC index, graduation rates may not reflect its goals in the usual way.
But a low graduation rate is serious, and the ETC Index does not weight it very heavily. Choosing a school with a low graduation rate is a leap of faith, and the Oregonian should have said something about it.