What’s wrong with using graduation rates to rank colleges?

Naturally, a high graduation rate is a positive indicator. And we won’t suggest that this information is not pertinent to your college search. But speaking strictly in terms of how colleges are ranked against one another, the graduation rate is a problematic metric—one which is diminished by selection bias, limited in scope, and given over to innate socioeconomic inequalities.

Previously, we explored the outsized role that reputation surveys play in forming college rankings. By contrast to reputation surveys, graduation rates aren’t impeded by opaqueness or subjectivity. But they do carry their own problematic shortcomings, especially in the context of college rankings, where a causal connection is implied between the quality of a college education and the outcome of graduation.

If you’re interested in searching for excellent colleges which are ranked by a more objective and comprehensive metric, check out The 50 Best Colleges & Universities. Of course, you’ll find all the usual specs like cost, acceptance rate, and graduation rate. However, the ranking itself is driven by a unique metric called Concentrated Influence. Concentrated Influence takes the combined influence score of a college or university’s top academic influencers (including faculty and alums) and divides it by the school’s total number of undergraduates. Check out the schools whose professors, students, and graduates are making the most profound impact on the real world today.

Or read on to find out why graduation rates are a poor basis for ranking colleges and universities.

Where do graduation rates come from?

Most of the major ranking systems that use graduation rate as a factor source this information from the U.S. Department of Education’s Integrated Postsecondary Education Data System (IPEDS). The “overall graduation rate” measures the rate of graduation for first-time, full-time students who earn a bachelor’s degree from the institution where they began their college education within the scope of six years. According to Forbes, this figure is an important foundational metric for ranking systems from industry leaders like U.S. News & World Report, College Scorecard, and even Google’s emergent college search function.

Graduation rates are not without value. Indeed, a college with an exceptionally low graduation rate should give a prospective student pause. It is fair to say that schools with low graduation rates are doing a less effective job at delivering on their educational promise than are schools with uniformly high rates. One need only consider the sharp divergence in overall 6-year graduation rates between public institutions (61%), private nonprofit institutions (67%), and private for-profit schools (25%).

The for-profit education sector’s vulnerability to costly, low-performing, and even fraudulent enterprises has been well-publicized over the last two decades. Looking at the figures above, it’s clear that low graduation rates offer some clues about the associated risk of attending a for-profit college. So if you’re looking to eliminate the lesser options from your list of colleges, low graduation rates can help.

But things get a lot hazier when it comes to college ranking. Naturally, for-profit schools with problematically low graduation rates aren’t likely to appear on the widely publicized annual “best school” rankings. The question then becomes, how effective are graduation rates at telling us about the average, excellent, and elite colleges and universities?

And consequently, how trustworthy are college rankings which rely heavily on graduation rates? U.S. News & World Report, for instance, uses a weighted ranking system which incorporates various factors into a total score. Among those factors, “Outcomes” account for 40% of a school’s ranking position. This figure is composed of “graduation (17.6%) and retention (4.4%) (22% combined); graduation rate performance (8%); social mobility (5%) and; new this year, graduate indebtedness (5%).”

These figures are almost entirely dependent upon incidences of student graduation under the terms conditioned by IPEDS, excluding retention, which refers to the rate of returning students between their first and second year of college. So while USNWR offers what would appear to be a nuanced breakdown of weighted factors, it may not be too reductive to say that 35.6% (Outcomes minus retention) of its rank scoring is based on the IPEDS graduation rate figure.

So if graduation rates are serviceable for at least eliminating low performers from your list of prospective schools, why aren’t they just as useful for ranking the rest?

Elite Schools and Selection Bias

The short answer is that graduation rates say more about who is being admitted into a school than they say about the excellence of the school itself. This is not to suggest that a school with a high graduation rate is not also excellent, in all probability. But it also prevents us from concluding that this excellence is the cause for a high graduation rate.

Inevitably, the most elite schools are also those with the lowest admission rates. Harvard and Princeton led the pack in 2020 with minuscule overall acceptance rates of 5%. Fellow Ivy Leaguers Columbia (6%) and Yale (6%) and Princeton (6.5%) are nearly just as exclusive. Each of these schools also boasts a graduation rate of 95% or higher.

Intuitively, it would make sense that an elite school would demonstrate a high graduation rate. But we should examine what that means a little more closely. What can we presume about a graduating cohort culled from the top 5-7% of all students? Well, for one thing, we can presume that the majority of these students entered the pool of college applicants with every conceivable advantage.

As Forbes explains it, “The current system makes colleges look good when they enroll students who are already well positioned to succeed, which really means students who are affluent. It makes a college look bad when it does the hard work of patiently supporting less well-off, less well-prepared students as they work toward their degrees.”

Students attending elite colleges already have an economic, educational, and circumstantial edge over the average student in reaching graduation before ever setting foot on campus. Schools that cater to elite students naturally tilt the scales in their own favor by selecting students who profile as far likelier to graduate in six years or less.

So what does the graduation rate actually measure in the case of an elite institution?

“Simply put,” says Forbes, it measures inputs more than it measures outcomes.”

Graduation rates can tell you quite a bit about the types of students enrolled at elite universities—most particularly that these students are among the least likely to drop out of college for financial reasons. As a result, it may actually tell you quite a bit less about the role played by the institution, its courses, or its professors in helping students advance toward graduation or securing a meaningful education.

Simply stated, selection bias is too great a force in shaping graduation rates, and may therefore tell us little about what students actually experience while attending elite institutions. This is to say nothing of how small the population of elite students actually is. In fact, even beyond the top tier of elite institutions, the picture offered by six-year graduation rates is deceptively limited in scope.

Graduation Rates Overlook The Changing Student Profile

To reiterate, when we speak of “graduation rates” as a source of data for the leading ranking entities, we refer to the IPEDS data. Its calculation of first-time, full-time students attending and graduating from a single institution in six years or less imposes a very specific definition on the aspiring college graduate. In the past, we might have referred to any student falling outside of this definition as a non-traditional student. Transfer students, adult-learners, and part-time students made up a fringe cross-section of students who didn’t neatly fit the definition of a campus-dwelling recent high school graduate.

The college landscape is dramatically different today, shaped by a patchwork of on-campus and online colleges offering an extremely wide range of programs including 2- and 4-year degrees, degree completion programs, part-time scheduling, asynchronous courses, and much more. The consequence is a higher education industry which caters to a far wider range of student types than those included in the IPEDS graduation rate metric.

As Forbes explains, graduation rate “is a statistic that favors the traditional college model, with a group of 18- to 22-year-olds immersed in a residential setting for four years. But, given many important changes in our country and economy, fewer and fewer students fit that old definition. It does not represent how many colleges work these days, and, as a result, the traditional overall graduation rate doesn’t really capture true college success.”

Forbes cites the National Center for Education Statistics, noting that more than a third of all students in college are part-timers. A quarter of all students are attending community colleges, with most going on to pursue a bachelor’s degree at a 4-year school. On top of that, some 40% of college students today are older than 24 years of age. The vast majority of these adult learners are students who have previously lapsed in their education before ultimately returning to pursue a degree.

Not only are these enormous data points sidelined by the standard graduation rate metric, but in many cases, lapsed and returning students will count as points of failure against their originating schools. So schools who do lose students to the circumstances of life—most likely events such as financial strain, work demands, or parenting responsibilities—will see their rankings dinged regardless of whether or not those students do eventually get back on track.

In other words, when it comes to rankings, schools take an inherent risk by making room for students facing challenging financial or personal circumstances. It’s worth asking just how many top schools are generally averse to this risk.

The Inequality Baked Into Graduation Rate Rankings

This is troubling for more reasons than the simple statistical emptiness of rankings which rely on graduation rates. Consider for just a moment the factors that might prevent a student from qualifying as full-time. This might describe a student who must balance work and family responsibilities with the pursuit of a degree. It might describe a person who must stagger the purchase of credits as a way of affording a college education. It may be a student with learning challenges who is better served by carrying a lighter course-load.

And what about students who are excluded from the overall graduation rate calculation because they are not first-time students. This may describe working adults who have departed from and consequently returned to their pursuit of a degree because family, personal, or economic challenges required them to do so. It may describe parents who are returning to school after having a child. It may be those who have parlayed work experience into a degree completion program.

What about savings-conscious students who have first earned an associate degree at a community college before continuing into a four-year degree program. The institutions that create access and emphasize opportunities for transfer students receive zero ranking credit for their largesse.

In other words, the snapshot provided by graduation rates does more than simply train our focus on the elite, exclusive, and expensive institutions on the college landscape. It also provides an extremely narrow definition for those students worthy of consideration. In doing so, this ranking metric nudges some of the fastest growing student populations out of the picture. This population includes those navigating their college experience over and around personal and financial challenges, those returning to school in the interests of career advancement and those who have taken strategic steps to limit their post-graduate debt.

By elbowing these demographics out of the picture, IPEDS graduation rates don’t simply undercut schools who make genuine overtures to addressing socioeconomic inequalities. They also inform rankings that are substantially less useful and relevant to the millions of students who can’t be described as first-time, full-time, single-school, six-year graduates. If you don’t fall comfortably into this set of stringent qualifiers, college rankings based on graduation rates aren’t actually speaking to you. They may be advertising to you. But they aren’t speaking to you.

Why Influence Matters

This problem speaks precisely to the reason that Academic Influence takes a completely alternate approach to ranking colleges. While graduation rates presume to measure a school’s success at producing successful students, the definition of a successful student is so deeply limited by IPEDS qualifiers that it undermines the very real and meaningful achievements of far too many schools and students.

To wit, Forbes points out that John F. Kennedy and Barack Obama were both transfer students. The former presidents of the United States are not counted when calculating the graduation rates for their respective institutions. By contrast, they are indeed counted when calculating Concentrated Influence based on the achievements of alumni.

If you’re interested in searching for excellent colleges which are ranked by the more meaningful, comprehensive, and unbiased metric of Concentrated Influence, check out our ranking of The 50 Best Colleges & Universities.