The Ranking Metrics at AcademicInfluence.com
AcademicInfluence.com ranks academic institutions and their disciplinary programs. We focus especially on ranking by influence as computed by our InfluenceRanking™ engine. Basing our rankings on influence sets us apart from other academic rankings. Indeed, our contribution to the academic ranking business stands or falls with the value of insights gained through our influence-based rankings.
Even though we emphasize influence in our rankings, we don’t see influence as the only legitimate criterion for academic rankings. People interested in higher education come with many questions about how to compare academic institutions and disciplinary programs. Some of these questions may have clear answers, others less so.
What is the richest college or university? This question has a clear answer, and requires simply looking at the endowments of schools (it’s Harvard). What is the most beautiful college campus in America? This question addresses a matter of taste, and has no single right answer. Do you like mountains, then try Colorado. Do you like gorges, then try New York’s Finger Lakes. Do you like open prairies, then try Kansas.
At AcademicInfluence.com, we believe that prospective students are best served by being able to rank schools in many different ways in line with the many different questions they may have. Different queries require different ranking metrics. Let a thousand flowers bloom. And yet, we insist that they be flowers and not weeds. That’s why we base our rankings on metrics.
At AcademicInfluence.com, we insist that our rankings be objective and even quantifiable, where the numbers are not drawn out of a hat but depend on applying a clearly defined algorithm to publicly available data. The meaning of the numbers needs to have a clear meaning, and anyone with access to the input data should be in a position to reverse-engineer our algorithm.
Obviously, the algorithm, our InfluenceRanking™ engine, is our secret sauce, so we are not going to make its code publicly available. But we are happy to describe what it is doing and how it arrives at its answers, at least in broad strokes.
The Problems with Subjective Rankings
Why is it important to develop an algorithm that ranks schools objectively? Most of the leading ranking organizations employ ranking approaches with subjective elements. Many depend heavily on self-reports, in which survey responders detail their own personal assessment or perception of a school’s characteristics, such as reputation (20 percent of the U.S. News ranking, for instance, is based on a reputation survey). The problems with self-reports include lying (making a school seem better or worse than it may actually be), misperception (simply being mistaken in one’s perception of the school), and moral hazard (perverse incentives to portray a school one way rather than another).
Rankings of schools by alumni on a five-star scale or by their salary data after graduation face these same problems. But they also face a selection effect: What sort of alumni tend to give a five-star rating to a school from which they graduated? What if people like to think well of things into which they invested time and money? What if they want to warn people about mistakes they’ve committed? And which impulse is stronger? Such rankings are not only subjective but also easily skewed. Ditto for salary reports: Do people working at minimum wage upon graduation really want the world to know this, even if they share this knowledge anonymously?
We’re not saying that subjective ranking approaches like this have no value. At their best, they provide social validation, which can be valuable. It’s just that at AcademicInfluence.com, we have become convinced, through our long experience with academic rankings, that such ranking approaches, especially without extensive caveats, do more to mislead than enlighten. In place of rankings that employ subjective elements, AcademicInfluence.com therefore focuses on ranking metrics whose numbers are based on precise mathematical calculations and which are drawn from unambiguous, publicly available data. No hand-waving, no charades, no voodoo.
Our Growing List of Ranking Metrics
What follows is a growing list of ranking metrics developed at AcademicInfluence.com, along with links to their explanation on this website:
- Influence
- This is our bread-and-butter ranking metric, as computed by the InfluenceRanking™ engine. For its explanation, see ”Methodology: How and Why We Rank by Influence″ and ”The InfluenceRanking™ Engine: The Nuts and Bolts of Our Ranking Technology.”
- Concentrated Influence
- This metric, derived from the previous one by controlling for size of undergraduate student body, seems especially apt for capturing academic excellence at the undergraduate level and thus for characterizing the best institutions of higher learning and the best disciplinary or departmental programs for college majors. For its explanation, see ”Concentrated Influence.”
- Academic Stewardship
- This metric, derived from the previous one by also controlling for a school’s financial resources, is especially apt for capturing how well schools are at stewarding their resources, especially in maintaining and growing their influence. For its explanation, see ”Measuring Academic Stewardship.”
- Desirability
- Unlike influence, which looks to the influential people (faculty and alumni) who are already associated with schools, desirability asks how students admitted to two or more schools vote with their feet in deciding to attend one school to the exclusion of others. Desirability asks how schools rank if we simply watch the enrollment decisions of students in light of the schools to which they were admitted. Whereas rankings based on influence are faculty- and alumni-centric, rankings based on desirability are student-centric. For a fuller explanation of desirability, see ”What Is Desirability?″
