Methodology: How and Why We Rank by Influence

What InfluenceRanking™ Measures and How

Our InfluenceRanking™ engine assesses influence in the academic world by ingesting, analyzing, and evaluating huge amounts of data. At AcademicInfluence.com, we love Big Data. We continually demand

• ever more data points,
• ever more data sources for those data points, and
• ever more variety of those data sources.

The more such data we have (provided we keep it well organized, which we do!), the less prone the data are to staleness, bias, and manipulation by bad actors.

This is why InfluenceRanking consults the largest repositories of open source educational data on the Web and entrusts their analysis to a machine-learning technology developed with funding from the Defense Advanced Research Projects Agency (DARPA). These databases contain billions of continuously updated data points, which by their nature resist biased manipulation and by their scope contain built-in protections against wholesale gaming.

Our technology thus uses machine learning to

• perform statistical document analyses on databases such as Wikipedia and Wikidata,
• collate scholarly and academic citations from databases such as Crossref and Semantic Scholar
• evaluate link networks and semantic signals contained in these databases,
• form influence rankings of academic persons, institutions, and disciplinary programs, and
• make sure the rankings make sense by performing “sanity checks” against other independent information sources such as periodicals, journals, and global media outlets.

Importantly, our machine-learning technology doesn’t simply scrape the web for mentions. Influence at AcademicInfluence.com is therefore not a popularity measure. Rather, the artificial intelligence underlying our InfluenceRanking engine drills down deeply to

• identify people and institutions, locating mentions of them across the Web in 15 different languages,
• associate those mentions with records of achievement in specific fields of study,
• map the people that make up an institution, keeping track of independent work done by those people as well as their affiliations with other institutions, and
• aggregate the influential output of people at an institution, thereby gauging the institution’s influence within disciplinary programs and overall.

Our machine-learning algorithms improve over time at measuring influence of people and institutions. As independent corroboration that our technology is indeed getting better at gauging influence, we employ web traffic analysis from third parties (including Semrush.com and Ahrefs.com) to track the organic search traffic leading to web domains, each domain’s keyword footprints, the number of referring domains, and the authority of those referring domains.

As a further part of this corroboration effort, we also pull in additional secondary sources such as the Integrated Postsecondary Education Data System (IPEDS) data and Bureau of Labor Statistics (BLS) data. These data allow for specialized rankings, including those that track costs, test scores, acceptance rates, and expected income upon graduation. Such specialized rankings are often of interest in their own right and may then be correlated with our influence-based rankings.

Finally, because the InfluenceRanking AI technology is database driven, its results can be updated as quickly as the databases on which it depends. InfluenceRanking is thus capable of generating hourly result updates. This rate of updates turns out to be excessive. To ensure results can be accurately compared and cited, AcademicInfluence.com limits itself to quarterly updates. Quarterly updates incentivize the AcademicInfluence.com team to get the InfluenceRanking engine, along with the databases on which it depends, into the best shape possible for the next update.

For more technical detail about this ranking technology, see ”The InfluenceRanking™ Engine: The Nuts and Bolts of Our Ranking Technology.”

High-Stakes College Rankings

College and university rankings carry enormous weight. A school’s reputation, enrollment numbers, hiring, firing, and revenue stream are all at stake.

A school’s rise in a leading ranking is experienced as recognition and praise, and can translate into greater prestige for alumni diplomas, more grant money for researchers, and a larger pool of top students from which admissions creates the next Class of 20XX.

Conversely, a school’s decline in a leading ranking is experienced as a slight and even humiliation, translating into powerful negatives for the school. As educational journalist Jeffrey Selingo writes in his book Who Gets In and Why, “When a top school drops in the rankings, it’s forced to admit more students in subsequent years and is more generous with financial aid to make up for falling yield rates, according to a study for the National Bureau of Economic Research.”

All this drama leads to perverse incentives. Schools are motivated to “game” the rankings, introducing superficial or cosmetic changes that elevate their ranking but do little to provide a better education for students or a more fruitful work environment for faculty.

Annual announcements of college rankings come with so much fanfare that they make the news. Unfortunately, students often go missing amid all this excitement, gaining few clear insights into which schools or programs are indeed best for them.

This is why we created AcademicInfluence.com. Our rankings are objective and non-gameable, giving students the reliable information they need to draw insightful comparisons among schools. The stakes are too high to settle for anything less.

The Need for InfluenceRanking™

Everyone is drowning in data. That’s especially true of students struggling to make sense of the many conflicting college and university rankings proliferating across the Web. In response, a team of data scientists and academics decided to rethink college and university rankings entirely from the ground up. Taking the view that higher education is ultimately about ideas and the people who promote them, we sought to answer these questions:

• Which factor speaks the most truth about the stature of an idea or of its promoter?
• How can we measure that factor while avoiding “gamed” or biased sources?
• How can we form rankings from such a measure that people can understand and use to answer their questions?

In biology, genetic factors that confer the greatest advantage to their possessors express themselves more widely. Similarly, on the internet, viral memes that resonate with people express themselves more widely.

In all such propagation phenomena—and thus at the core of all ideas and information everywhere—the factor that determines whether something persists into the future is influence, the level of attention and penetration that ideas and entities achieve in the world.

For higher education and the world of knowledge, influence always starts with specific disciplines and specific contributions to those disciplines. Einstein did not start by being influential in the world of learning as such. He became influential through his specific contributions to physics.

From influence in specific disciplines, it is possible to extend influence to wider spheres of research and wider professional impact. Faculty and alumni with influential ideas thus become the foundation of influential degree programs, and a critical mass of such influential degree programs make for influential colleges and universities.

Such is the influence we seek to measure through InfluenceRanking, the influence-based ranking engine developed by our AcademicInfluence.com team of data scientists and academics.

What Constitutes “Best”?

What is the best school or degree program? Answering this question depends on the meaning assigned to “best.”

For higher education rankings, “best” may mean having the most knowledgeable departmental faculty, the most accomplished professors, or the most noteworthy alumni. For students passionately interested in a specific discipline, the best school is likely to be one widely recognized for excellence in that discipline. Other students may put a premium on economic, geographic, and academic imperatives, such as educational outcomes, career prospects, and lifetime earning potential.

Many dimensions of institutional excellence can factor into “best,” and competing ranking systems prioritize these dimensions according to their own internal (and often impenetrable) ranking criteria as well as their detailed (and usually arbitrary) weighting schemes. The criteria are somehow supposed to track academic excellence and the weights are somehow supposed to coordinate the relative importance of these criteria. Yet what emerges is a weird alchemy that transforms marginally relevant data into rankings that the American academic world awaits with bated breath.

Our experience with and research into academic rankings, however, suggests that influence, measured properly, gets at the heart of what is truly best in education while also offering a portal into countless ways of viewing and analyzing the different dimensions of educational value. This is why AcademicInfluence.com allows users to customize and personalize their own rankings based on the factors that matter most in their search for the “best school” (see our Custom College Rankings), while maintaining influence as the master key (especially in the form of Concentrated Influence).

Influence Isn’t Always Pretty

Because our ranking methodology is not just objective but also driven by algorithms and databases, its results emerge from a computational process and therefore not by direct human intervention. This is a strength of our approach, removing the hand-waving and “gameability” that are so common in higher-education ranking approaches. But this absence of direct human intervention can also lead to counterintuitive results. When an influential person or institution ranks highly at AcademicInfluence.com, it’s because our InfluenceRanking engine has picked up on some signal indicating influence. But sometimes it’s not immediately clear what that signal is. Fortunately, we are often able to “look under the hood” of our InfluenceRanking engine and see why it delivered the results that it did.

Our machine-learning approach to academic rankings is necessarily morally neutral. In the case of influential individuals, some exhibit significant impact in a field despite an apparent lack of proper education or socialization or achievement in it. But even in such cases, our InfluenceRanking engine is detecting a real signal (it’s not just noise). For more on this important issue, see our article ”Influence, Infamy, and the Case of Osama bin Laden.”

With our InfluenceRanking engine, the onus is on all to use their common sense and practical wisdom in interpreting its results. Think of the results from our InfluenceRanking engine as a starting point and not as an end point for inquiry. An otherwise influential school may still leave you with an unhappy educational experience. And a widely influential thought leader may engage in thinking that is now archaic. In the end, the users of our InfluenceRanking engine remain the arbiters of the persons and institutions whose influence they must gauge in light of their own system of values. Simply put, influence as presented at AcademicInfluence.com is objective, but its value and relevance to you is not.

In Conclusion

AcademicInfluence.com’s InfluenceRanking engine opens up exciting, new means for students, teachers, and researchers to find answers to what constitutes true value in education.

As our AcademicInfluence.com website has matured and we have continued to publish more rankings of persons and institutions, influential academics across the range of disciplines tell us that they find our rankings remarkable for their accuracy, insights, and serendipitous turns. Some have even reached out to connect with younger thought leaders, whose influence in their field is growing but, without our technology, remained unsuspected.

Such connections thrill us because our mission is to connect learners to leaders, and that includes leaders (who are always also learners) to other leaders. But that mission begins with ensuring that our method for making these connections is sophisticated, scientific, and sound. Our InfluenceRanking engine is by now a mature technology with years of research and effort behind it. It has proven itself. Whatever your educational odyssey, your discoveries at AcademicInfluence.com will enhance it, helping you to realize your academic goals and aspirations.