Are Rankings Meaningless? This Expert Offers His Thoughts

Rankings often play a big role in where people decide to pursue their MBA degree. A 2021 study by the Graduate Management Admission Council (GMAC) found that roughly 45% of domestic MBA applicants and 49% of international MBA applicants said they rely on rankings as much as they did on school-related websites.

But one expert says that maybe it’s time to rethink how we view MBA rankings. Robert Zafft, an adjunct lecturer at Olin Business School, recently discussed why rankings, such as the one by U.S. News, need to be closely examined.

RANKINGS SCANDAL

Business schools, for the most part, see rankings as very important.

“Rankings bring notoriety, prestige, and revenue to the organizations that produce the rankings,” Zafft writes. “Rankings also drive the careers of people who run governmental, for-profit, and educational institutions. This in turn drives institutional behavior.”

The lure of rankings has, in turn, resulted in some big scandals—the most recent one being at Temple University, where the former B-school dean Moshe Porat was indicted on fraud charges in an MBA rankings scandal.

U.S. NEWS RANKING METHODOLOGY

Zafft offered a close look into US. News’ ranking methodology highlighting areas that raise legitimate questions.

For one, academic quality (which makes up 40% of the U.S. News ranking) relies on subjective data from MBA Deans and Directors (25%) survey responses and corporate recruiters.

“The survey asked respondents to rate the MBA-program-quality of other schools on a scale of 1-5,” Zafft writes. “Respondents may also state they lack sufficient familiarity with a program to answer. U.S. News takes the average of survey scores from Deans/Directors and a weighted average score for recruiter-submitted surveys over the past three years.”

Surveys, Zafft says, tend to turn subjective unreliable opinion into quantified data.

“MBA-school deans and directors might be assumed to keep a watchful eye on their nearest competitors,” he writes. “But will such people evaluate these competitors fairly? And how far out does respondents’ knowledge reliably extend?”

WORTHY ALTERNATIVES

Rather than rely on surveys, Zaffit notes, U.S. News should consider ranking schools’ academic quality based on factors used for faculty promotion and recruitment such as publishing and citation numbers—data that schools themselves use for faculty promotion and recruitment.

“B-school websites typically provide faculty CVs, which include publishing and employment histories,” Zafft writes. “Publishing success and job movement can be fed into an algorithm, which will be consistent and can be adjusted over time after open discussion.”

At the end of the day, people value rankings because they shape a business school’s brand perception. But that, Zafft says, is the exact reason why we need to take them with a grain of salt.

“In a rankings-mad world, we need to look hard at the people and the methodologies doing the ranking,” Zafft writes. “A quick review of U.S. News’ approach to B-Schools raises legitimate questions. With regard to rankings of any kind, we owe it to ourselves to dig deeper and to look harder. Otherwise, we risk making crucial decisions based on meaningless data.”

Sources: Forbes, P&Q, P&Q

Next Page: University of San Diego Lands Game-Changing Donation.