When MBA Rankings Lag The Facts

ranking

You can’t put a price on a good reputation. That’s true in business…and even more so in academia. Take business schools, for example. Prospective employers and partners often make assumptions based on where you graduated. List an MBA from a top tier and doors magically open. Fellow alums give you the secret handshake. Your superiors bother to learn your name.  And your peers listen more intently when you speak (at least at first).

A business school’s reputation carries that kind of weight. Question is, how much of that reputation is based on quantitative data over carefully-calibrated branding?

U.S. News and World Report recently attempted to answer that question for undergraduate programs in their 2014 Business School Rankings. As the proverbial “gold standard” for postsecondary rankings, U.S. News conducted its second annual analysis of schools whose rank doesn’t match its reputation. Using a methodology developed by Bob Morse, the director of research at U.S. News, undergraduate programs were broken into two categories: “overperformers” and “underperformers.”

WHICH SCHOOLS UNDERPERFORM THEIR RANKINGS?

Here’s how it worked: U.S. News bases its undergraduate rankings on factors like “admissions selectivity, financial and faculty resources, graduation and retention rates, alumni giving, and graduation rate performance.” Each of these indicators is given a particular weight in a formula used to calculate a school’s overall rank. However, U.S. News also factors in “peer assessments,” where administrators and tenured faculty rate other schools using a 5 point scale, with 1 being “marginal” and 5 being “outstanding.”

After ranking schools from highest to lowest on peer assessment, U.S. News subtracts a school’s overall rank from its peer assessment rank. A positive number indicates that a school is an “over performer,” while a negative number reflects an “underperformer.” In Morse’s words, an overperforming school has a “reputation among its academic peers [that] has not kept pace with what it has achieved in the underlying academic indicators. This could be because academic reputation is a lagging indicator – it can take time for a school’s academic peers to understand the real progress of a university.” Conversely, an “underperforming” school’s ranking may be artificially inflated based on the subjective peer assessments of the academics who were polled.

Impressed with this formula, we wondered what it might reveal about MBA programs. In the U.S. News 2014 Business School Rankings, peer assessment scores were derived from “business school deans and directors of accredited master’s programs in business.” According to U.S. News, 43% of the deans and directors surveyed responded, with these peer assessments accounting for 25% of each school’s weighted ranking. As a result, a fourth of a business program’s ranking is based on the opinions of experts who may not necessarily be basing their ratings on empirical data.

A SCHOOL WITH A RANK HIGHER THAN ITS REPUTATION IS BEING UNDERSOLD

Of course, to agree with the premise you have to buy into the notion that the remaining part of U.S. News’ methodology accurately measures the relative rank of the full-time MBA program it purports to measure. Besides the peer assessment portion, the rest of the rank is pretty much based on quality factors: average GMAT and GPA, acceptance rate for applicants, average starting salary and bonus, percentage of class with jobs at graduation and three months later, and an opinion poll of corporate recruiters.

If a school’s core stats award it a rank above the rank given to the school by rival deans and MBA directors, it suggests the school hasn’t done a very good job of tooting its own horn and letting others know that its base quality exceeds its reputation. If a school’s stats show that it is below a rank accorded to the school by peers, it suggests the school has a reputation that is higher than deserved. The school should be working hard to improve its underlying quality measures.

(See following pages for full analysis and tables on 2014 Business School Rankings)