When MBA Rankings Lag The Facts

ranking

You can’t put a price on a good reputation. That’s true in business…and even more so in academia. Take business schools, for example. Prospective employers and partners often make assumptions based on where you graduated. List an MBA from a top tier and doors magically open. Fellow alums give you the secret handshake. Your superiors bother to learn your name.  And your peers listen more intently when you speak (at least at first).

A business school’s reputation carries that kind of weight. Question is, how much of that reputation is based on quantitative data over carefully-calibrated branding?

U.S. News and World Report recently attempted to answer that question for undergraduate programs. As the proverbial “gold standard” for postsecondary rankings, U.S. News conducted its second annual analysis of schools whose rank doesn’t match its reputation. Using a methodology developed by Bob Morse, the director of research at U.S. News, undergraduate programs were broken into two categories: “overperformers” and “underperformers.”

WHICH SCHOOLS UNDERPERFORM THEIR RANKINGS?

Here’s how it worked: U.S. News bases its undergraduate rankings on factors like “admissions selectivity, financial and faculty resources, graduation and retention rates, alumni giving, and graduation rate performance.” Each of these indicators is given a particular weight in a formula used to calculate a school’s overall rank. However, U.S. News also factors in “peer assessments,” where administrators and tenured faculty rate other schools using a 5 point scale, with 1 being “marginal” and 5 being “outstanding.”

After ranking schools from highest to lowest on peer assessment, U.S. News subtracts a school’s overall rank from its peer assessment rank. A positive number indicates that a school is an “over performer,” while a negative number reflects an “underperformer.” In Morse’s words, an overperforming school has a “reputation among its academic peers [that] has not kept pace with what it has achieved in the underlying academic indicators. This could be because academic reputation is a lagging indicator – it can take time for a school’s academic peers to understand the real progress of a university.” Conversely, an “underperforming” school’s ranking may be artificially inflated based on the subjective peer assessments of the academics who were polled.

Impressed with this formula, we wondered what it might reveal about MBA programs. In the 2014 U.S. News Business School Rankings, peer assessment scores were derived from “business school deans and directors of accredited master’s programs in business.” According to U.S. News, 43% of the deans and directors surveyed responded, with these peer assessments accounting for 25% of each school’s weighted ranking. As a result, a fourth of a business program’s ranking is based on the opinions of experts who may not necessarily be basing their ratings on empirical data.

A SCHOOL WITH A RANK HIGHER THAN ITS REPUTATION IS BEING UNDERSOLD

Of course, to agree with the premise you have to buy into the notion that the remaining part of U.S. News’ methodology accurately measures the relative rank of the full-time MBA program it purports to measure. Besides the peer assessment portion, the rest of the rank is pretty much based on quality factors: average GMAT and GPA, acceptance rate for applicants, average starting salary and bonus, percentage of class with jobs at graduation and three months later, and an opinion poll of corporate recruiters.

If a school’s core stats award it a rank above the rank given to the school by rival deans and MBA directors, it suggests the school hasn’t done a very good job of tooting its own horn and letting others know that its base quality exceeds its reputation. If a school’s stats show that it is below a rank accorded to the school by peers, it suggests the school has a reputation that is higher than deserved. The school should be working hard to improve its underlying quality measures.

(See following pages for full analysis and tables)

  • bofdem

    who are “most people” ? Sounds like you need to go back to 5th grade, son.

  • bofdem

    As an anonymous poster, I expect this continued ignorance. The brand is the business school name – Wharton vs. Smeal. Big difference. Now go back to your job at McDonald’s.

  • Jenny

    I did my MBA at one of those underachiever MBA programs and I thank you for this piece ! I also appreciate John’s annual article re. MBA programs with the worst job placement, usually done after the annual US News MBA ranking. What this does is highlight certain programs that cannot get their act together, rather they fraud students with dubious achievements and often misleading information. Refereeing this grand arena of quality and dubious programs is no easy task given the constant manipulations of ranking data submitted and job placement stats. What it does is allow students and alumni to present the information in a concise manner to said administration or share amongst without being singled out as a malcontent. My program likes to shoot the messenger of reality and Poet & Quants is credible and concise albeit too often focused on Harvard vs Wharton etc.

  • Matt

    Marcia, I was not talking about University of Pennsylvania..

  • public

    Well, it sounds like a public school. Big f****n deal! Who cares if it’s Ivy League. It’s not at the level of Harvard, Yale or Princeton. Even Columbia is better. Most people don’t know that Penn is part of the Ivy League anyway and associate it with PennState. Perception is reality, like it or not.

  • Marcia

    University of Pennsylvania is NOT a public school — the public university is Penn StateState. UPenn is private and a member of the Ivy League.