Most Reliable & Valid MBA Ranking?

BUSINESSWEEK’S STUDENT SATISFACTION POLL TENDS TO PRODUCE MORE VARIABLE RESULTS

The student poll in BusinessWeek’s methodology accounts for 45% of its ranking and is, in many ways, the secret sauce of the BW ranking (see The Questions BusinesWeek Asks MBAs). Another 45% weight goes to a survey of corporate recruiters, while the remaining 10% of the methodology is attributable to a BusinessWeek compilation of articles written by professors. Though the recruiter and intellectual capital portions of the methodology remain relatively consistent, it’s the student satisfaction poll that often results in the more significant changes in overall rank for a school in the BusinessWeek survey. Says Iacobbuci, “If it changes every time you give it, then you are hard pressed there is anything in particular meaningful. Schools just don’t change that much from time to time. That is for better or worse. If you have a school that is middling and they do something innovative, they might not get recognized in a ranking. It is just lumpy data. It’s not reflecting the constancy of the quality of the school.”

The consistency in the U.S. News rankings are largely based on average GMAT and GPA scores as well as starting salary , which rarely shows dramatic change from year to year. Nor do the opinions of business school deans or recruiters, who also are surveyed by U.S. News. According to the professor, that makes the U.S. News survey harder for schools to manipulate–unless, of course, they submit inflated data to the publication as Tulane University’s A.B. Freeman School did in recent years.

‘IT’S EASIER TO GAME BUSINESSWEEK AND IT’S SURPRISING THAT SCHOOLS HAVEN’T GAMED THEM MORE’

“Of the three, it’s easier to game BusinessWeek,” says Iacobbuci.  “It’s surprising that schools haven’t gamed them more, only because it’s subjective. A GMAT score is a GMAT score and it’s hard to imagine playing with that. If I were an MBA student I would rank my school as highly as I could. You pop that into a regression and it’s kind of useless. They don’t and they haven’t.”

From her study, to be published in a forthcoming issue of the Journal of Marketing Education, she also concluded that many top-ranked business schools have inherent advantages that would be hard for other schools to surmount when it comes to the rankings game. For example, wealthier schools, described as those that charge more, spend more, and sit on larger nest eggs, tend to rank more highly on the lists. Yet, Iacobbuci adds, “the correlations are not so high as to suggest that one’s standing is forever determined or that change is too daunting to undertake if one does not have access to the bounty of resources of another school.  That is, the correlations do not altogether indicate that ‘to the rich, go the spoils.’”

And if a business school is in a metropolis which happens to be home to the headquarters of Fortune 500 companies, boasts large graduating classes and is part of a university with a good basketball team, it’s also likely to rank more highly, according to Iacobbuci.

“Better business schools tended to be in or near a larger city and metropolis,” she says. “Yet once again, while the correlations are significant, they are not perfect, so schools that are not so situated can still provide excellence for their students.” Larger MBA programs tend to be ranked more highly, she discovered, perhaps due to various scales of economies, “companies desirous of efficient yields recruiting only at campuses large enough to warrant their attention. And U.S. Schools situated in or near cities with more Fortune 500 headquarters are ranked better. “The proximity probably affords certain benefits—perhaps real jobs or simply salient perceptions of neighbors,” she says.

MBA RANKINGS ARE BETTER AT SCHOOLS WITH BETTER BASKETBALL TEAMS

Other interesting observations from her study:

  • Top business schools attract top students: all three rankings are significantly correlated with GMAT scores and incoming GPAs.
  • Top business schools produce commercially successful graduates: all three rankings are strongly correlated with starting salaries.
  • Top business schools feature top faculty as measured by their academic publications records.
  • Whether due to shared resources or halo judgments, MBA rankings are significantly correlated with undergraduate rankings. (Still, the correlations are not perfect—there are good universities with less than stellar business schools, and good business schools at so-so universities.)
  • The distinction between private and public universities is unrelated to all three polls—there are good (and so-so) public MBA programs and good (and so-so) private MBA programs.
  • MBA rankings are better at schools with better basketball teams (no relationships held for football or soccer), throwing a dash of skepticism onto the validity of the rankings. 

The author of the study also gives credence to PoetsandQuants’ methodology of combining the findings of major rankings in a composite index. As she points out in her paper, “Between the three (lists), the rankings seem to cover the entirety of a throughput model of students in business school–for example, the incoming student quality is captured by U.S. News via GMAT scores, students’ satisfaction with their school experience by BusinessWeek, and students’ resulting jobs and salaries are captured by U.S. News and the FT. The heterogeneity across the three rankings covers the full model–the answer to the age-old question as to whether students are raw material inputs, or students are customers, or students are products seem to be: students are all three.”