Nine Biggest Mistakes In MBA Rankings

Author Matt Turner

Recruiter Surveys:

These pose a variety of problems. Companies have differing hierarchies (national vs regional recruiters) with differing levels of autonomy, so it is difficult to know if the right people are being targeted for the survey. What a senior HR person might say about a school at the national level could vary radically from what a local recruiter would say about the same school. What recruiters have to say about students may reflect more a given year’s hiring quota than the quality of the students overall. Media usually keep schools in the dark about the details and the results are often quite at odds with what recruiters report to the schools about their own satisfaction with a school and its graduates.

Failure to Audit Data:

Only the Financial Times officially audits MBA ranking data. All other rankings assume the data is good and clean. Rarely do media outlets even spot-check data against what schools publish on their own websites (which sometimes differs radically).

Failure to Gather Data Consistently:

Sometimes ranking outlets will attempt to gather data from many schools, but when those schools decline to participate, they gather the data anyway from “various public sources” such as websites or by emailing enrolled students. This inevitably leads to inconsistent data.

Failure to Check Volatility:

Academic institutions tend to change slowly. Barring a catastrophe, it is difficult to believe that any school could rise or drop 10 ranks in a single year on any comprehensive ranking, and yet this happens all the time because of poor methodology. Older, more established rankings have learned to check volatility somewhat by conducting biennial (rather than annual) rankings or by weighting past surveys in the current-year rank, or both, but many rankings fail to do either, which produces roller-coaster effects.

Self-Promotion:

Several surveys contain criteria which are tied to outcomes from their own surveys, creating a self-fulfilling prophecy of sorts. For instance, a school will get extra points if their grads take up another degree or a faculty position at a top-ranked business school, “top-ranked” being defined by the media outlet’s own ranking of top schools.

Matt Turner covers business school rankings for McCombs Today published by the McCombs School of Business at The University of Texas at Austin.