Turning the Tables: Ranking the MBA Rankings

5. The Economist. This is probably the most flawed, if not downright silly, of all the MBA rankings cranked out by a major media brand. With the publication of its 2010 list, the magazine is now up to its ninth annual ranking of full-time MBA programs. The ranking is based 20% on student and alumni surveys and 80% on data provided by the schools. There are some incredibly peculiar results in this ranking, which raise significant credibility issues. Pretty much no one in business education would agree that Dartmouth or IESE is better than Stanford. The biggest losers in the 2010 survey were all non-U.S. schools, with five of them based in Britain. Oxford University’s Said business school fell 24 places, while Cambridge University’s Judge School of Business dropped 19 spots. The Economist list ranks 100 top schools. The top five schools in the 2010 ranking:

University of Chicago (Booth)
Dartmouth College (Tuck)
University of California—Berkeley (Haas)
Harvard Business School
IESE Business School in Spain

Pro: Takes a global perspective on business school education.

Con: The odd results of this ranking raise meaningful credibility issues with the methodology and the accuracy of the data some of these schools are providing to The Economist. Because 80% of the ranking is based on unaudited information from business schools, there’s a high likelihood that some data has been fudged. The Economist also throws into its ranking formula criteria that has little to do with the quality of education, such as the percentages of international and female students (giving these two questions alone nearly a 17% of the weight in the ranking), the range of overseas exchange programs (a 6.25% weight), and the number of languages offered (also given a 6.25% weight). That latter would be something more appropriate to undergraduate education. Ultimately, the methodology produces some very odd, roller-coaster like results. Consider Vierick Leuven, the Belgium-based business school, which in the 2010 ranking plummeted 37 places in a single year, down to a rank of 47th from a rank of 10 in 2009. In its write-up describing the new ranking, The Economist doesn’t even bother to point out that this school experienced the biggest single drop.

Britain-based Ashridge dropped 32 spots, to a rank of 67 from 35 a year earlier. Obviously, such massive changes in a ranking over only 12 months raises significant questions about the credibility of The Economist’s ranking methodology. How can a school that The Economist says is the 10th best in the world fall to a rank of 47th within a single year? When rankings swings are this dramatic and common it means that the difference between one school’s rank and another has little, if any, statistical significance. The results are so “bunched” close together that it is misleading to rank one school No. 15 and another No. 20 because there is no real quality difference between them. You can’t tell how close these schools are, however, because The Economist does not provide an index to show you the differences among the schools at their ranks.

6. The Wall Street Journal. This prominent business newspaper ranked full-time MBA programs for seven years, but stopped regularly doing an annual ranking with the September, 2007, list, probably because its flawed methodology led to quirky results and significant criticism. We’re ranking it nonetheless because many schools (especially those faring quite well on the list) still include this ranking in their marketing materials. The Journal ranking was conducted by pollster Harris Interactive which surveyed recruiters on 21 different attributes, including students’ leadership potential and strategic thinking, their previous work experience, the faculty and the curriculum, and the career services office.

Pro: It’s simple to understand, measuring only the views of the recruiters who come to campus to hire MBAs.

Con: The Journal got its list of recruiters to survey from the business schools, which provided the newspaper with the names of the actual recruiters, who showed up on campus. Most major MBA recruiters, however, send alums back to their schools to interview candidates. If you merely survey the recruiters who visit a particular school, you’re surveying entirely biased people. Would a Wharton MBA who recruits only Wharton students rate his or her school poorly? Not likely. In fact, what happened is that single companies that recruit MBAs would get numerous votes in the Journal survey—each from the alum of the school they recruit from. The result: this was less a ranking of the best business schools and more a ranking of the loyalty of a school’s graduates who happen to recruit MBAs at their alma maters. Dartmouth’s Tuck School placed first in this survey four times largely because its small graduating classes and intimate educational setting make its alums more loyal than those from Harvard, which shockingly ranked 14^th, or Stanford, which just as shockingly ranked 19^th. In the years that the Journal did this study, it would tell readers that it surveyed more than 2,000 recruiters. Problem is, there’s less than 300 companies in the world that actively recruit MBAs from enough schools to make a legitimate judgment about the quality of the institutions and the MBAs they produce.

BusinessWeek, in contrast, surveys far fewer corporate recruiters because it spends months finding out who are in charge of MBA recruitment and development at all the companies, which make the market for these graduates. Only then does BusinessWeek survey this group, getting the opinions of the top executive for MBA hiring in each organization—not alums that revisit their schools to hire students. Thank goodness the Journal had the sense to drop out of the ranking game because this one was so flawed it lacked all credibility. The top five in the last full-time MBA ranking by the Wall Street Journal: