A Challenge To Businessweek’s European & Asian MBA Rankings

Only a month ago, Bloomberg Businessweek crowned IMD in Switzerland for having the best MBA program in Europe and CEIBS in Shanghai for best in Asia.

Not so fast.

It now turns out that the same challenge to the magazine’s ranking of U.S. business schools may well impact its European and Asian MBA rankings. An analysis of the published index scores behind those rankings by Yale School of Management Deputy Dean Anjani Jain finds a different outcome.

IESE BUSINESS SCHOOL APPEARS TO BE THE WINNER IN EUROPE; SHANGHAI UNIVERSITY THE WINNER IN ASIA

IESE Business School in Barcelona, Spain, would have been first in Europe, while Shanghai University of Finance and Economics would take top honors in Asia if the weighting Businessweek assigns to each of four indexes were applied according to Businessweek’s stated methodology.

Instead, the only way to come up with the same ranking would be the weighting given to each of the four components were changed, in some cases dramatically. The compensation category would have to count for 62.1% of the ranking instead of the 36.5% that Businessweek claims to have applied. The “learning” index would fall in importance to a mere 9.8% from Businessweek’s 25.3% weight. Networking would drop to 13.4% from 23.8%, while entrepreneurship would decline to 9.7% from 14.4% (see table below).

Among the European schools, in fact, Jain found that 19 MBA programs out of 23 ranked by Businessweek would have changed positions. Most of the changes are minor compared to the big swings in the U.S. business schools. IMD would fall to second place from first, for example, while INSEAD would increase its ranking to third from fourth. The largest single difference impacts Cranfield Business School in the United Kingdom which would improve by five positions to rank 13th instead of 18th.

BLOOMBERG BUSINESSWEEK EDITOR: 'WE STAND BY OUR RANKINGS AND OUR METHODOLOGY'

Among the seven Asian MBA programs ranked by Businessweek, every school would have a different rank. The Indian School of Business, placed fourth by Businessweek, would have a true ranking of second place if the weights were applied to the index scores as described by the methodology. ISB would replace Hong Kong University of Science and Technology which would drop five places to seventh (see table below).

The Bloomberg Businessweek editor in charge of the rankings, Caleb Solomon, does not agree with Jain, an expert in optimization algorithms and the analysis of combinatorial problems. “We can’t disclose all aspects of our methodology beyond what we’ve already published,” maintained Solomon in a Oct. 7th email to Jain. “As I’m sure you can appreciate, we invest a great deal of time and effort into creating a proprietary ranking that can be neither replicated nor gamed. As I said before, we stand by our rankings and our methodology.”

Responds Jain, “It is not the case that some secret element of the methodology successfully thwarted my attempts to replicate or game the published ranking. The problem, as I describe in the P&Q article, is that the ranking cannot be replicated by using BBW’s methodology: i.e., by applying the published weights to the published raw data. The only way to replicate the published ranking is by greatly distorting the weights, which makes a lie of the methodology.”

THE ONLY WAY TO REPLICATE THE RANKING IS TO APPLY VASTLY DIFFERENT WEIGHTS FOR THE FOUR RANKING COMPONENTS

The analysis by Jain, who oversees academic affairs at Yale SOM, found that Bloomberg Businessweek’s published ranking of both the European and Asian schools could not be replicated by applying their weights to the four indexes that make up the overall score for each school. "Using BBW’s published index scores and index weights produces a ranking very different from the one that BBW has published," says Jain.

"This replication crisis of the BBW ranking remains unmitigated whether one uses the ‘normalized scores’ provided by BBW for the five indexes, each scaled from 0 to 100, or the standardized ‘z-scores’ computed from the ‘normalized’ scores. The rankings that result by applying BBW’s index weights either to the published ‘normalized’ scores, or to the z-scores computed from the published scores, are egregiously off-kilter when compared to the published ranking."

The only way to replicate the published ranking, adds Jain, is to apply index weights that are vastly different from the weights that BBW claims to have used. He computed these true weights by using a constrained optimization model that minimizes variances from the published ranking. When assigned to BBW’s ‘normalized scores’, the true weights replicate very closely the overall scores published by BBW for the European and Asian schools, and thus its published ranking (Jain's full analysis is on the following page).