BusinessWeek’s Big Oops Ranking Moment

Arguably the most famous of all the cover stories published by BusinessWeek magazine carried the simple yet highly provocative headline: Oops! The story chronicled how many of the companies extolled for their excellence had badly stumbled.

Now Bloomberg BusinessWeek has its own “Oops” to contend with, a mistake that significantly impacted its MBA ranking of last year. The errors in the magazine’s calculations change the overall numerical rank of a dozen schools–10 in the U.S. and two non-U.S. schools. More significantly, the “intellectual capital” rank of some schools moved by as many as 10 positions. Harvard Business School, for example, fell to 19th from 9th in intellectual capital as a result of the revision.

This time, Bloomberg BusinessWeek was not so eager to publicized its own failings. The magazine quietly revised its rankings online a month ago with little fanfare or notice. At the top of its corrected table, in small hard-to-read type is this rather vague, less-than-clear statement: “(Corrects to revise 2012 overall rankings, Ranking index and 2012 Intellectual Capital rankings following errors in the calculation of the Intellectual Capital score.)”

Even BusinessWeek’s updated methodology does not acknowledge the mistake. Instead, the magazine simply crosses out part of the description of the methodology (see below).

BUSINESSWEEK ADMITS IT CHANGED METHODOLOGY WITHOUT TELLING ITS READERS OR THE BUSINESS SCHOOLS

Even worse, the magazine had to admit that it had changed its methodology for ranking the intellectual capital produced by business schools without official notice or even a mention in an accompanying story describing the way it tabulates the results of academic research.

In an email obtained by Poets&Quants, Bloomberg BusinessWeek Assistant Managing Editor Janet Paskin informed the schools last month that “we have discovered that some errors were made in calculating one element of the 2012 ranking, the intellectual capital score, which accounts for 10 percent of each school’s total score.”

Paskin went on to describe the problem in more detail that the magazine has still failed to disclose to its readers. “In a recent review of last year’s intellectual capital rankings,” she wrote, “we discovered errors in how we collected and tallied faculty research. We also realized that we had failed to update our stated methodology to reflect our current practice: Faculty are no longer awarded points for reviews of books they have authored, and the time period surveyed spans four years, not five.” In response to a question on how the errors occurred, Paskin said in an email that “inn a few instances, changes in faculty names were missed; some articles were published online but not in print, and vice versa, and were missed.”

THE MISTAKES RESULTED IN SOME DRAMATIC CHANGES IN THE MAGAZINE’S INTELLECTUAL CAPITAL RANKINGS

The screwup by BusinessWeek resulted in some especially dramatic changes in the intellectual capital ranking given to some schools. Yale University’s School of Management, for example, zoomed up eight places to 9th from an inaccurate ranking of 17th. The University of Chicago’s Booth School of Business plunged to 11th from fifth. Harvard Business School dropped 10 places to 19th from ninth. Rice University’s Jones School climbed to 21st from 27th. And Notre Dame moved up six places to 31st from 37th.

All told, BusinessWeek admitted that “miscalculated intellectual capital scores” were made by BusinessWeek for 40 American business schools and 10 international schools–a fact that was buried deep inside the email sent to the schools. The detail regarding the significant drops in intellectual capital rank was in the eighth paragraph of the ten-paragraph email written by Paskin. In journalism school, they call that “burying the lead.”

Paskin’s email, moreover, contains a link to what she refers to as the corrected rankings. However, BusinessWeek makes no mention of its mistake or how its own errors in calculating the ranking resulted in significant changes on the table.