A Furor Erupts Over BusinessWeek Ranking

‘I CAN’T BE 100% SPECIFIC’

But exactly how it awards points to schools from those surveys is still unknown. As Rodkin recently told school administrators during the online chat, “Since our evaluation is ongoing, I will be unable to state with full specificity how every aspect of the employer survey will work this year.”

At another point in the meeting, Rodkin told administrators: “I can’t be 100% specific about how each person’s response will be weighted, but all people will be invited to participate. There is a chance we will weight responses by the number of MBA hires overall (so, for example, a company that hired 100 MBAs and for whom we had 10 contacts, would have equal weight in the final analysis as a company that hired 100 MBAs and for whom we had 20 contacts). but I cannot confirm that that will be our practice at this time.”

And at another segment of the discussion, he noted: “Our analytic methods for intellectual capital are still under review.”

A SUBTLE THOUGH MEANINGFUL CHANGE IN HOW IT WILL SURVEY EMPLOYERS

Rodkin has told the business schools that there will be one change for certain. Instead of ensuring that the recruiter survey goes to the person in charge of MBA recruiting at a company or major division that hires business school graduates, Bloomberg BusinessWeek will be surveying multiple recruiters at each corporate employer—the same methodology adopted by U.S. News & World Report for its survey of recruiters and used by The Wall Street Journal in the past. The magazine has said it will survey every recruiter contact given to it by a business school.

The change, though subtle, is significant. Most mainstream employers of MBAs send alumni back to the schools to recruit for them. So a person who graduated with an MBA from Dartmouth College’s Tuck School will more often than not return to campus to recruit Tuck candidates for his or her company. Surveying those on-the-campus recruiters will increase the odds of a biased response. No less crucial, those recruiters are less likely to have a basis for comparison across the schools or knowledge of the performance of MBAs from different schools in their company’s workforce. Rodkin has told the schools that the change will increase the size of the survey sample “in order to achieve more valid and reliable results.”

Some school officials don’t see it that way. “By allowing schools to list their own MBA alumni as the respondents for the employer survey, the BusinessWeek survey has lost its last shred of legitimacy,” said one business school representative. “What BusinessWeek will present to students as an unbiased employer assessment of a given MBA program is, in fact, nothing more than the ranking of an MBA program by its own graduates who, by the way, have every incentive to improve the ranking of their alma mater. In reality, what schools do is pick their most supportive alumni and list them as the employer contact to be surveyed. It’s a farce.”

THE WALL STREET JOURNAL ABANDONED A SIMILAR METHODOLOGY YEARS AGO

When The Wall Street Journal used to rank business schools, it did so on the basis of a similar survey to corporate recruiters, using the same methodology the BusinessWeek business school ranking has now adopted. The Journal was able to get far more responses–4,430 recruiters responded in 2007, the last year the newspaper ranked schools, versus 206 responses from recruiters in BusinessWeek’s 2012 survey. But the results showed that the Journal’s list tended to be more a measurement of alumni loyalty than one that captured the quality of the MBA experience. Dartmouth Tuck, which is known to have one of the tightest MBA alumni networks in the world, was first.

In the past, BusinessWeek held back certain details of its methodology so that schools would have more difficulty gaming the ranking. This time, however, Rodkin says Bloomberg BusinessWeek is committed to a more transparent approach. “Over the last several months,” he told Poets&Quants, “we have had many constructive conversations with schools about our rankings methodology. We are proud to be providing an unprecedented level of detail about our methods to schools.”

Some officials give high marks to Rodkin for his openness in dealing with the school. Joe Fox, associate dean and director of MBA programs at Washington University’s Olin School of Business, applauds the new approach. Unwittingly, however, it has also opened up Bloomberg BusinessWeek to much lobbying, particularly from schools with smaller MBA programs who believe the survey design is biased against institutions that graduate fewer MBAs.

‘FEEDBACK FROM BUSINESS SCHOOLS DIRECTLY INFORMED THE SMALL CHANGES WE HAVE MADE’

The dialogue the magazine is having with business schools has already resulted in several changes, according to Rodkin. “Feedback from business schools directly informed the small changes we have made to this year’s employer survey,” he said in an email response to questions from Poets&Quants. “To best measure employer opinion of MBA programs, we are including a larger number of respondents in this year’s survey. We asked schools to provide as much employer information as they wish, rather than limiting their list to 30 recruiting contacts as in previous surveys. Because professionals outside of the HR department often do MBA recruiting, we allowed schools to list up to three contacts per employer.

“In the past, Bloomberg Businessweek and Cambria Consulting contacted employers to determine the most qualified survey respondent at each employer, since it was common for different contacts to be listed by different schools per employer. But schools were not told how we selected whom to contact, nor which of their 30 listed contacts were asked to participate. That made it difficult for schools to trust and understand the employer survey findings. As a result, we designed the 2014 employer survey without any component that limits whom we contact.”

DON’T MISS: MBA RANKINGS & BUSINESS SCHOOLS: A LOVE-HATE RELATIONSHIP or CONFESSIONS OF AN MBA RANKINGS GURU