9) The Weakest Ranking Links: Peers & Recruiters

The worst part of U.S. News ranking are the two surveys done to muster up the opinions of peers and recruiters. Together, those two surveys account for 25% of the ranking, exactly 12.5% for recruiters and for deans and directors, most often senior faculty.

At first blush, it makes sense to gather the perspective of top business school leaders and the people who actually recruit their students. But you don’t have to dig very deeply to determine that the survey results are garbage. Why? Let’s take the peer survey first. U.S. News asks business school officials to rate the academic quality of every single AACSB-accredited MBA program on a scale of one to five, with one being “marginal” and five representing “outstanding.” Some 58.9% of the that submitted their stats responded.

One issue is that most deans and directors may know little more than a handful of schools well enough to render an honest opinion. More often than not, respondents simply take out the previous year’s ranking and use it as a guide to complete the survey. Even worse, the surveys are filled out by public relations and marketing staff eager to promote their own MBA programs.

Listen to one dean who asked to remain anonymous in an interview with Poets&Quants: “What happens in most schools is that the dean forwards it to the marketing person who fills it out. What dean is going to sit there and rate programs one through five for 150 schools? What do I know about the University of Maryland at Hagerstow? That seems to be a head scratcher in my view.”

Even worse, U.S. News relies on merely this peer assessment survey for its 14 specialty rankings, including lists ranking Executive MBA programs and disciplines such as accounting and finance. The editors allow schools with as few as seven votes to be included in these rankings.

“But if you have such a low participation rate, five deans voting for one school can skew the results,” a dean points out. “And that counts for 12.5% in the overall ranking. If you are in the Top 20, it won’t make a difference. But you have five schools with an index score of 49. So if you get three schools giving you a five on that rating, it can bump your overall score by two points and that can represent 10 to 15 positions on the ranking.”

Then, there is the recruiter survey, with the same rating scale. U.S. News doesn’t bother to report a response rate for this questionnaire, most probably because it is embarrassingly low. After all, would anyone at McKinsey, Goldman Sachs, or Google bother to fill out a U.S. News survey? As U.S. News itself admits, all a school needs is rating from 10 cumulative recruiters over the past three years. If not, U.S. News wily-lily assigns a value to a school by matching the lowest average score achieved by any ranked program with at least 10 ratings.

To find recruiters to survey, U.S. News asks for names and email addresses of those who show up on the campus to recruit students. In most cases, however, companies send alumni back to their alma mater. Most of them know just as little as the deans about how one MBA program truly compares to another. In many cases, a company staffer will only visit one or two schools, anyway. And, of course, if the person completes the survey, you can expect that person to more highly rate their own alma mater. U.S. News does no upfront diligence to get to a human resources official who oversees MBA recruiting for their organization and tracks the success of those hires at their companies.

As some accountants are want to say, garbage in, garbage out.

10) Roller-Coaster Rides Up & Down The Rankings Mountain

Every year, among the biggest surprises in this or any other MBA ranking are the schools that climb or tumble in double-digits, even though there has been little to no year-over-year change in the curriculum or the program overall. These ranking shifts are most typically caused because the differences among the schools are so small that they are statistically insignificant. Schools tend to cluster in groups that make numerical ranks on the programs highly questionable and intellectually dishonest.

This year, 27 of the MBA programs ranked in the Top 100 by U.S. News experienced double-digit changes in their rankings. Most surprising, however, was that the vast majority of them went up! Over all the ranked programs by U.S. News, in fact, 23 climbed in double-digits. Ten of those full-time MBA programs climbed by 20 or more places, with the University of Pittsburgh making the biggest leap, 39 spots up, to the University of California at Riverside, up 20 places to rank 90th Here’s the list of double-digit gainers.

DON’T MISS: STANFORD & WHARTON TIE FOR FIRST IN U.S. NEWS 2024 MBA RANKING