Are Academics Out Of Step With Recruiters & Students?

Grade inflation isn’t just for students anymore. Turns out, academics give their fellow MBA programs high marks, too. Question is, are those grades based on rigorous examination — or just slipshod opinion?

That’s the question at the heart of the annual U.S. News & World Report Best Business School Rankings, which uses academic opinion as its foundation. In theory, business academics may appear well positioned to evaluate their peers. Unlike their liberal arts kin, who are traditionally lampooned as insulated fusspots, business profs are often regarded as thought leaders who help shape their field.

Just because they’re attuned to the macro trends of industry, however, doesn’t necessarily translate to being literate in the micro developments at individual business schools.

EVALUATING SCHOOLS BASED ON LIMITED EXPOSURE

It’s no secret: “Peer” assessments can make or break a business school’s annual U.S. News ranking, the most closely followed ranking of U.S. MBA programs. These assessments account for 25% of a ranking, but the process is shrouded in mystery. According to U.S. News’ methodology, business school deans and directors of accredited MBA programs score schools using a scale from 1 (Marginal) to 5 (Outstanding). As a caveat, U.S. News notes that respondents — the 41% who actually participated — are told to just mark “Don’t Know” for schools where they have limited exposure.

You can imagine the issues this creates.

For one, the actual size of the sample is never disclosed by U.S. News, let alone the number of evaluations a school received. That means readers have no idea just how much weight each assessment carried — particularly considering U.S. News’ narrow five-point system, which could be a boon or albatross for schools with lower response rates. More importantly, the scoring is vague. Are respondents appraising academic acumen, student quality, or outputs like salary and placement — or some mixture that may not be consistent from one respondent to the next?

Above all, the survey’s simplistic requirements rouse concerns about the respondents’ actual exposure to schools where they neither work nor (in all likelihood) retain a regular presence. As a result, the results risk being tainted by chatter or bias — or, most likely, the year-prior rankings.

ACADEMICS GAVE A 0.2-0.3 AVERAGE POINT ADVANTAGE TO NEARLY TWO-THIRDS OF TOP 25 SCHOOLS   

This distance between evaluator and subject may be one reason for the gap between peer and recruiter assessments with U.S. News. Both rely on the same five-point scale. However, their school averages are noticeably different. Take U.S. News’ Top 10 MBA programs in the 2017 rankings. In seven out of 10 schools, two- or three-tenths of a point separates peer and recruiter scores. Notably, academics gave Stanford a 4.8 average, tying the GSB with Harvard Business School for the top score. However, recruiter scores for Stanford and Harvard averaged 4.5, the same scores as Kellogg, Booth, Sloan, and Wharton. A similar result occurs with UC-Berkeley’s Haas School of Business (4.6 peer versus 4.3 recruiter) and the Columbia Business School (4.4 versus 4.1). Not to mention, academics scored four top 10 programs —Harvard, Sloan, Wharton, and Booth —at a 0.2 point clip higher than recruiters.

It doesn’t just stop with the elite schools. Among U.S. News’ top 25 schools overall, there were 12 schools where academics gave a 0.2-point average advantage over recruiter scores, with another four schools sporting a 0.3-point gain. In this segment, just three schools came away with the same average in both categories: Yale (4.3), Cornell (4.1), and Notre Dame (3.5).

Big picture, these disparities are even more pronounced. Academics confer higher scores, as a whole, by a 4.32-to-4.20 margin among top 25 schools. However, there were exceptions to academics rewarding perceived top schools with higher scores — notably among schools ranked below 15th. They included Texas and Georgetown (both 0.2 of a point better with recruiters) and Emory, Vanderbilt, and Rice (+0.1 among recruiters).

(Go to next page to see peer assessments from 2006-2017)