U.S. News Rankings: What Are Deans Really Assessing?

rankings

“The more things change, the more they stay the same.”

Philosopher Yogi Berra referred to this phenomenon as “déjà vu all over again.” Optimists describe it as the circle of life. And cynics call it “the system” (and argue it is stacked against us). Indeed, the status quo protects a lot of vested interests. And that’s especially true with the U.S. News & World Report business school rankings.

SAME SCHOOLS TOP U.S. RANKINGS YEAR-AFTER-YEAR

Picture this: In the 2010 rankings, the top five schools were (in order): Harvard, Stanford, MIT Sloan, Northwestern Kellogg, Chicago Booth, and Wharton (with Booth and Wharton tied for fifth). In 2016, the rankings stack up as follows Stanford, Harvard, Wharton, Chicago Booth, and MIT Sloan (with Northwestern placing sixth). In fact, if you go back to the 2003 rankings, you won’t find any schools besides these six in the top five (though Columbia, Dartmouth Tuck, and Berkeley Haas have each done a stint in the sixth spot ahead of Booth).

Think about leading companies or sports teams. Over a decade or more, how often do you find the same six holding their spots near the top? Rarely (in a free market, at least). So what’s behind these schools’ dominance? Here’s one theory: Rankings foster a virtuous cycle, where renown draws top faculty, students, employers, and support that ultimately feeds into each other. How about another: These six schools may simply execute better, by virtue of their foresight, investments, collective talent, and commitment.

Or perhaps, to some extent, these schools are getting a free ride.

When it comes to the U.S. News peer assessment scores, the top schools are virtually the academic equivalent of “made men.” Once they establish a certain notoriety, nothing can touch them. Their scores rarely vary, as if their evaluations have gone on autopilot. As a result, they are seemingly impossible to dislodge from their lofty positions.

PEER ASSESSMENT SCORE A SELF-FULFILLING PROPHECY 

Here’s how the U.S. News ranking works: 25% of a school’s ranking–more than any other single component measured by the magazine–is based off these peer assessments, which are filled out by “business school deans and directors of accredited master’s programs.” Using a scale of 1 (marginal) to 5 (outstanding), these peers evaluate the perceived quality of full-time MBA programs. In fact, U.S. News touts this sample – and its 40% response rate – as “the people in the best position to have an informed opinion – “academics who administer and teach in these programs.”

The truth suggests otherwise. Years ago, New Yorker writer Malcolm Gladwell made the point that asking deans and MBA program directors to rate other schools is less a measure of a school’s reputation than it is a collection of prejudices partly based on the self-fulfilling prophecy of U.S. News’ own ranking. In other words, deans fill the magazine’s surveys out by consulting the previous year’s rankings and simply fill in the blanks.

After all, how well do these peers really know the other schools? Take the Stanford Graduate School of Business, U.S. News’ top-ranked MBA program, for example. Since the 2003 rankings, Stanford has notched a 4.8 score for 14 consecutive years. That’s right: Stanford’s score hasn’t fluctuated up or down during this entire period. In other words, Stanford can count on receiving the highest peer assessment score on record (4.8) to comprise a quarter of its ranking.

HARVARD HAS EARNED A 4.8 SCORE IN 13 OF THE PAST 14 YEARS

And Stanford’s peers enjoy a similar leg up when it comes to their academic reputation. Harvard Business School, for example, earned a 4.8 score in 13 of 14 years, only slipping to 4.7 in the 2006 U.S. News rankings. Similarly, Wharton scored a 4.8 in all but one year (2016). MIT Sloan’s scores reflect a modicum of diversity, as the school yielded a 4.7 score in 10 years and a 4.8 in four others (though it has maintained a 4.7 for the past six years).

Even Chicago Booth, whose climb from 9th to 4th over the past 14 years validates the notion that schools can move up in the top 10, has also produced relatively torpid (albeit enviable) peer assessments, with the school earning a 4.7 in seven of the past eight years (only broken up by a 4.8 blip in the 2015 ranking). Kellogg follows a similar path. Aside from a 4.4 low in 2013 – and a 4.8 high in 2003, Kellogg has notched either a 4.7 or a 4.6 in the peer assessment over the past 14 years.

In fact, these six schools are the only ones that seemingly merited a score a 4.7 or higher among peers. And only Berkeley Haas, which has consistently ranked seventh in U.S. News’ full-time MBA rankings, has scored a 4.6. In other words, peer assessment scores do correlate (or show some causation) with schools’ overall rankings. These seven schools are seemingly impervious to slipping below the seventh spot – with the peer assessment scores acting almost as a safety net.

LITTLE MOVEMENT IN PEER SCORES ACROSS THE TOP 50

This same result plays out for schools outside the top seven. Beginning with the 2006 rankings, for example, the University of Michigan’s Ross School of Business has scored a 4.4 on the peer assessment seven times – with its 4.3 scores coming over four consecutive rankings (2011-2014). The same trend follows at Columbia, Duke Fuqua, Yale, Cornell Johnson, Carnegie Mellon Tepper, Emory Goizueta, Indiana Kelley, USC Marshall, and Notre Dame Mendoza. At these schools, just 0.1 of a point separates their top and bottom scores over the last 11 years. And the trend is even more pronounced at UCLA Anderson. Here, the school received scores of either 4.1 or 4.2 during this same time period. And this includes two stretches – each lasting four years – where Anderson received the same peer assessment score.

  • K.K. Hiranjgarbh Missier

    Thats absolutely true. Regards, K.K. Hiranjgarbh Missier

  • Hiranjgarbh K K Missier

    I agree with that! Eitherway, its and excellent article! Regards, Hiranjgarbh K K Missier

  • Hiranjgarbh K K Missier

    I agree with that. Best rankings are and always will be Forbes! Regards, Hiranjgarbh K K Missier

  • Hiranjgarbh K K Missier

    I also agree with the conclusion. People have to earn everything else with their degree. Regards, Hiranjgarbh K K Missier

  • pacifier please

    now now – Don’t be mean Amingo. Your prior posts discussed mean people who did not have warm hearts. I expect more from you so please do not be a poor sport when there is any post that is contrary to your Wharton 1st POV.

  • amingo

    idiot

  • 2cents

    No need to be offensive and insult someone’s IQ. It’s a comment board, pretty much making us all part of the bottom of the barrel. Having said that, none of those schools are mimicking other programs as he/she seems to suggest – it’s a misguided view at best and shows a lack of understanding for those programs.

  • Subnay

    Another person who is talking garbage. Booth and Columbia derivative of Wharton…first of all you have used the word derivative incorrectly; derivative is not the product or real thing it self; it is a reflection of some asset, product or thing..it is kind of image. All modern economic theories basis for finance ( and derivatives) originated from two schools, Chicago and MIT. learn the facts before commenting.

  • Subnay

    Have you ever stepped into one of these institutes….what garbage are your throwing….well…my bad..what can i expect from a 70 IQ person

  • HS Type

    No, unless your girlfriend would like to join me 🙂

  • readthestats_nojust_therank

    The link to U.S. News Best Business Schools site I provided is being reviewed by the P & Q moderator. Feel free to visit the site directly…job placement stats, average salary & bonus, recruiter & dean assessment scores and more are part of U.S. News’ Compass subscription service ($29.95/year). The free stats available – program size and tuition – don’t give MBA candidates much data to truly compare program performance. The job placement ranks I mentioned are based on the top 63 ranked schools in U.S. News Best Business Schools print edition…the Compass service includes 91 ranked schools (and of those, Harvard ranks #49 for job placement at 3 months).

  • amingo

    nice one 🙂 so entertaining to chat with you. have you considered a career in comedy ?

  • amingo

    I am sorry I couldn’t come up with anything but BS when dealing with people of your type.

  • HS Hype

    Forgot that online MBA doesn’t require GMAT 🙂

  • SandDune

    No problem. I just needed a fellow warm-hearted soul to show me the way to the truth. You sell Finance hard at Wharton (and with good reason in fairness), but I think you should also consider pushing Marketing as well as another clear area of excellence. Us fellow warm hearted souls may be receptive. Cheers!

  • HS Hype

    Sounds like any other BS “someone knows someone that someone ” stories… grow up…

  • amino

    my pleasure. I am happy you are finally settled down and saw the truth. god bless you. you are a nice person with warm heart.

  • SandDune

    Ok great thanks. Cheers!

  • amingo

    US News speciality ranking is clear evidence for me. for decades Wharton always the number 1 in finance.

  • amingo

    come on. .don’t be silly. . People like me don’t need gmat..

  • HS Hype

    I tried, Wharton is still not on HBS level. Tell me any successful Wharton MBA and I will tell you one from HBS on exactly the same level or even higher. Be it in Finance, Marketing or others.

  • HS Hype

    He predicted your GMAT score required for your second tier institution 🙂 How’s that?

  • HS Hype

    Exactly what Fidel305 said. Network does not equal to reputation. Brand name is the most important factor to position yourself in countries worldwide, and Harvard University and Stanford University have certainly a better brand name than University of Pennsylvania. Likewise, HBS and Stanford GSB have a better brand recognition than Wharton.

  • SandDune

    Nice try again with the diversionary tactics, but where is YOUR evidence. You have offered none. I did. Easy to keep punting.

  • HS Hype

    Employment rate at Harvard is low, that’s why it isn’t supposed to be #1 program? I am sure that any HBS grad can get any job that the #41, University of Minnesota’s top students are struggling to get. There might be some other factors why certain percentage of HBS grads don’t get a job offer by 3 months, but it is certainly not because companies won’t hire them.

  • amingo

    I did read the “obsolete” info you provided. I appreciate the way you argue but all your tries are in fact immature. they aren’t a genuine ones. I would like you to be more specific and straightforward and support your case with a clear, obvious, and an undisputed evidence. Otherwise, I better off going to the nearby pizza store and talk to the cashier about the number of customers that cancel the order in sundays afternoon.

  • toleen

    the yield rate isn’t always a solid evidence as there are so many lower ranked schools with over 90% yield.

  • SandDune

    Sorry this reply was meant for amingo and not you, as I replied to the wrong response.
    Cheers!

  • SandDune

    Hilarious! Sorry but nice try yourself. Your turn now with facts and not just being a cheerleader.
    Did you actually read the article and the information included?
    Google the article – they will not let me post the link…I just tried again. Cheers!

  • amingo

    nice try but I am not convinced. try again.

  • thawk

    Link?

  • readthestats_notjust_therank

    University of Minnesota was #41 with 89.7% placement. Harvard placed 89.4% of MBA grads within 3 months, and two other schools in the top 10 posted even lower job placement rates – Northwestern – #45 of 63 (88.6%) and Berkeley – #56 of 63 (86.7%).

  • SandDune

    @Amingo,
    I appreciate your intelligent but clearly snarky reply, so please standby for a pending ZipFile to answer ALL of your questions! Also, I posted a longer reply with a link – but it is being held up and not allowed to be posted, so I am sending this as a temp.
    The response that I sent had an article link included that was specifically about student rejection rates of the business schools that they were accepted to, and it was posted on 1/17/2012 by this website and entitled “The Ultimate Ranking: School Rejection Rates”. This article shows factually that there is simply no comparison between Harvard and Stanford and then all the rest of the business schools. I am not picking on Wharton specifically at all here, because it is truly one of the finest in the World, but it is not comparable by admitted students to Harvard or Stanford. According to the article, only 11% of those accepted by HBS turn it down. Also, only 13% of those accepted by Stanford GSB turn it down. Then there is a huge space with Columbia next at 29% of its accepted students turning it down and then Wharton with 30% of its accepted students turning it down.
    Let’s be reasonable here and not use opinion – Harvard and Stanford stand alone and then the rest. Cheers!

  • amingo

    “people will agree in the main and amongst a majority that HBS and Stanford stand alone “… can you provide a clear evidence on this claim? who are those people? where do they stay? what do they do for living? are they women or men? or mix? old or young? what kind of education they have? how did you get their opinions? through an online surveys or by latters? press conference? an advertisement? have you met any of them? do they have a biased view? are they in east coast or west coast ? US citizens or internationals? whites or black? in general, do you feel they are smart to judge or just like you? after getting these info, come and tell us which school is better! otherwise i will be very upset.

  • TheTruth34

    I meant better in strategy general management than HBS but the point remains more than clear

  • TheTruth34

    Dude your argument is just silly. You cant compare the top three schools or divide them in tiers…Would you say HBS is better than Stanford in entrepreneurship? Or that Stanford is better in finance than Wharton? Or that Wharton is better in consulting, general management than Wharton? This is where youre whole argument just falls apart. As someone above mentioned when you start diving deeper down into the rankings you start finding derivative schools like Booth, Columbia for Wharton, Ross and Darden for HBS, MIT and Berkeley for Stanford. That why the HSW are called the top elite schools and the reason why each of them are just so different to the other

  • 2cents

    Not a fan of the analogy because the Rembrandt can be awful and still acrue value on brand name alone.

    The summer between my first and second year I worked in a corporate M&A gig next to two other interns in a different program; one a HBS woman and one a Wharton guy. The girl was incredibly nice but with a non-excel background premba. They were competing for the same job, and the Wharton guy would continually on purpose give her models that would REF out. They were easy to fix if you knew what you were doing, but she was a work in progress on a 20 minute deadline. Obviously the higher ups would come over, see it as incomplete and quickly fix the problem themselves. Neither of them received the one spot – but the HBS girl at least left liked as the higher ups figured out what the Wharton guy did.

    Both of these individuals also fit a reputation for their respective programs, but I wouldn’t let the brand define an individual who wanted to work with/for me.

  • fidel305

    Again, the point is alumni network =/= reputation

    but as to yours, you need to recheck that. With the new economies Stanford is doing great things. As to the old Euro Economies, where HBS is strong, I don’t think they care as much

  • SandDune

    LOL even more now! You W folks on this thread are hilarious! It is tough to refute your arguments (strong sarcasm), with such compelling evidence offered by you as, “HBS is a joke in finance, guys there can’t even do a plain DCF” and “even CBS can do better than that”. Also, others have suggested and discussed as evidence the size of chairs.
    Look being a big fan of a certain school is a cool and great thing but PLEASE. Reasonable people will agree in the main and amongst a majority that HBS and Stanford stand alone and then comes several other great schools in no particular order except with where particular people are most interested in attending. These include Booth, W, Tuck, CBS, Kellogg, MIT, and Haas. After that comes another tier that is also fantastic and includes but has slightly less prestige still and includes Duke, Darden, Cornell, Michigan, Yale, UCLA, CMU, and UNC, and perhaps a few others. As I said, I appreciate fans of schools but no one that is representing a school as powerful as W as an ambassador of sorts should resort to blasting other schools or minimizing them. It actually turns people off on the very brand you are aiming to support. Cheers!

  • i-alum

    stanford alumni is strong in US particularly in west cost. internationally, it is not what it should be.

  • Gilles

    W might not be #1 in your mind, but it is according to recruiters in most reputable surveys. Stanford and Wharton RARELY go toe to toe for ANY job. How can you compare the two schools? They’re not even the same size or on the same coast!

  • W is H’s equal

    Try finance, marketing, operations, research, etc. And humility.

  • NoLove

    HBS is a joke in finance, guys there can’t even do a plain DCF, its really just sad…even CBS can do better than that…

  • P.I.M.P

    THIS IS SO TRUE, it hurts but so true, its simple though HBS – General management, Stanford entreneurship but the undisputied king in finance is Wharton, period.

  • haha

    You should see how Wharton guys and people in the street make fun at HBS people when they come saying how great doing case studies for 2 years was!! OMFG! They can’t even crack a simple model!!

  • OMFG

    hahaha hey I totally agree with this guy’s post, but I coudn’t help laughing at this!!!

  • Esuric

    This article sucks. Reputations don’t just “appear” out of nowhere. They aren’t random phenomena. There’s a reason why HBS has a better reputation than Cornell – a damn good reason.

  • SandDune

    LOL – you are actually pretty funny! However, I remain quite unconvinced by your argument, but at least entertained. Not sure who with or where any dispute exists that Wharton may be tied with or even superior to Harvard or Stanford, except maybe with Wharton folks, and even there it would sadly not be unanimous? I think we have probably beaten this one to a boring pulp but do sincerely appreciate the delta in opinion, as it makes these boards more interesting. Cheers!

  • amingo

    #42 out of 63 is an unfair for HBS. who’s in 41?

  • amingo

    someone told me that his friend knows someone work in a firm that usually recruits MBA kids, the one in the middle isn’t impressed by HBS grads there.

  • gerhard

    without t is much smoother. wartonies don’t fit into the back office seats..too small for them. cornell kids are just fantastic there. HBS and Stanford are inferior to wharton when it comes to finance industry. generally speaking, they might be placed ahead of wharton in some ranking, though disputed, but reputation wise the power house for finance is wharton..it used to be called wharton school of finance. please, don’t get me wrong, i like cornell and appreciate the hard work and efforts they make to be seen, but the world of business education is so severe and i am afraid they may get hurt.

  • amingo

    sounds interesting. what does 305 mean anyway ?

  • readthestats_notjust_therank

    Harvard was #42 out of the top 63 MBA programs for job placement at 3 months (one of the most important metrics for MBA ROI) yet still ranked #1 in U.S. News. When perception outweighs performance, students may end up paying more for prestige than results.

  • fidel305

    Its track team isn’t too bad sometimes.

  • fidel305

    Stanford’s alumni network is small. Its reputation is not. Don’t confuse the two.

  • fidel305

    Based on those well reasoned arguments and clever insights, I, of course, have to moderate my opinion.
    If you have a degree from a second tier institution whose reputation isn’t what you’d wish, then reputation doesn’t matter to you.
    How’s that

  • fidel305

    If you disagree with the Rembrandt analogy, don’t buy modern art. The market for it operates like the market for junk bonds in the Milken era.

    And, I found the recent breakdown in P&Q on salaries to be a good first start, although the “finance” category is far too broad to be very meaningful.

  • SandDune

    LOL – “Whartonies” – you mean Whartonites right? Clever but silly. Cornell is a tremendous business school. I am not saying it is the same as Wharton, but Wharton is also not the same as HBS or Stanford. Perhaps a Whartonite would be very satisfied doing back office work for an HBS alum? Right…

  • gerhard

    poster means cornell grads are good for back office for whartonies.

  • HSW hype

    LOFR I think you’re just delusional. Stanford’s network worldwide is just laughable.
    HBS maybe for everything except finance, they are really beaten up there even by lower ranked schools as Booth or Columbia.

  • HS hype

    UPENN will never be on the same level at anything compared to Harvard. So, it’s H/S and the rest

  • HS hype

    Haha, you’re funny. In reality, it is more like H=S>W in the U.S. and H>S>W internationally. Go travel abroad, the only people who will recognize Wharton will be bankers at top organizations. Harvard and Stanford names will be recognized by almost any human being in the world.

  • SandDune

    Who cares where Cornell or any business school is derived from, as I believe they are largely their own creations. But there can be no doubt that Cornell is an outstanding business school in its own right. I think that is serious and history proves that out.

  • amingo

    Forbes is chinese now. with all due respect, no trust.

  • amingo

    I disagree.

  • amingo

    cornell from wharton ! are you serious?

  • Golden Ring

    despite its weird results, the economist ranking is really challenging the status quo in a professional way. Forbes, yeah it is straight forward method with a very business oriented logic. no emotions with forbes.

  • Golden Ring

    reputation is built over years and based on experiences and proven record. in business world, almost everyone knows Harvard for general management, wharton for finance, stanford for tech, kellogg for marketing. After those names, there are their derivatives, Tuck, Darden, and Ross from Harvard, Columbia, Chicago, Stern, and Cornell from Wharton, MIT Sloan, Haas from Stanford, … and so on. you can extend this list based on speciality for the top 20 schools. all will provide terrific opportunities. similar applied to schools outside US, for example, INSEAD for consulting, London for finance, and IMD for general management. this is the perception in business people minds irrespective of rankings or any other measures.

  • HSW hype

    Erbody knows its H/S/W and the rest, I wouldn’t care about splitting hairs there’s the elite schools and the rest, period.

  • 2cents

    Honestly I agree with the conclusion, though I’m not sure how I feel about the logic that gets you there (or the analogy). Reputation will get a door opened, everything else is yours to earn with the degree.

    MBAs are to the 2010s what JDs were to the 1970s – a catchall degree that will open doors; hence the movement to the former from the latter. Quant factors are in fact useful for the individual to decide their own value proposition (in fact, I would love as others have suggested to see these values split by industry and benchmarked by company peer reputation but that’s a pipe dream). As a whole though a rational person will judge a school on the past/present value of its alumni base, the quality of their academic output/industry ties through private retainers and adjuncts, and, despite national appeal for the top programs, the local market where it is based, especially for career switchers.

    All of these should drive a reputation, and the article simply is stating that none of these really appear to be considered by peer rankings. It seems that building a new main campus can have more effect on a school’s rankings (see Booth or Kellogg in two years) than anything else. Given his recent popularity in MBA curriculum, it’s almost an ironic observation in Kahneman-based inherent biases.

  • fidel305

    when it comes to the MBA degree, reputation matters much more than quant factors like average salary numbers, percentage employed at graduation etc. These supposed quant factors are incredibly flawed.

    A Rembrandt has value because experts assert that it has value.

    And, face it. Unlike MD or doctoral engineering programs, much of the MBA curriculum is generic and / or touchy feely.

    The value is in the imprimatur and, of course, the contacts.

  • BOMtoNYC

    The best rankings are Forbes – which is straight numbers, and The Economist which despite its goofy results is the ranking most trying to get at “strength of programming”.

  • JustChecking

    Excellent article and well done. This article seems to confirm something that I have always believed: that USN, for all of its positives, is a ranking that places far too much significance on the “because that’s just the way it always has been” factor in its methodology. The trash bin of history is littered with empires, companies, and individuals who thought the same way and did not fully recognize the benefits of potential change and competition. This is not to say that many of these b-schools are not fully deserving of their vaunted positions – I am sure many are, but there is a dangerous assumption out there that just expects a certain order. I think this is a mistake. For all of its faults, I still favor the Forbes ranking and its simple measure of ROI as most relevant. Also, it is troubling to me that P&Q weights USN most prominently in its composite ranking, even though a large portion of the USN ranking is based upon the intangible peer assessment that just keeps schools predominantly in the same place/rank. Lastly, and as expected by many, there is likely not any meaningful quality difference not a huge difference between many top 20-25 b-schools. They are all excellent.