It’s hard to get into any discussion about the newest MBA ranking from Bloomberg Businessweek without first addressing the big elephant in the room: Is it even legit?

For the second year in a row, one of business education’s most prominent academics has found something fishy about this ranking. Yale School of Management Deputy Dean Anjani Jain believes that Businessweek incorrectly applied its stated weights to each of the five metrics the magazine uses to rank business schools. The upshot: The results reported by Businessweek would not be the same if, in fact, the magazine used the actual weights it reports. While Businessweek claims to have placed a weight of 37.4% on compensation, for example, the actual weight on this dimension is nearly double that at 68.9%, according to Jain’s analysis. While the magazine contends it put a 25.5% weight on its “learning” index, the actual weight is about a quarter of that at 6.4%, Jain found.

In fact, if Businessweek used the weights they contend they use, the ranking outcomes would be so topsy-turvy as to raise even more serious questions about this ranking. The Wharton School’s full-time MBA program, currently undervalued in this ranking in seventh place in the U.S., would end up near the bottom of the Top 20 U.S. schools in 20th place.

It needs to be said that Bloomberg Businessweek refuses to acknowledge any problem with its ranking. The magazine stands by this list as it did last year after questions arose about its 2021 MBA ranking. In response to several emails Jain recently sent to Bloomberg Businessweek seeking greater clarification on its methodology, Senior Executive Editor Laura Zelenko tersely shot back: “We don’t plan to disclose any more about our methodology than we already have.”

So we have to lead our ten biggest surprises in this ranking with some of the nitty-gritty detail that makes this list less credible and then move on from there with the big shocks and revelations that spills out of Businessweek‘s conclusions (also see our Business Casual podcast on the Bloomberg Businessweek MBA ranking).

1) Can Wharton’s MBA Program Really Rank 20th Best In The U.S.?

You already know the answer to this question. Of course not. Yet, Jain points out that if Businessweek actually ranked MBA programs by the stated methodology, that’s exactly where Wharton would rank. This isn’t a surprise, really. It’s shocking, plain and simple. The reason why this would occur is that Wharton failed to rank in the Top Ten on three of the five metrics used by Businessweek for its ranking: Learning, networking, and entrepreneurship. And when it ranked in the Top Ten, it was closer to the bottom than the top, ranking seventh in compensation and eighth in diversity.

It was especially harmed by lowish scores in the magazine’s surveys from students and alumni, particularly on the “learning” index where Wharton was ranked a lowly 71st. Jain’s analysis shows that Businessweek significantly undervalued this metric, where Wharton received its lowest rank, and overvalued compensation, where Wharton got its highest ranking. That change helped Wharton improve by two places from its lowest position ever in the Businessweek ranking, ninth place, where it resided last year.

How would the ranks of other top business schools be impacted by Jain’s recalculated ranking? NYU Stern’s MBA program would plunge to 21st from 11; UCLA would fall to 40th from 20th, and Rice University’s MBA would drop to 49 from 29. And some schools would, of course, gain. Indiana University’s Kelley School of Business would see its full-time MBA program ranked 13th instead of 22nd; Washington University’s Olin Business School would have a 14th ranked MBA vs. 21st, and USC Marshall would see its MBA ranking jump to ninth from 16th (see below for how the Top 25 would all be impacted this year and last by Jain’s analysis):

What to make of all of this? Do not trust this ranking. It lacks credibility, even integrity. For sheer entertainment, we give it a seven or an eight on a scale of ten. For being helpful to MBA candidates, we sadly give it a two.

2) The Best Places To Learn & Network: Underrated Gems Or Unforgivable Errors?

Everyone knows what learning is. It is the mastery of a body of knowledge or set of skills through study and experience. How do you measure it? Well, that’s where learning gets tricky.

Take a test? It’s easy to cram in the short term. Then again, who needs memory when you have Google? And what good is learning if you haven’t applied or built on it?

Face it: Shouldn’t we hold off doling out accolades until we see if students’ titles and achievements first – if those measures even matter, that is?

Bloomberg Businessweek factors Learning into its methodology. Carrying a reported 25.5% weight among American schools, the Learning category is 100% survey-based. Using a 1-7 scale – which is scaled up to a 100 index, Bloomberg Businessweek claims it reached out to 6,422 current MBA students, 11.304 alumni (Classes of 2013-2016), and 778 employers. To qualify, the 113 business schools involved were required to meet individual thresholds based on the sizes of their graduating and alumni classes. In other words, Bloomberg Businessweek readers have no idea of each school’s sample size.

And what exactly is Learning? Like sample sizes, Bloomberg Businessweek doesn’t disclose the questions or include school-by-school breakdowns of scores. In other words, they operate off the honor system, where Learning is defined in the following terms:

“For the schools’ core mission, we explore the quality, depth, and range of instruction, focusing on the curriculum relative to real-world business situations; emphasis on innovation, problem-solving, and strategic thinking; mentoring and support from instructors; class size; and collaboration.”

If you’re wondering whether Bloomberg Businessweek’s Learning scores are credible, it helps to follow your gut. In the United States, the top MBA program for Learning is…the College of William & Mary. The 41st-ranked program with BW, Mason finished 95th (Financial Times) and unranked (The Economist) in the other survey-driven rankings. However, Mason also placed 1st in Learning in two o the past three rankings. In the Princeton Review survey, which was released nine months ago, Mason ranked 5th for Teaching Excellence (and Top 10 for being Family Friendly and providing Resources For Minorities). On a 10-point scale, Mason also posted a 9.34 average in the Financial Times’ Overall Satisfaction measure. This alone reinforces the notion that Mason’s Learning prowess in BW’s sample is hardly an anomaly (even if it may be slightly exaggerated).

Still, the Learning ranking can be a bit disorienting. Emory Goizueta and the University of Georgia Terry tied for 2nd – despite not standing out in any comparable category in the Financial Times, Economist, or Princeton Review MBA surveys (Terry did place 1st for Facilities in The Economist). The University of Texas-Dallas – which traditionally ranks high for Faculty Research – and the University of California-San Diego – an Entrepreneurship powerhouse – rounded out the Top 5.

Beyond the Top 5, Learning begins to settle into convention. Stanford GSB, Yale SOM, Dartmouth Tuck, and Virginia Darden – all programs regaled for teaching excellence – cracked the Top 10. Still, SDA Bocconi ranked 12th in Europe for Learning – after ranking #1 worldwide in Overall Satisfaction in the Financial Times survey. That’s a gap that defies easy explanation, particularly when Bloomberg Businessweek doesn’t offer any sense of transparency with their data. That’s why readers must be skeptical in the Learning category when BW surveys place American University over Harvard Business School or Hult above Northwestern University’s Kellogg School (or South Carolina ahead of MIT Sloan or the University of Chicago’s Booth School).

And who in their right mind would rank the University of Mississippi 35th when it comes out over Columbia Business School (39th), Berkeley Haas (59th), NYU Stern (64th), and Wharton (71st). Such deviations should tip off BW to adjust either its questions or the way it poses them.

Like the Learning category, Bloomberg Businessweek’s Networking category is almost as dubious. Holding a stated 17.4% ranking weight in the United States, Networking is defined by these parameters:

“This index gauges the quality of networks cultivated by classmates; students’ interactions with alumni; effectiveness of the career services office; quality and breadth of alumni-to-alumni interactions; and the school’s brand power, according to recruiters.”

Again, readers cannot unpack the results to see, for example, how a career center’s results match up against its brand power. Why does that matter? Take UCLA Anderson, where surveys consistently show the program boasts a top 3 career services center globally. However, you wouldn’t know that from Bloomberg Businessweek, where UCLA Anderson ranks 29th overall. When it comes to network fervor, you’d be hard-pressed to match Dartmouth Tuck’s alumni fund-raising or USC Marshall’s alumni network. In BW’s survey, the schools finish a little low at 5th and 8th respectively.

Alas, BW’s Network ranking features more familiar faces. Stanford GSB tops the list, while Harvard Business School and the University of Chicago’s Booth School hold the Top 5 spots. Indiana University’s Kelley School may rank 2nd, but its career center has historically ranked among the world’s best according to student and alumni surveys conducted by The Financial Times and The Economist. Alas, the College of William & Mary ranks 6th for Network. However, that comes with the territory in a BW survey – as does the University of Georgia finishing 15th (and ahead of Columbia Business School and MIT Sloan).

Bottom line: Readers need real data from Bloomberg Businessweek before they can decide how credible its surveys are. In the past, BW has leaked out select data from narrowly-tailored categories. Considering the head-scratchers in Learning – and even Network – they may be wise to deliver a full-on data dump.