QS Co-Founder Defends His MBA Ranking

After being called out as one of the worst MBA rankings, QS has launched a defense pf the company’s MBA ranking that Co-Founder Nunzio Quacquarelli claims is the world’s most popular of the annual lists.

Without mentioning that the QS MBA ranking was named one of the three absolute worst accountings of MBA programs, Quacquarelli stood behind the methodology he uses to rank business schools. “QS invests heavily in data scientists, often with PhDs, and data analysts to derive innovative metrics and to verify data to ensure our rankings don’t just tick the box of conventional wisdom, but use verified metrics to provide prospective students and education leaders with valuable insights to help them make better data-informed decisions,” he wrote in an essay published on Aug. 10, roughly two weeks after Poets&Quants singled out the QS ranking.

The co-founder of QS did not address concerns raised by an academic at UC-Berkeley who found a correlation between a school’s numerical ranking and the money it spends with QS on admission fairs and other services nor does he address criticism that his company’s methodology is vague, results in highly questionable outcomes, and lacks the transparency of rival rankings published by the Financial Times, U.S. News & World Report, and Bloomberg Businessweek. Among other things, QS does not reveal the response rates for the three surveys upon which its ranking is based. QS also fails to show the full calculation behind each of the 13 metrics it claims to measure in the ranking.

THE QS MBA RANKING IS ‘UNIQUE’ WITH A ‘STRONG FOCUS ON THE METRICS GEN Z CARE ABOUT’

In his article, Quacquarelli maintains that QS has developed “a unique MBA ranking with a strong focus on the metrics Gen Z care about. We gained this insight through our QS International Student Survey, which reaches over 125,000 individuals annually.” The QS MBA ranking, however, does not incorporate the views of those students, instead relying on surveys to employers, business schools, and academics, without ever disclosing the numbers of surveys sent or completed in any of those questionnaires. QS also does not share the response rate for the student survey Quacquarelli referred in his article.

“Throughout the two decades since the inception of the QS World University Rankings in 2004, we have continuously collected and responded to feedback from stakeholders, including academics, students, industry professionals and employers. This valuable input has informed the development and improvement of all our rankings’ methodologies,” adds Quacquarelli.

David White, a co-founder of Menlo Coaching, a leading MBA admissions firm, finds the QS MBA ranking methodology sorely wanting. “In terms of the results,” he says, “the least useful MBA ranking is the QS Global MBA Ranking. In that ranking, HEC Paris and IE Business School are ranked significantly higher than M7 schools, including Kellogg, Booth and Columbia. This does not match reality: many applicants who are qualified for admission to HEC Paris and IE would be declined at Kellogg, Booth and Columbia, and very few applicants would accept an HEC Paris or IE offer if they also had an offer from Columbia, Booth or Kellogg.”

IN THE QS MBA RANKING, HEC PARIS BEATS CHICAGO BOOTH FOR THOUGHT LEADERSHIP

His reservations, however, go well beyond where a specific school finished in the ranking. “QS does not show the full calculation behind each factor, but I’d speculate that the way HEC and IE did so well in the ranking is related to: 1) Strong employer perceptions in their own countries despite a weaker ability than M7 schools to place graduates into roles in other locations, and 2) Lower incoming salaries for students helps with the ROI calculation. HEC and IE recruit more students from Europe, where salaries are lower than in the US. It is a fact (based on each school’s employment report) that Columbia, Booth and Kellogg have higher average post-MBA salaries than HEC, but HEC does better in the ROI calculation, probably because their graduates’ salaries were lower before the MBA.

“It is also curious that HEC beats Chicago Booth for “thought leadership” despite Chicago Booth’s numerous Nobel Prize winning faculty. HEC Paris and IE are great MBA programs for the right candidates, but cannot seriously be ranked above M7 schools.”

Igor Chirikov, director of the student experience at the Research University Consortium at UC-Berkeley, has more serious concerns about QS rankings in general. In a paper entitled “Does Conflict of Interest Distort Global University Rankings?,” Chirikov examined the impact of contracting with rankers on university ranking outcomes by gaining access to the advertising spend of universities with QS. He focused on the positions of 28 Russian universities in QS World University Rankings between 2016 and 2021 with information on contracts these universities had for services from QS. The 128 QS-related contracts he discovered between 2013 and 2020 showed that these Russian universities spent $3,871,378 on QS services during the five-year time period.

STUDY FOUND THAT SCHOOLS WITH QS CONTRACTS GOT A 140-RANKING POSITION ADVANTAGE

His conclusion: “The study finds that frequent contracts for services from QS contribute to a substantial increase in the rankings of universities,” he writes. “Positions of Russian universities that had frequent QS-related contracts increased on 0.75 standard deviations (approximately 140 positions) more than they would have increased without frequent QS-related contracts. In a similar way, QS faculty-student ratio scores of Russian universities that had frequent QS-related contracts increased on 0.9 standard deviations more than they would have increased without frequent QS-related contracts…On average, QS ‘frequent clients’ universities also improved their faculty-student ratio scores by 14 points during the same period while QS “occasional or non-clients” did not.”

Chirikov says he focused on the QS rankings over other global lists for three reasons. “First, Quacquarelli Symonds (QS) offers universities a wider array of services than other rankers. For example, it offers a fee-based rating system, QS Stars, that evaluates universities and awards them from 0 to 5+ “gold stars”. The stars indicate “quality” of the university and appear on the website next to the university name, including in the main QS World University Rankings table. Second, Russian universities had a much larger number of contracts with QS than with any other ranker. Third, QS has been frequently called out by media outlets and higher education experts for having and not reporting conflicts of interest.”

In his essay defending his rankings, Quacquarelli says nothing about Chirikov’s study.

VAGUE METHODOLOGY AND LACK OF TRANSPARENCY IN THE QS MBA RANKING

The problems with the QS MBA ranking go right to the heart of the public’s increasing skepticism about rankings. For example, the most significant metric in the QS MBA ranking, accounting for 35% of a school’s position, is based on an employer survey. QS does not reveal how many employers it surveys in a given year, nor the response rate to the survey. The lack of transparency goes against fundamental reporting guidelines on surveys based on journalistic standards. QS also fails to reveal how it calculates this score, merely noting that it “asks employers to select the schools that they prefer to hire from…Employers across all sectors and industries take part in the survey, and include Facebook, Google, Uber, Wells Fargo, Bank of America etc.” It is not clear whether these companies completed their surveys or were merely surveyed and did not respond. No less important, QS does not report the raw scores for any of the ranked MBA programs.

The second most heavily weighed metric, accounting for 15% of the QS MBA ranking, is a 10-year return on investment calculation which also is not shared with readers. QS also fails to reveal how it obtains post-MBA salary data ten years after graduation, a metric that is not available from business schools nor is publicly available. Yet, QS claims that “salary increases are factored into both pre and post-MBA salary, with the latter increasing at a higher rate, as you would expect. Schools are given a graded bonus for the number of entrepreneurs produced, to account for the slower, but potentially much higher return for those starting a business.”

Some 10% of the ranking comes from what QS calls an “Alumni Outcomes Index” which the company says is based on “schools associated with over 50,000 CEOs, executives and board members at the biggest companies in the world including Apple, Amazon, UBS, IBM, Microsoft, JPMorgan Chase, ExxonMobil, AT&T, PepsiCo.” Again, QS provides no explanation for how it calculates this metric nor exactly how it is collected or counted. Even worse, the vagueness of this and most other elements of the methodology insure that no outsider can doublecheck the accuracy of the data.

THE QS MBA RANKING USES MANY COMMONLY REPORTED METRICS THAT ARE HARDLY ‘PROPRIETARY’

And while Quacquarelli asserts that “QS has developed proprietary metrics around employability, entrepreneurship, ROI, thought leadership and diversity, collected by QS research analysts,” the vast majority of the measures it uses are commonly reported by business schools and used in other rankings. Among them: The percentage of graduates with jobs three months after graduation, the percentage of MBAs accepting employment within three months, the percentage of graduates who start companies after graduation, the percentage of female and international students in a cohort, and the percentage of female and international faculty.

Even so, Quacquarelli boasts that “no other rankings organisation, that we are aware of, goes to this level of detail in gathering, analysing and presenting fresh data to prospective students. We believe this methodology provides a unique lens for Gen Z students. Moreover, our detailed evaluation methodology provides an ideal platform for innovative business schools to shine. Those institutions that demonstrate exceptional results in promoting employment opportunities or fostering entrepreneurship among their graduates can truly distinguish themselves in our assessments.”

The 2023 Top Ten According To QS

2023 Rank & School	Index Score
1. Stanford Graduate School of Business	93.6
2. Harvard Business School	92.4
3. University of Pennsylvania (The Wharton School)	92.1
4. HEC Paris	92.0
5. London Business School	91.6
6. MIT (Sloan School of Management)	91.5
7. IE Business School	91.4
8. Columbia Business School	90.7
9. INSEAD	89.9
10. IESE Business School	89.8

DON’T MISS: THE ABSOLUTE WORST MBA RANKINGS or