How The Wall Street Journal Plans To Rank MBA Programs

Ranking business schools isn’t as easly as it looks. At least that is the inevitable conclusion by how long it is taking Times Higher Education (THE) to crank out its first list of the best full-time MBA programs.

It was last June when the British publication announced that it would team up with The Wall Street Journal to publish yet another set of rankings on MBA programs and specialty master’s programs in finance and management. The hope was to produce the first lists this spring, but now the debut ranking has been pushed out to November of this year.

Two editors from The Wall Street Journal—Dave Pettit, editor of specialized news, and John Simons, deputy bureau chief for management and careers—have been doing a road show of sorts, meeting with business schools to present the methodology for the new lists. Simons declined comment, saying in an email he was too busy for an interview. Poets&Quants obtained their Powerpoint presentation. Among other things, the new MBA ranking will try to measure the “social good” of MBA graduates, likely to be a highly controversial data point, as well as the self-perceived entrepreneurship skills received in an MBA program.

STILL DECIDING WHETHER TO COMBIN ONE-YEAR AND TWO-YEAR PROGRAM RANKINGS

It’s clear that not every aspect of the rankings are settled. The pair, for example, told business school deans that they have yet to decide whether to combine one-year and two-year MBA programs in a single ranking, similar to what the Financial Times does, or rank these programs separately, as Forbes does.

The methodology will be based on surveys to both alumni and schools, with the possibility that THE and the Journal may survey employers and recruiters at another time. The surveys will inform a rather complicated methodology that includes 21 different metrics. The ranking is heavily dependent on the views of alumni, with 12 of the 21 data points informed by responses from alumni who are both two years and four years out of school.

Obviously, one issue that THE and the WSJ will have is getting candid answers from alumni who know their responses will ultimately reflect on where their alma mater ranks. That is a problem with all rankings that survey students or graduates of MBA programs.

NO METRICS ON INCOMING STUDENT QUALITY PLANNED

The new rankings come at a time when more business school deans and research faculty are taking issue with the controversial lists. Academics often argue that rankings are misleading and disingenuous. Times Higher Education, moreover, is entering the business school market 31 years after Businessweek published its first MBA ranking and 20 years after the debut of the global MBA ranking from The Financial Times. The proliferation of rankings, fueled by consumer interest in them, has largely diminished the influence any one ranking has on the market. But the WSJ has an average paid print circulation of 2.2 million which will give the new list widespread exposure.

One big surprise: The new ranking will not attempt to measure the quality of incoming students in a program, a significant part of the way U.S. News ranks MBA programs by using GMAT and GRE scores, grade-point-averages and acceptance rates. Not surprisingly, career outcomes loom large, accounting for 38% of the total ranking. The two metrics getting the most weight are the difference between pre-MBA and post-MBA salaries, which will be given a weight of 12%, and faculty-per-student ratios, which will account for 10% of the ranking.

According to Pettit and Simons, the salary metric “will take account of salary differences related to the sector and country before and after the business degree to provide a real indication of the change.” Exactly how this adjusment to the data would be made was undisclosed.

Faculty-per-student ratios, moreover, are often a difficult metric to nail down. That’s because most schools employ adjunct teachers, clinical professors and tenure-track faculty. How their time is alloted can make a singificant difference in the reported ratio. Another complicating factor is that many professors teach undergraduate students as well as graduate-level candidates.

WILL RANK MBA PROGRAMS ON FOUR OVERALL CATEGORIES

In general, the WSJ plans to rank programs under four categories: resources (with a weight of 25%), engagement (25%), outcomes (38%), and environment (12%), the latter attempting to measure “the social and human environment the students find themselves in and how well the school will prepare them for a global market.”

Under resources, meant to measure the resources available to the school to ensure quality teaching and support, there are five measures with four of them taken from the self-reported data by schools and one data point informed by alumni surveys:

1. Faculty per student (10%)
2. Publication per faculty (4%)
3. Faculty with Phd/terminal degree (5%)
4. Career support staff per student (3%)
5. Career support effectiveness (3%)

MEASURING TEACHING QUALITY AND THE LEARNING EXPERIENCE

Alumni surveys will account for the five metrics the THE and WSJ will use to attempt to measure what it calls “the school’s teaching quality and the student’s learning experience:

1. Recommend (5%)
2. Collaborate (5%)
3. Engagement (5%)
4. Real world (5%)
5. Research (5%)

This section will include a series of questions on the alumni survey, including “If a friend or family member were considering going to business school, based on your experience, how likely or unlikely are you to recommend your school to them?”

To get at the degree of collaboration at a school, alumni are being asked, “To what extent did you have the opportunity to interact with the faculty and teachers at your school as part of your learning experience?”

HOW REAL WORLD IS THE PROGRAM?

To measure the real world implications of the program, alumni are asked to response to these questions: “To what extent did the teaching at your school support applying your learning to the real world?;” “To what extent did your teachers present and discuss recent real world cases?,” and “To what extent did you have opportunities to meet and work with professionals with current real world knowledge (outside your teachers/lecturers)?”

And when it came to research, alumni are asked the following question: “To what extent did your teachers present and discuss current research?”