The Case Against The GMAT

Frey deployed advanced data analysis to get to the more critical variables that directly forecast whether an applicant will have a job three months after graduating from Rotman’s MBA program. “What shakes out are a set of five variables that are significantly predictive of employment,” he says—and not one of them is the GMAT. “It’s an exciting finding because the whole industry pivots around the GMAT and it was at best marginally significant. We don’t have hard statistical confidence the GMAT actually matters at all and if it does, it doesn’t matter very much. It had a very small effect size.”

ADMISSION INTERVIEW SCORES & AWA SCORES WERE STRONGLY CORRELATED WITH EMPLOYABILITY

What does matter? Here are the five most significant warning signs that an applicant is most likely to be an employment risk, in order of importance:

• Citizenship from a region of the world for which Frey declines to identify.
• Years of work experience.
• Admissions interview scores for candidates.
• Analytical Writing Assessment (AWA) test.
• Undergraduate grade point average.
• Then, finally, came an applicant’s GMAT score—barely.

“Some statisticians would not even include the GMAT score because it was only marginally significant,” adds Frey. “It’s barely in the score.”

Based on the study, Rotman then devised an “employability risk score” for every person who applied to its MBA program in the 2014-2015 admissions cycle. A perfect score is zero. A score of four or more signals that the candidate is a high risk of not getting a job within three months of graduation. “Like a credit card company that assesses the likelihood that an applicant will be able to service their debts, we wanted to cut through all the noise and identify the variables that could predict the probability that prospective students would be unemployed post-graduation,” explains Frey.

In contrast to the GMAT, the AWA score (which ranges from zero to six) and the scores admission officials gave applicants for their face-to-face interviews were highly predictive of a candidate’s ability to land a job and to gain a solid starting salary. “They are far more important than a GMAT score,” says Frey. “If an employability risk score of four makes you high risk then having an AWA score under a certain number (which Frey will not disclose) would get you two points on the employment risk score and having a GMAT below a certain score would get you only one point.” Translation: Both AWA scores and admission interviews are at least twice as predictive as the GMAT for employability.

‘A BIG, BIG RISK FACTOR: TEN OR MORE YEARS OF WORK EXPERIENCE’

Candidates with ten or more years of work experience were more likely to be unemployed three months after graduation. “It’s a big, big risk factor,” says Frey. “That person is three times more likely to be unemployed post grad than our control group which had three to five years of work experience. It was highly predictive.”

Frey believes the predictive nature of the admission interview scores was “completely logical because to get a job you have to do well in an interview. If you get ranked well on the admissions interview, chances are you probably will do well on a job interview later.” At Rotman, all interviews are done not by alumni or second year students but by admissions staff.

More surprising was the predictive ability of the AWA. “The AWA had a substantial effect size, yet we had never looked at it before. But it is highly predictive of your ability to get a job.”

References, admission essays, academic and industry background, among other variables, had no significant impact on employability, says Frey. “What you majored in didn’t matter. Neither does gender.”

How has it changed admission decisions at Rotman. Rather than routinely shovel scholarship money to high GMAT candidates, the admissions committee has a more reasoned approach. “We want students who will have an impact in the world,” adds Frey. “The people we want to give scholarships to are those that organizations will want to hire. So we have reversed engineered this. We don’t want to give scholarship dollars to students who will likely underperform in their careers. GMATs are becoming more and more irrelevant to the scholarship discussion. A high AWA will definitely help your case for a scholarship now. And yet only a year ago we would not have even considered the AWA.”

Acting Admissions Director Gauthier believes that the GMAT assumes so much importance in MBA admissions “because it’s easy. When you look at thousands of files, you have to sort on something. So it has become a sorting mechanism that some find helpful.”