It could have been an admissions committee meeting.
Actor Brad Pitt, playing Oakland Athletics General Manager Billy Beane, has assembled his team of scouts and coaches to debate the merit of available talent to draft onto their baseball team. Ultimately, he challenges his colleagues to find players that competitors routinely undervalue by looking at vastly different metrics of success.
“You guys,” lectures Pitt, “are talking the same old nonsense. We’ve got to think different.”
A similar scene will likely play out this Friday morning (June 26) at the annual Graduate Management Admissions Council conference in Denver. Leigh Gauthier, acting admissions director of the Rotman School of Management at the University of Toronto, plans to drop a Billy Beene bombshell. She will cite research which shows that the GMAT, the de facto entry exam for admission into an MBA program, is essentially worthless in predicting the employability of a school’s graduates—even though that is exactly why most students enter an MBA program.
The finding—backed by an extensive study of the admission files and employment outcomes of more than 1,000 MBA graduates at Rotman over a five-year period from 2008 to 2013—comes at a time when business school admission directors privately say they are putting more weight on the GMAT than ever before, chasing 700-plus scores, yet quietly acknowledging that they are passing on candidates who they believe hold the promise to have far more successful careers than the higher GMAT-scored applicants they accept.
‘SHOCKING: HOW CLOSE TO IRRELEVANT THE GMAT WAS IN PREDICTING EMPLOYABILITY’
That’s largely because of the widespread belief that the GMAT is a proxy for the overall quality of a class and also largely because U.S. News & World Report’s annual MBA ranking puts significant weight on the average GMAT score of the latest incoming class. The result: More schools are using scholarship dollars to buy high GMAT scores to lift their reported numbers to U.S. News, and more admission officials are asking candidates they would like to admit to simply retake the test and get a higher score so they can report the bigger number to U.S. for their rankings. And before U.S. News began including GRE scores in its calculations three years ago, some schools would tell sub-par GMAT candidates they wanted to accept to take the GRE so they wouldn’t have to include the lower GMAT scores in their report to U.S. News.
It makes sense that Rotman, which is not subject to U.S. News’ rankings because the publication only ranks U.S. business schools, would make the shift. Average GMAT scores make up slightly more than 16% of the U.S. ranking, a weight that is greater than either starting salary and bonus or survey results of corporate recruiters, two other metrics used by U.S. News for its ranking. Bloomberg Businessweek, Forbes, and The Financial Times do not use GMAT scores for their rankings.
Rotman undertook its study nearly three years ago, even putting a full-time statistician on the payroll to crunch the data. “The most shocking surprise for us was how close to irrelevant the GMAT was,” says Kevin Frey, managing director for the full-time MBA program at Rotman. “The one proviso is that GMAC has never claimed the GMAT exam is predictive of employability. It’s predictive of your ability to perform in the program.
“But schools that are just trying to measure how their applicants will perform in the classroom are measuring the wrong stuff. We have a moral obligation to these students. They are going to invest a quarter of a million dollars into this degree. When we accept them, we need to feel very confident they are going to get the career outcomes they want. But the GMAT cannot be used as a proxy for employability.”
‘WE NEED TO IDENTIFY & EVALUATE TALENT THAT OTHERS ARE DISCOUNTING’
Frey used Billy Beane logic to launch the multi-year study that first looked into some 95 different candidate variables, from academic undergraduate grade point averages and admission interviews to citizenship an TOEFL test scores. Frey was the “principal investigator” and Mihnea Moldoveanu, then associate dean of the MBA program and now vice dean of learning and innovation, was the faculty lead on the project, while Don Ilodigwe was the initiative’s statistician.
To his way of thinking, Frey sees Rotman resembling the more resource constrained Oakland Athletics, competing on the world stage with the likes of Harvard, Stanford, Wharton and others, all of whom have sizable endowments, access to significantly larger scholarship funds and bigger brand names. It was not dissimilar to the situation of an Oakland team, with highly limited resources, competing for talent against the higher spending Los Angeles Dodgers, the New York Yankees, and the Boston Red Sox.
“MoneyBall,” he says, “is not a bad analogy. We need to identify and evaluate talent that others are discounting and not seeing the value in. A lot of people think we are in the sleepy education business, but we are also in the talent game. We have to be able to identify the talent, bring it in, develop it and then match it with employers. Rotman doesn’t win too many bidding wars with the Harvards, Stanfords and Kelloggs of the world, so we decided to find ways to identify hidden talent that those schools may not be able to identify or may not be fully valuing. To do that, we had to move beyond simple correlations, ratios and descriptive statistics and into the powerful world of predictive analytics.”