GMAC ‘Jazzed’ By Early IR Section Results

Since the launch of the integrated reasoning section of the GMAT on June 1, more than 110,000 test takers have endured the new 30-minute IR test with a dozen questions. The Graduate Management Admission Council says the early results of those tests are now in and they’re pleased with what the data shows.

“I’m totally jazzed about how this has turned out,” says Peg Jobst, executive vice president of GMAT. “Larry Rudner is our chief psychometric guru and he has taken our 110,000 records and banded them. He plotted a bell shape curve, which showed there’s a new piece of information the test is giving admissions committees. You really have a new data point on an indivudal test that you didn’t have before.”

Jobst says that the results thus far show only a modest correlation between the IR scores and the GMAT’s quant and verbal sections. That suggests that the new section is providing information over and above the traditional GMAT exam. So someone who does well on quant and verbal may not do as well on the IR section and vice versa.

Rudner’s study, which examined the distribution of IR scores for all GMAT test takers scoring between 600 and 640, shows “a convincing distribution of students” with IR skills. In other words, the test appears to be accurately measuring what it is intended to measure: a person’s ability to synthesize data in graphics, text and numbers as well as combine and manipulate data to solve problems.

Some 5% of the study’s test takers got the lowest IR grade of one, while 8% received the highest grade of eight. The largest percentages of students—19%–scored either a five or a six, creating the bell curve shape, which demonstrates a normal distribution pattern (see left). GMAC says the mean score for the IR is 4.34 on the eight-point scale.

“Our first big question was to ask whether different groups of test takers with the same general ability level receive the same IR score,” explains Rudner in an essay. He concluded that the test results are meeting GMAC standards as an “impartial, objective” measure.

“The differences between native and non-native English speakers, U.S. and non-U.S. citizens, U.S. white and non-white test takers, and business versus non-business undergraduate majors in their IR scores are, in all cases, less than one quarter of a standard deviation,” writes Rudner, vice president of research and development for GMAC.

Translation: the differences are minor and practically inconsequential among these different groups.

A remaining issue is whether the new IR section adds to the ability of the GMAT to predict core course grades. “Quant, Verbal, and IR are all measuring something related, yet also different,” says Rudner. “The observed IR-GMAT total score correlation indicates that IR is likely to add meaningfully to the predictive validity of the GMAT.”

Since the test’s launch last June, many admission officers have taken a wait-and-see attitude toward the IR, largely because it’s so new and many candidates in the current applicant pool did not take the IR section. GMAT scores are good for five years after taking the test, and many test takers scrambled to take the older GMAT to avoid the need to study and practice for the new section. As schools become more comfortable with the new data point, admissions committees will begin weighing the measure as an additional piece of information to decide who to admit or reject.

GMAC began thinking about the new section after a 2009 survey of 740 management school faculty suggested that integrated reasoning skills are a prerequisite for 21st century management students as they prepare to work in technologically advanced, data-driven environments.

Unofficial score reports, containing the Total score and Verbal and Quantitative section scores, are provided immediately after the test to those who accept their scores. Official Score Reports, including the results of the new IR section, are generally available 20 days after the test date.