The Grand Mystery Of GMAT Scoring

What does all this mean?  It leads to this executive summary of how adaptive scoring works: the system uses a, b, and c parameters to estimate the probability of your “true” score.  After 4 questions, for example, the system might be computing these five data points:

• Correct on a very informative (high a-value) 50^th percentile question
• Incorrect on a fairly informative (medium a) 80^th percentile question
• Correct on a very informative 70^th percentile question
• Incorrect on a very informative 80^th percentile question

Even though your % correct is 50%, the computer would likely assess this response pattern as giving your score a very high probability of being at around the 75^th percentile, a relatively low probability of being below the 65^th, a reasonable probability of being at the 70^th, and a relatively low probability of being above the 80^th.  So while you might think “I only got half right”, really the system sees “this student is very likely well above average”.

Which is all a long way of saying that the IRT system is complicated and sophisticated.  In many ways it’s more important to avoid “negative understanding” of the testing system (and there’s plenty of it out there) than it is to try to understand it.  So with that in mind…

Adaptive Scoring Fact vs. Fiction

Now, let’s dispel some potentially damaging takeaways you may get from that demonstration, because again the crudeness of quick-and-easy demonstrations of IRT tends to lead people to bad conclusions.  Does this demo mean that it’s absolutely essential to get the first question right?  No – absolutely not.  This was just a four question demonstration; if this user misses the next 6 questions he’s likely well below average. And if a different user missed the first two there’s still a path to a very, very high score.  It’s just a harder path.

So don’t believe the hype – you don’t need to get all the first ten problems correct in order to have a great score; you can score insanely high even if you only get 6 or 7 right.  And if you do allocate more of your time to the first ten questions such that you’re scrambling by the end of the test, the system will learn that you’re not as impressive as you pretended to be.  A couple of incorrect answers to highly informative questions in the 50s and 60s will start to chop down the weight of those earlier data points.

What is important for a high score?

• • Avoid making silly/careless mistakes on questions you should get right.  Missing easier questions comes with two negatives: one, it increases the probability that your ability level is lower than you want it to be.  And two, it informs the delivery function to serve you another lower-level question – a question that doesn’t have a high information value at the ability levels you want to achieve, so it won’t add a particularly helpful data point to your case even if you answer it correctly.

 

• •  Learn to let go.  Many people spend a disproportionate amount of their time on the harder questions, which then leaves them vulnerable to mistakes like the above.  We’ve often used the phrase “your floor is more important than your ceiling” – missing questions you should get right hurts you more than getting a question right that was probably above your head helps you.

 

• • Worry less about percent correct and more about what you can learn from your results.  It’s fairly common for a student to see that she missed 5 Critical Reasoning and only 3 Sentence Correction and determine that she needs to improve on Critical Reasoning.  Which may be the case – but since degree-of-difficulty can vary among those questions, it’s not a guaranteed strategy.  Avoid making rash judgments simply based on percent correct, and dig a little deeper into why you made the mistakes you did and how you can improve.

 

• Trust the IRT system on the GMAT.  A lot of the handwringing about the adaptive nature of the test comes from the idea that other people who seem to understand CAT testing might have an edge on you because they’ve figured out a way to game it.  They don’t.  The system deals in probabilities based on millions of data points it has collected over time. There’s no magic warp zone or extra life – it’s not a video game.  You can’t game the system, and that’s a good thing – you’ll be rewarded for your hard work and those who think they can figure out a loophole in the CAT will be wasting their time. 
  

  Brian Galvin of Veritas

  Brian Galvin is Director of Academic Programs at Veritas Prep, a GMAT prep and graduate school admissions consulting provider. This is his second column for Poets&Quants.com. His contrarian views appear monthly.

  DON’T MISS: THE MISTAKE HIGH GMAT SCORERS MAKE or  HOW BRAIN CHEMISTRY LOWERS YOUR GMAT SCORE