Understanding the GMAT Scoring Algorithm

As you likely know, the GMAT is a computer-adaptive test (CAT), in which your score is calculated by an algorithm that provides you with harder questions (and higher score returns) when you answer previous questions correctly, and with easier questions (and lower returns) when you’ve answered previous questions incorrectly.

Through this method, the GMAT can ascertain your ability level in a relatively short period – 37 math and 41 verbal questions – and provide you with an immediate score upon completion of the test. On the flip side – and unintentionally – the GMAT can cause you a great deal of stress as you try to find ways to better understand and game its scoring system. To save you that stress, here are some important things you should know about GMAT scoring:

1) Good news: You can get a lot of questions wrong and still do well!

The job of the GMAT scoring algorithm is to determine your ability level by asking you questions that begin to close in on it. Think of how you’d play a game of 20 Questions as you attempt to zero in on the historical figure that your “opponent” has selected:

Was this person famous in the era BC? (No – too early)

Was this person famous before the Middle Ages? (No – still too early)

Was this person famous before the Declaration of Independence? (Yes – 1776 is too late)

Was this person famous before 1600? (Yes – 1600 is still too late)

Did this person become famous before 1500? (Yes – now we’re getting close to that period between around 1300-1500)

Was this person famous in the late 1400s? (Yes – now we’re getting close to really knowing the answer)

Was this person famous for something that happened in the 1490s? (Yes)

Is it Christopher Columbus in 1492? (Yes – once you get to the 1490s, you can be pretty sure that you’re talking Columbus. We’ve managed to narrow down our assessment of the figure in question by getting some “yes” and “no” answers)

Essentially, that’s what the GMAT is trying to do with the questions it feeds you. “Is this person above a 700? Yes.” “Is this person above a 750? No.” Because the test needs to get those “no” answers at the upper limit of your ability, it will continue to feed you harder questions that you will likely answer incorrectly as it tests your upper threshold, and at the lower end it will feed you easier questions to test your minimum ability.

The upshot for you? You’re supposed to answer a fair number of questions incorrectly. Everyone does. Akil over at BellCurves wrote up a pretty extensive analysis of official practice test scores that demonstrated some trends in the ways that scores are calculated. If you don’t want to sort through the dense analysis to draw your own conclusion, know this: those scoring in the 46-50 scaled score range on the quant section (the upper limit of high scores) answered between 21 and 26 of the 37 math questions correctly. You can score well above the 90th percentile on math and miss more than a dozen questions! (And if you’re good enough at math to do that, you’ll note that it equates to your missing roughly a third of the questions)

2) The first ten questions are no more important than the last ten.

The Graduate Management Admissions Council goes as far as to spell this out clearly in the Official Guide for GMAT Review and on its blog, but the rumors still persist. Let’s go back to the game of 20 questions; if it’s that easy for you or I to correctly guess “Christopher Columbus” in just 20 questions, why does the GMAT need 37 or 41 to determine your score? After all, GMAC is run by a sophisticated team of statisticians and psychometricians.

The answer? The GMAT needs to account for false positives and false negatives. In a game of 20 questions, it’s in unbelievably poor form to lie about the answer to a question, but on the GMAT you’ll “lie” about your ability whenever you guess correctly or make a silly mistake and answer incorrectly. The test needs to account for that, and so more questions are needed.

Going back to Columbus, say that the 20 questions proprietor had forgotten that catchy rhyme “In 1492 Columbus sailed the ocean blue”, and tried to estimate your line of time-sensitive questioning by thinking “Jamestown was in the early 1600s, so Columbus must have reached the Americas in the 1500s.” He’d have answered your question “before 1500?” as “no”. But your subsequent questions – “was he an explorer/sponsored by the Spanish crown/etc.” – would eventually bear out that the answer to the pre-1500 question was false.

The GMAT knows that you’ll have some false positives/negatives in your answers, and accordingly it’s equipped to filter through those. And, accordingly, it can’t make any major decisions about your ability level that early in the test. If you’re taking the GMAT soon, hopefully you’re studying probability concepts thoroughly enough to recognize this – 1 out of every 525 test-takers who guesses blindly on the first four questions will get them all right. The GMAT can’t hand that person a 700-ish score on the basis of pure guessing and then some mediocrity for the rest of the test. With well over 200,000 GMATs taken per year, the overseers of the test (statisticians to boot) know that they’ll see quite a few tests that include sets of 3-4 consecutive correct guesses, and the algorithm is set up to mitigate those and accurately reflect your scores.

3) Knowing how the scoring algorithm works neither significantly impacts your score nor excuses you from having to answer questions correctly!

Of the time that most examinees spend preparing for the GMAT, among the least-value-added is the time beyond that first, say, 20 minutes that they spend thinking about the scoring algorithm. Consider this quote from Dr. Eileen Talento-Miller, one of the chief psychometricians behind the GMAT, in her blog post about guessing on the exam:

“And because no one knows what item you would have gotten next if you don’t complete the items at the end, then there is no good way to estimate how that would affect your score.”

If the creators of the exam are willing to admit that they’re unsure how they’d game the test, it’s unlikely that you’ll somehow crack the code on your own. The psychometricians at GMAC are tasked by business schools with, above all else, the job of ensuring that the test is a valid assessment of a student’s performance. Quite frankly, they do that incredibly well, and so even if they did spot an opportunity to beat the system, rest assured that they’d identify and correct it before you would have the opportunity to use it.

The GMAT measures your score differently, but not entirely differently, than Omega measures the results of the Olympic 100 meter dash. Like GMAC, Omega has a job to ensure that the results of the race accurately reflect the performances of the athletes. Usain Bolt, the fastest man alive and defending Olympic sprint champion, could spend hours understanding the nuances of electronic timing, the ways in which a second is calculated, etc., but in the end he’ll only defend his championship and potentially lower his world record if he simply runs faster than everyone else. In fact, Omega and the international track & field governing bodies have even instituted a system that punishes runners
who try to beat the system. As it is scientifically understood as impossible for an athlete to react to the starter’s pistol in less than 0.10 second, any athlete who tries to time the start and begins his motion after the gun but before that natural reaction time has elapsed is charged with a false start (notably, 1992 Olympic champion Linford Christie was disqualified from defending his title in 1996 under this rule). When the stakes are high, those who perform the assessment of your performance have greater incentive to validate that numerical assessment than you have to beat the system.

Accordingly, like Bolt, you can spend your time much more effectively by preparing to succeed than by trying to fully understand the nuances of how you’ll be assessed. Ultimately, you should trust that the scoring system will accurately value your performance, and that the only thing you can control is that performance itself.

Photo courtesy of William Warby, under a Creative Commons license.

Tags:

Leave a Reply

Spam protection by WP Captcha-Free