GMAT Summit Report: Learnings From The Graduate Management Admissions Council

Last week, representatives from Veritas Prep attended the biannual GMAT Summit, hosted by the Graduate Management Admissions Council (GMAC).  The following is a report, focusing on what you can learn that will help you on the GMAT.

In the 1980s there was E.F. Hutton (corporate ad slogan: “When he talks, people listen”).  In the 1990s, it was Alan Greenspan, whose mere words could shake the financial markets in nearly any direction.  In the GMAT industry, that “when he talks, people listen” voice belongs to Dr. Larry Rudner of GMAC, and Dr. Rudner last week hosted a full-day’s event in which he and other GMAC representatives offered valuable insight into the GMAT today, tomorrow, and beyond.  The highlights:

The GMAT is a REASONING test, not a knowledge test

Dr. Rudner spoke at length about the purpose of the GMAT and about its execution to serve that purpose.  One one presentation slide, entitled “What It Measures,” he explicitly wrote:

The GMAT exam measures…  Verbal reasoning, quantitative reasoning, and analytical writing skills.

Note the underline under the word “reasoning” – he was quite emphatic that the GMAT is concerned with higher-order thinking skills and not with mere memory or knowledge.  The slide went on to further read

The GMAT exam DOES NOT measure…

and included “knowledge of facts”, “undergraduate competency” and a few other items (related to leadership, emotional intelligence, etc.)

As we’ve said in this space for years, the GMAT is a unique test in that, by far, most others you have taken have been tests of knowledge.  The GMAT, on the other hand, is not a content-based exam, it’s a reasoning exam that tests your ability to think logically, efficiently, and intuitively.  Where this most notably sent shockwaves through some attendees at the conference was related to Sentence Correction.  Dr. Rudner did not “announce” but rather reiterated that the GMAT does not explicitly test idioms and that logical meaning of sentences is, indeed, fair game and testable, much to the consternation of those who have staked their claim on having the industry’s most comprehensive “idioms list”.  As we’ve noted here repeatedly, and as Dr. Rudner once again confirmed in no uncertain terms, the GMAT is not a test of idioms but rather a test of analytical and reasoning abilities.

Insight into GMAT scoring

Relevant to the above was a demonstration of how the GMAT is scored, a topic that we’ve covered in this space previously with an analogy that continues to hold as a convenient way to understand the scoring.  The GMAT’s adaptive engine first researches each question to find a “tipping point” at which examinees below a certain ability level on that section (quant or verbal) are extremely likely to answer incorrectly and examinees above that same level are overwhelmingly likely to answer correctly.  That makes that question a good item to differentiate at that level – a 65th percentile question, for example, will separate those above the 65th percentile (they almost always get it right) from those below that level (they almost always get it wrong).  This allows the adaptive engine to continue to refine its estimate of someone’s ability level by choosing questions that provide the most data to help correct that estimate over time.

A quick note on the scoring – the algorithm is self-correcting, so if you happen to miss a question beneath your ability level, the system will quickly realize as you get subsequent, lower-difficulty questions right, that it’s dealing with one awry data point.  The mission of the adaptive engine is to refine and correct that ability estimate throughout the exam to arrive at a quite-accurate assessment of your ability.

Now, consider this as insight into GMAT content and study strategy. That ability level estimate continues to refine within each section.  And you might receive, as your first 8 questions on the verbal section, one CR question, then one SC question, then one CR question, then a 5-question RC passage, at which point the computer will have a fair estimate of your ability level and choose an appropriate next question to help refine that estimate.  This process cannot work if Sentence Correction questions were as centered on idiomatic recall or grammatical knowledge as many examinees think that it is!!!!! RC and CR questions we know to be extremely reasoning-based, but for SC questions to participate fairly in that estimate of your ability they have to have a common thread with the others.  Otherwise the algorithm would continually receive false readings – someone excellent at 2/3 of the questions (those that deal with reasoning) would disproportionately miss the knowledge-based (they’re not knowledge-based!!!) SC questions and throw off the efficiency of the scoring.  This simply does not make sense.  So if you don’t believe Dr. Rudner and GMAC, believe the logic behind the scoring algorithm: it doesn’t make sense that a question type could disproportinately favor topic knowledge over reasoning!

Other takeaways regarding the scoring system:

  • As the scoring system constantly corrects itself over time, Dr. Rudner reiterated that the first 10 questions do not matter proportionally more than the others.  Even if you excel on the first 10 questions, the algorithm will “find you out” if you’re performing above your ability.
  • The computer’s estimate of your ability level (which becomes your scaled score via a lookup table) is multiplied by the proportion of questions you completed before it becomes a score on your report.  So if you only answered 34/37 math questions, your initial ability estimate is multiplied by 34 and then divided by 37 to form your “adjusted interim score”, at which point it is then converted via lookup table to a score on the 0-60 scale that keeps scores consistent with the historical versions of the GMAT.  What does that mean to you?  It’s important to pace yourself accordingly to attempt nearly all questions (if not all).  Now, remember that your ability estimate will be affected if you guess your way to several incorrect answers, so in those waning seconds you don’t have that great of a decision (high ability level * lower proportion or lower ability level *100% proportion), but know that your goal should be to manage time effectively – either way if you don’t, your score will suffer.
  • Questions are tested in the unscored, experimental phase to create that ability level data,and are also tested continually even while “live” on the exam to ensure that they do not disproportionately favor or punish a particular demographic group.  These questions are researched thoroughly to be fair, accurate predictors of ability level and can be removed at any time if they either fail to provide such an estimate OR if their usefulness for separating ability levels flattens out.  Many questions that you find in the Official Guide for GMAT Review series are accurate questions but fail to provide that separation tipping point – there’s no great point at which the probability of correctness jumps, so they’re not useful as live GMAT questions.
  • Perhaps most notably: Dr. Rudner revealed that, in his own recent GMAT attempt, he of all people lost track of  time in his pursuit of perfection, spending 5 full minutes on a math question and feeling that familiar twinge of panic because he needed to regain control of his pacing.  Lo and behold, he was able to see his own detailed scoring breakdown (his security clearance outranks yours significantly!) and, drumroll…  That question didn’t even count – it was one of the experimental, unscored items.  So from the horse’s mouth – don’t fall in love with any one question or become infatuated with perfection.  Pace yourself.

Next-Generation GMAT

As you likely know already, the GMAT will change gears slightly in June, 2012, replacing one of the AWA essays with a new Integrated Reasoning section.  GMAC released more information on the new section:

  • The new order of the GMAT will be:

    AWA – 30 minutes
    Integrated Reasoning – 30 minutes
    Break – 8 minutes
    Quantitative – 75 minutes
    Break – 8 minutes
    Verbal – 75 minutes

    Essentially, we now know that the Analysis of  an Argument essay will come first, followed by the Integrated Reasoning section, then the multiple choice sections as always.

  • GMAC will have new IR questions published and available for study at www.mba.com by December, with the official practice tests containing an IR section early next year.
  • The question types – Multi-Source Reasoning, Table Analysis, Graphics Interpretation, and Two-Part Analysis – that have been previously announced will remain the types of questions seen on IR, but the formats themselves (number of answer choices, pull-down menus vs. multiple-choice radio buttons, etc.) are all still in development.
  • GMAC expects that it will take at least 2-3 admissions cycles for MBA programs to use the IR scores as a primary method of candidate evaluation, as schools will want to see for themselves the validity of IR scores to student success and preparedness.  But schools will begin seeing IR scores in the fall of 2012 as applications roll in and can consider them as they wish.
  • Speaking of admissions and scoring, the IR section will receive a separate score that does not factor into your 200-800 “overall” score, so on your score report you will see an AWA score, an IR score, a quantitative scaled score, a verbal scaled score, and an overall (quant/verbal combination) score.  Official scores will still be sent to schools within 20 days.

New Products Coming From GMAC

GMAC announced that the 13th edition of the Official Guide for GMAT Review will be released in April, 2012 with new questions and a new Integrated Reasoning section available on computer as a companion to the Guide.  In the same time range, GMAC will update the GMATPrep practice tests and the GMATWrite software tool.

Miscellaneous/Editorial

Most striking at the GMAT Summit, at least to your author, were the transparency and earnestness of  the GMAC representatives.  In an ever-competitive marketplace (the GMAT now competes to some extent with the GRE, accepted at some MBA programs; we were presented with information regarding the new Pearson Test of English which competes with the TOEFL as a language-proficiency exam), the reps from GMAC are eager and almost giddy to show the world how valid and effective its test is as a predictor of student success.

GMAC wants to be transparent to you, too, with a cadre of official preparation materials available at its online store, and a helpful social media interface (on Facebook and the Official GMAT Blog at www.mba.com).  What you gain from using these resources and, in our case, from being lucky enough to meet with the people behind the GMAT, is an appreciation for a group that stands true to its mission.  They take pride in a fair, valid assessment; they want to offer examinees the information and resources that they need to prepare for success; they value their position as “gatekeepers” to business school but do not want to be the decision-makers themselves, preferring to simply report to member schools how correlated the GMAT is to success in school and allow schools to set their own criteria.  One comes away from a meeting like this with an appreciation for the GMAT as a well-written, well-administered exam and for its staff as smart, earnest folks who are passionate about their role and product.

Getting ready to take the GMAT soon? Take a look at our GMAT prep courses and start your journey toward a great score. And, as always, be sure to find us on Facebook and follow us on Twitter!

Leave a Reply

Spam protection by WP Captcha-Free