How to review mock CATs

Everyone tells you that the value in taking mock CATs lies in analyzing them, and not just in mechanically taking them. How exactly does one analyze a mock CAT? Here is a simple guide

Check your percentile.
The starting point for any mock analysis is the percentile. Not because it is very relevant, but because it is impossible to ignore. So, go ahead, knock yourself out. Check the section-wise percentiles, stare long and hard at the numbers, extrapolate the percentiles from the last 3 mocks and see where they are going, imagine how high your overall percentile would be if only you could produce your best section performances in the same mock, give in to curiosity, triumphalism and envy by figuring out others' percentiles.

Do all this to get the damn thing out of the system. Now, ignore it and focus on something that can actually be useful. There are three parts to this -

1. Mocks have the best questions, learn how to solve all of these
Mocks are often seen merely as a tool for assessment and bench-marking rather than as a tool for learning. Mock CATs usually have the best questions, the ones that have an elaborate 6-minute solution while also having the elegant 1-minute solution. So, make it a habit to review the questions that have gone wrong, questions that you have skipped and then the ones that you got right as well. Very often, students ignore the ones they have gotten correct. If all your attempts were through the best approaches, chances are you would have attempted 5 more in each section. There is a learning angle to every mock CAT. Do not ignore this. As this article says, the mock CATs have quite a few Now-you-know questions.

This is why it is very important to pick a mock CAT provider who focuses on providing detailed solutions and helps with the thought process. In other words, have a look at the 2IIM mocks before you go looking elsewhere. :-)

2. Which 3 get kicked out, which ones get in?
After every mock, do a simple exercise to improve decision-making. Select at least three attempts from each section that you should have skipped, and replace these with three you should have attempted. In your first few mocks, you might even be able to select 5-6 questions in each section. The big gains in mocks come from improved decision-making; and you have to take a conscious effort to improve this. If you can reach your point where you cannot find more than 1 question in each section that you had incorrectly chosen to attempt, you can count yourself ready for the exam.

3. Topic-wise and timing-wise analysis
You should ask yourself a few questions similar to the ones given below -
How good is the hit-rate in Sentence Rearrangament? Was it worth doing the DI even after taking 15 minutes, did I at least get all 4 correct? Should I completely skip Sentence Elimination questions from now on? Did I get a Permutation Combination question wrong again? Should I completely dump this topic?

On the timing front, figure out when the fatigue errors creep in. All of us find that there are 2-3 questions where we cannot really fathom why we marked that stupid wrong choice in the first place. This is essentially down to fatigue. There is a spell of 15-20 minutes where not much gets done and errors creep in. Locate this spell, see when it usually happens and reduce it methodically.

On the timing front, it is important to know which type of question gets you the best marks/minute. You might easily attempt 3 RC passages accounting for 12 questions within 30 minutes. But if you get 6 of these wrong, that is effectively same as attempting only one RC and getting all questions correctly. Four correct answers in 30 minutes is a very poor return.

The truth about Mock CATs and percentiles

Whenever you take a mock CAT offered by any provider and see a percentile score, you should bear in mind that most of these percentile numbers are from 'look-up tables'.

In the first decade of this millennium, mock CATs had a good run. Back in 2000/01, you could announce an All-India mock CAT series and expect to fill large colleges across cities with eager students trying to figure out where they would 'rank'. However, 2009 marked a turning point in proceedings. The number of people taking the exam started declining, the number of people preparing seriously also fell. This also coincided with the exam going online with the tests being conducted across a 'testing window'.

The exam providers have tried gamely to keep pace. They have all devised online mock CAT series that can be taken anywhere and at any place, while also offering 'proctored exams' for the more serious-minded souls. All the percentile numbers run of the data collected from proctored exams. Now, the proctored exam suffers from two serious issues - 1) Sample size and 2) Representativeness. Far far fewer students take proctored mock exams than the numbers that used to take the All-India mocks of yore. And these far fewer ones that take it are the ones that are ultra serious about CAT. So, instead of having a database of 20,000 students across the spectrum, we are probably dealing with a sample size of 2000 very serious candidates. The look-up tables are supposedly based on the scores in the 'proctored' exams, but they have to be 'adjusted' a little to account for the sampling bias.

So, effectively we have guesstimates of numbers for various percentiles. This word guesstimate is a classic MBA word. It is a combination of guess and estimate, both of which standalone leave you with a feeling that some arbitrariness is involved. "Guesstimate" however fills one with confidence and leaves one with the joyful feeling that one's percentile is in good hands. The guesstimation process probably follows some strand of what is seen below.

So, where does this leave us?

Dont sweat the exact percentile numbers too much. Take the mock CATs seriously so as to learn based on them. And keep in mind that the term All-India Percentile is very similar to World series baseball.

Pick the mock CAT series that is most representative of the actual CAT. By this I mean how close is it to the actual exam. Take a free mock test and make your judgement. "How many people take the mock exam?" is an irrelevant question these days. The big providers will tell you that 20,000 people take their mock CATs. They might not be interested in telling you that only about 10% of these took it in proctored settings from which they made hajaar adjustments to arrive at a look-up table.

In any case, all these inputs are for the second mock CAT series that you should be thinking about. Anyone preparing for CAT should definitely take the test series provided by 2IIM. :)

Mock CATs, Mock CAT percentiles, predictions etc

I am going to start with a series of quotes.

There are two ways of lying. One, not telling the truth; the other, making up Statistics. 

There are lies, damn lies, and statistics.

Statistics are used much like a drunk uses a lamppost: for support, not illumination.

Replace Statistics with Percentile analytics and this would fit in perfectly in the context of CAT. No matter how many fancy charts are shown, it is the responsibility of the students to not over-interpret the results. When you take a mock CAT, the review happens on three planks -

1) Learn better methods by looking at the solutions
2) Think about how you could have improved your attempts by selecting better questions.
3) Evaluate your percentile.

Of the three, the least useful is no 3 (probably also the most harmful). Because it is the least reliable. Almost all mock CAT providers use look-up tables to give percentile scores. Gone are the days where a mock CAT provider could get 10,000 people to beat themselves into a frenzy on a given Sunday to take a mock CAT at 100 god-forsaken locations in the Country.

These days, proctored CATs probably have a sample size in the range of 1% of CAT takers. These are likely to be the most serious, most experienced, most-driven of the CAT takers. So, the test providers "adjust" the look-up table to factor this in. In layman terms, this boils down to - they look at the actual numbers, feed in some other numbers, and pat themselves on the back.

So, the percentiles are fairly unpredictable. And contrary to what all the service providers claim, there is no way to back-verify that database. The only ones who will call a coaching institute and tell them that they scored a decent percentile are the ones who got a decent percentile. So, the reverse-verification process is bogus. So, no one actually knows whether any percentile number has a serious correlation with the actual numbers. There is way more anecdotal evidence to say that the percentile numbers are erratic, than to say that they are accurate.

I recently heard that there was a percentile predictor that predicts percentiles based on numbers you feed in based on the scores you have got on other exams (some of my students called me to give me an update on their predicted results.). I almost fell from my chair laughing. Students who believe that there is a scientific basis for this kind of predictor should re-examine their brains. This level of gullibility cannot be good for the body.

The thought process behind the predictor is as follows - Take three not scientifically proven indicators with no established proof from back-verification, define arbitrary weightages based on your genius to these, come up with a fourth metric and call it a prediction. And supposedly students are flocking to this predictor engine. No wonder soothsaying, tarot card reading are all flourishing industries in our Country. If smart, ambitious people can be made to believe there is such a thing called a percentile predictor, god save the masses.

Anxiety does strange things to the mind. Fretting and fuming about this exam makes you do all kinds of over-analysis. The mind needs something to cling on to, to feel that you are on the right track. Ergo, predictor.

Think of all the guys who have completed their MBAs from IIM A, B, C and think about how each of them would react if you said "percentile predictor" in their midst. I will give you a hint. I spoke to a bunch of them and the only common factor was laughter.

So, what should we do?

I am going to say something that is difficult to implement. The last 2-3 weeks, you must focus on three things

1. Chuck the percentiles. Seriously, chuck them. Forget about them. Banish them.

2. Focus on still being curious and not getting sucked into this pressure vortex. The first 2-3 mocks that you would have taken, you would have enjoyed grappling with the questions. Slowly, the pressure takes over and you stop having fun in the mocks. You are too worried about the score (and that damned percentile). The ones who crack this exam are the ones who are likely to look at a question and go "Wow. Super question. I think I will enjoy solving this". If that joie de vivre is gone, you should rekindle it.

3. Mentally prepare for a paper that is far simpler than your mocks. In our Country, the service providers have a feeling that to create a great mock CAT, you must make it very tough. So, they create phantom tough questions. CAT does not do that. Lot of students panic in a paper because it is way easier than all the mocks they have done. They have difficulty adjusting to the fact that they can attempt 34 questions in Quant in CAT, whereas they never managed more than 22 in any mock. Finding a paper that is easier than mocks can disorient you. You end up shooting for too less, end up adjusting too late, and end up with a just-miss scoreline.

We at 2IIM have resisted every urge to make our mocks tougher and fought tooth-and-nail to keep it representative. This is why good students can sometimes attempt 75-80 questions in our mocks. Remember, the very best ones can and will attempt that many in actual CAT. So, it is very important to keep in mind that the level of difficulty will not be outlandish. If you are really good, but have never attempted more than 60 questions overall in a CAT paper, how will you adjust on the day of the exam to have a crack at 84 questions?

Try your best to have some fun in the paper. be very zen and forget the percentile, and go in thinking "this exam is going to be easy". Best wishes for CAT.

CAT - New Format: How does this affect preparation?

A video on how to plan for the CAT in its new avatar of 100 questions in 170 minutes.

Strategies for taking Mock CATs: