The truth about Mock CATs and percentiles

Whenever you take a mock CAT offered by any provider and see a percentile score, you should bear in mind that most of these percentile numbers are from 'look-up tables'.

In the first decade of this millennium, mock CATs had a good run. Back in 2000/01, you could announce an All-India mock CAT series and expect to fill large colleges across cities with eager students trying to figure out where they would 'rank'. However, 2009 marked a turning point in proceedings. The number of people taking the exam started declining, the number of people preparing seriously also fell. This also coincided with the exam going online with the tests being conducted across a 'testing window'.

The exam providers have tried gamely to keep pace. They have all devised online mock CAT series that can be taken anywhere and at any place, while also offering 'proctored exams' for the more serious-minded souls. All the percentile numbers run of the data collected from proctored exams. Now, the proctored exam suffers from two serious issues - 1) Sample size and 2) Representativeness. Far far fewer students take proctored mock exams than the numbers that used to take the All-India mocks of yore. And these far fewer ones that take it are the ones that are ultra serious about CAT. So, instead of having a database of 20,000 students across the spectrum, we are probably dealing with a sample size of 2000 very serious candidates. The look-up tables are supposedly based on the scores in the 'proctored' exams, but they have to be 'adjusted' a little to account for the sampling bias.

So, effectively we have guesstimates of numbers for various percentiles. This word guesstimate is a classic MBA word. It is a combination of guess and estimate, both of which standalone leave you with a feeling that some arbitrariness is involved. "Guesstimate" however fills one with confidence and leaves one with the joyful feeling that one's percentile is in good hands. The guesstimation process probably follows some strand of what is seen below.









So, where does this leave us?

Dont sweat the exact percentile numbers too much. Take the mock CATs seriously so as to learn based on them. And keep in mind that the term All-India Percentile is very similar to World series baseball.

Pick the mock CAT series that is most representative of the actual CAT. By this I mean how close is it to the actual exam. Take a free mock test and make your judgement. "How many people take the mock exam?" is an irrelevant question these days. The big providers will tell you that 20,000 people take their mock CATs. They might not be interested in telling you that only about 10% of these took it in proctored settings from which they made hajaar adjustments to arrive at a look-up table.

In any case, all these inputs are for the second mock CAT series that you should be thinking about. Anyone preparing for CAT should definitely take the test series provided by 2IIM. :)

Labels: , ,

IIM - CAT Coaching: Experts' Insights: The truth about Mock CATs and percentiles

Wednesday, May 27, 2015

The truth about Mock CATs and percentiles

Whenever you take a mock CAT offered by any provider and see a percentile score, you should bear in mind that most of these percentile numbers are from 'look-up tables'.

In the first decade of this millennium, mock CATs had a good run. Back in 2000/01, you could announce an All-India mock CAT series and expect to fill large colleges across cities with eager students trying to figure out where they would 'rank'. However, 2009 marked a turning point in proceedings. The number of people taking the exam started declining, the number of people preparing seriously also fell. This also coincided with the exam going online with the tests being conducted across a 'testing window'.

The exam providers have tried gamely to keep pace. They have all devised online mock CAT series that can be taken anywhere and at any place, while also offering 'proctored exams' for the more serious-minded souls. All the percentile numbers run of the data collected from proctored exams. Now, the proctored exam suffers from two serious issues - 1) Sample size and 2) Representativeness. Far far fewer students take proctored mock exams than the numbers that used to take the All-India mocks of yore. And these far fewer ones that take it are the ones that are ultra serious about CAT. So, instead of having a database of 20,000 students across the spectrum, we are probably dealing with a sample size of 2000 very serious candidates. The look-up tables are supposedly based on the scores in the 'proctored' exams, but they have to be 'adjusted' a little to account for the sampling bias.

So, effectively we have guesstimates of numbers for various percentiles. This word guesstimate is a classic MBA word. It is a combination of guess and estimate, both of which standalone leave you with a feeling that some arbitrariness is involved. "Guesstimate" however fills one with confidence and leaves one with the joyful feeling that one's percentile is in good hands. The guesstimation process probably follows some strand of what is seen below.









So, where does this leave us?

Dont sweat the exact percentile numbers too much. Take the mock CATs seriously so as to learn based on them. And keep in mind that the term All-India Percentile is very similar to World series baseball.

Pick the mock CAT series that is most representative of the actual CAT. By this I mean how close is it to the actual exam. Take a free mock test and make your judgement. "How many people take the mock exam?" is an irrelevant question these days. The big providers will tell you that 20,000 people take their mock CATs. They might not be interested in telling you that only about 10% of these took it in proctored settings from which they made hajaar adjustments to arrive at a look-up table.

In any case, all these inputs are for the second mock CAT series that you should be thinking about. Anyone preparing for CAT should definitely take the test series provided by 2IIM. :)

Labels: , ,

0 Comments:

Post a Comment

Subscribe to Post Comments [Atom]

<< Home