genbot
Forum Elite
Topics: 11
Posts: 490
| | 03/12/08 - 11:36 AM
0 of 1 forum leaders found this post helpful
| #1 |
Hi friends!
I'm here to discuss the often confused topic of 3 digit scores and their percentile correlation.
I started out with usual idea of the 2 digit score corresponding to the percentile score and the 3 digit score corresponding to probably the number of questions u got right on the test. But since then i have learned quite a few things about the correlations and have also made some inferences of my own. I also seem to have stumbled upon a new way of calculating ur percentiles once u know ur 3 digit score.
I'm here today to share this knowledge with u and hope it proves useful to all the uninitiated and the newbies.
First lets be clear on a few things which are often confused.
When u recieve ur USMLE score report they mention the following things;
Pass/Fail
2 digit score
3 digit score
MEAN and STANDARD DEVIATION for recent test takers (American and Canadian)
And a couple more things which we dont need to go into at this moment.
First of all the 2 digit score is NOT ur percentile score.
It is just a means of interpreting ur score in terms of a 2 digit number, where the passing grade is 75 and the highest attainable grade is 99. However all 99s are not equal 99s. Some are more equal than others.
A good guide to ur percentile score is ur 3 digit score.
While these days a 2 digit score of 99 starts with a 3 digit score of 236, it does not correspond to the same percentile as say a score of 260.
As common sense demands 260 should be a higher percentile than 236, and so it is.
This was all pretty basic stuff. Most u guys must already know this well.
Now comes the difficult part; how do u calculate ur percentiles using ur 3 digit score.
Most times i have seen people say and write that u use the MEAN and STANDARD DEVIATION mentioned on ur score report to calculate this.
They say if u have a score of say 250 while the mean for the test is 220 and the standard deviation is 20 then u go about calculating ur z-score and then look up the percentile in the z-score table (the z-score in this case would be 1.5 i.e the score is 1.5 standard deviations from the mean: 250-220/20=1.5).
The big flaw with this method is that it does not take into account the fact that USMLE scores are NOT NORMALLY DISTRIBUTED, and therefore, u cannot use a mean and standard deviation to calculate ur percentile score. Effectively this means that if for example the mean is 220 and the standard deviation is 20, while 1 standard deviation from the mean may equal 240 (which may not infact be the case, but we are only assuming here), 2 standard deviations from the mean may not necessarily equal 260 (and infact it does not, it is LESS than that!).
What i discovered while giving NBME mock tests is that they have corrected there scores for the skewed distribution of USMLE scores (i.e they have effectively NORMALIZED their scores).
This means that whenever u get a particular NBME score u can use the standard deviation and the mean to calculate ur percentile.
To illustrate this let me give u an example. But first let me tell u that the NBME tests have a mean score of 500 and a standard deviation of 100.
So if we have scores of 600, 650, 700, 750 and so on...u can calculate ur exact percentile by calculating the z-score for each. In this example they would be 1, 1.5, 2, and 2.5 standard deviations from the mean (the z-scores infact).
U can use this method for any score u achieve.
What i also found with the NBME scores was that they give a correlation to ur 3 digit score; so that while the mean corresponds to 217 (500), +1 standard deviation is 240 (600), +2 standard deviation is 256 (700), and +3 standard deviations is >265 (800).
As u can see the intervals are not uniform, which is exactly how it should be because the USMLE scores are not normally distributed!
So what these guys have effectively done is give us a table to calculate our percentiles!
The way u go about this is quite simple.
U locate ur 3 digit score on the NBME table and see what NBME score it corresponds to. Once u know ur corresponding NBME score u can calculate ur z-score (that is a score in terms of standard deviations from the mean).
And once u have ur z-score u look it up in a z-score table and u have ur percentile!
All very easy to do u see.
For example a score of 256 corresponds to 700 on the NBME and therefore 2 standard deviations from the mean. The percentile score using the z-score table in this case would be 97.7 i.e only 2.3 percent test takers have achieved a better score.
However, there is one small hitch to this (no system is perfect u see:rolleyes.
Apparently if we go by the currrent USMLE score reports the 2 standard deviation score may have gone up a little. And instead of 256 (as in 2004 when the NBME made up its correlations) it may currently be 260. This i infer from the fact that the present score reports mention that the most scores (which would mean +/- 2 standard deviations from the mean or 95% of scores) lie between 140 and 260.
But keeping everything in mind this system is probably as good as it gets, unless of course the USMLE guys sent u ur exact percentiles!
I just hope people find this easy to understand and have a happy time calculating their percentiles. It took me a long time figuring this out in its entirety but maybe u guys will now have it easy.
Good luck to u all and happy calculating!
Edited by genbot on 03/12/08 - 11:46 AM
___________________
Step1 267/99. Step2 269/99. Match 2009. MAY THE FORCE BE WITH ME (The one they call the strong nuclear force!)
|
| genbot
Forum Elite
Topics: 11
Posts: 490
|
Hey moderators how about makin this sticky?
___________________
Step1 267/99. Step2 269/99. Match 2009. MAY THE FORCE BE WITH ME (The one they call the strong nuclear force!)
|
| genbot
Forum Elite
Topics: 11
Posts: 490
|
Dont see a lot of activity here...
And I thought this was interesting!
  
___________________
Step1 267/99. Step2 269/99. Match 2009. MAY THE FORCE BE WITH ME (The one they call the strong nuclear force!)
|
| genbot
Forum Elite
Topics: 11
Posts: 490
|
This place is seriously serene and lonely
___________________
Step1 267/99. Step2 269/99. Match 2009. MAY THE FORCE BE WITH ME (The one they call the strong nuclear force!)
|
| commit
Forum Newbie
Topics: 2
Posts: 80
|
Good stuff, I like your theory. It makes a lot of sense to me. Did not know NBME score report give out so much information and a big secret.
Only problem to me, is how to define percentile if you score more than 265 or the current 269?
|
| genbot
Forum Elite
Topics: 11
Posts: 490
|
Yup thats a lil problem we have.
Not an unwanted one though
Every score >265 on step1 and a score >268 on step2 is beyond 3 SDs from the mean. Thats like 99.9 percentile!
I think we all should be happy with that and let it rest there.
Unless of course somebody comes up with some new idea.
But, seriously, for now i dont see how we can overcome this limitation.
Will give it a thought though...see what i can come up with (have plenty of time these days for such abstract thought processes!)
___________________
Step1 267/99. Step2 269/99. Match 2009. MAY THE FORCE BE WITH ME (The one they call the strong nuclear force!)
|
| genbot
Forum Elite
Topics: 11
Posts: 490
|
Nobody got any ideas to contribute???
___________________
Step1 267/99. Step2 269/99. Match 2009. MAY THE FORCE BE WITH ME (The one they call the strong nuclear force!)
|
| praying4a99
MS-III
Topics: 100
Posts: 2,428
|
Interesting, though I do not understand the obsession with the percentile score. Will we need it for something? Or you do it just for fun?
___________________
2008 Study Plan Discussion ..... My Personal Blog
|
| genbot
Forum Elite
Topics: 11
Posts: 490
|
Ya all this is pretty much for fun...i like playin around with numbers...now dont ask me why am I doc then!
___________________
Step1 267/99. Step2 269/99. Match 2009. MAY THE FORCE BE WITH ME (The one they call the strong nuclear force!)
|
| genbot
Forum Elite
Topics: 11
Posts: 490
|
And of course it clears up a whole lot of confusion people have.
___________________
Step1 267/99. Step2 269/99. Match 2009. MAY THE FORCE BE WITH ME (The one they call the strong nuclear force!)
|
| mukho
Forum Elite
Topics: 6
Posts: 299
|
very interesting...though by the end of ur post i had a rising panic attack just like used to when trying to solve biostats ques ...but good work!
|
| genbot
Forum Elite
Topics: 11
Posts: 490
|
Hehehe.
Very funny mukho! hehe.
Well its meant to serve two purposes, you know. Also gives you a biostat revision in no time at all!
But thanks if you found this useful.
Gives me encouragement to work on this in greater detail.
___________________
Step1 267/99. Step2 269/99. Match 2009. MAY THE FORCE BE WITH ME (The one they call the strong nuclear force!)
|
| lakshya_0_7
Forum Elite
Topics: 19
Posts: 240
|
hey genbot....
very very interesting indeed.....i had a q : how did u know that the USMLE scores arent normally distributed?
send this to the ECFMG so that they can post it on their website..... !
|
| hitesh
Forum Senior
Topics: 42
Posts: 160
|
I just read this post cos this had some activity and Man! I love it..lol
I have read a few other posts of yours and i really like the way you analyse things. I love to play with numbers too and all those famous sequences ..
you've definitely spent a lot of time doing this.. good job!
I'll be curiously waiting to see what's your take on people's score in step 1 and CK and their corelation and how the score of first step affects the second.. I assume you're working on that too, arent you?
|
| Boy_wonder
Forum Newbie
Topics: 1
Posts: 30
|
Good post genbot.
|
| genbot
Forum Elite
Topics: 11
Posts: 490
|
lakshya,
they gotta a graph of the score distributions, which if u see it is clearly negatively skewed, so this was a pretty simple thing to find out.
About sendin this to the ecfmg...well...i'm thinkin bout it.
hitesh,
ya i'm working on the thing. will be back with the analysis once i'm done with the CS.
And numbers sure are a lot of fun.
boy wonder,
I'm glad u liked the post. Thanks.
___________________
Step1 267/99. Step2 269/99. Match 2009. MAY THE FORCE BE WITH ME (The one they call the strong nuclear force!)
|
| perseverence
Forum Newbie
Topics: 1
Posts: 6
|
hey genbot,
great theory up there..do you think taking this exam with the American kids will be of benefit to the img's then...since the standard deviation may increase? (since american kids dont do too well first time around) ?? just wondering
|
| genbot
Forum Elite
Topics: 11
Posts: 490
|
hey perseverance, honestly dont really understand ur question.
but from wat i know anyway u take the exam its ur 3 digit score which would determine ur percentile not who u take it with.
and also the means and standard deviations the usmle score reports mention r always those of the american grads...nothin to do with those of the imgs.
the best thing anyways is to concentrate on preparin well and not worry bout the scores and the percentiles...thats how i feel it should be.
of course once u have the score u can have a lil fun with workin out where u stand...but thats very much post exam....this time its best to work hard and get urself exam ready...and good luck with that
___________________
Step1 267/99. Step2 269/99. Match 2009. MAY THE FORCE BE WITH ME (The one they call the strong nuclear force!)
|
|
| |
| | | | | | | | | | | | | | | | | | |
