arlete
Intern in 2009!!!!!
Topics: 30
Posts: 2,227
|
How can we estimate the wrong answers? I remembered only 6, but I bet there were many more than that!
___________________
When men make the rules, God decides the exceptions.
|
| new_n_lost
Politically InCorrect
Topics: 653
Posts: 6,074
|
Its a non linear scaling thingy.
The hard questions get more points if you got them correctly and lose less if you got them wrong. Similarly the easy questions dont have much points but if you got them wrong then you lose more points. For example There are 10 easy ones with total points of 6 if you got them all correct. Now if you got 1 wrong you get 5 points. 4 wrong then 3 points ( entirely dependent on teh type of question you got. Now there are 10 tough ones which have 7 points if you got all correct if you got 2 wrong you get 6 points.
Its entirely dependent on the type of questions you get.
What the seniors have always told me that easy questions get above teh 75-80 % score its the hard questions that decide whether you get the 99 mark or above it.
Again Moi 2 cents.
___________________
FORUM RULES-- Those who believe in telekinesis, raise my hand. I get enough exercise just by pushing my luck --P4U World.." The pure and simple truth is rarely pure and never simple."
|
| P4a99
Forum Fanatic
Topics: 34
Posts: 2,272
|
Till now I haven’t t dropped this point of view since I have heard it from p53.
___________________
2008 Step 1 Study Plan Discussion ..... I am a moderator not a source for download links. Please do not ask me for any.
|
| doctor123
Forum Senior
Topics: 12
Posts: 250
|
 eek:
if you guys wanna score 99, make sure you choosed wrong answer for 2 question in each block.
Dont mark correct all of them, you will get 100 percentile then.
I think this topic is a silly one. 
___________________
ALLAH; guide me to the Path of Success.
|
| arlete
Intern in 2009!!!!!
Topics: 30
Posts: 2,227
|
People want to know if they are well prepared before the test. But I think the score thread topic will help answering this.
___________________
When men make the rules, God decides the exceptions.
|
| deja_vu
Forum Senior
Topics: 4
Posts: 214
|
praying4a99 wrote:Two of you are 99ers. Can't you just estimate nearly approximately anywhere close how many answers you got wrong and subtract it from approximately 330 and give us an answer?  
Can't calculate exact number dear.
If you want an approximate number I may get one for you..
Since I started preparing for step-1, I had always marked those Qs where I was not sure of answer (which includes Qbooks,Qbanks,UW..etc..) And I had got ~50% (Range:40-60%) of them correct most of the times. This was a very consistent finding. I was really choosy in marking the doubtful Qs and I would always mark a Q if I had the slighest of doubt. So usually I never did Q wrong if I didn't mark it (of course there were some rare exceptions).
If I apply this rule to the real exams, I marked some 60 odd Qs in step-1 and 70 Qs in step-2. So that way I answered some 30Qs wrong in step-1 and 35 more in step-2.
Without any substantial confirmation, my perception has always been that, you get 99 if you get ~80% of Qs right.
(No offense but please don't start attacking that this topic is useless and no one can ever know...blah blah... THIS POST HAS BEEN WRITTEN JUST AS A REPLY TO P99s Q. He is interested in asking this kind of stuff and I am interested in answering)
|
| P4a99
Forum Fanatic
Topics: 34
Posts: 2,272
|
"this topic is useless and no one can ever know...blah blah.." WHO WOULD DARE AND SAY THAT ???!!! show him to me!!! 
Thanks for your reply
___________________
2008 Step 1 Study Plan Discussion ..... I am a moderator not a source for download links. Please do not ask me for any.
|
| arlete
Intern in 2009!!!!!
Topics: 30
Posts: 2,227
|
___________________
When men make the rules, God decides the exceptions.
|
| doctor123
Forum Senior
Topics: 12
Posts: 250
|
Well when forehead frowns andd eyebrows rise, then one should answer it.
99 is a percentile for sure. I read this myself.
Now consider Gaussian Distribution in your mind.
99 is a two digit score for those who are among top one percent of three digit scorers ( answering correct answers)
As all knows 189 is passing Three digit Score, ECFMG never released three digit cutoff for 99.
How they score? Well they compare particular performance to past performances and the standard set by them on the basis of record of examinees.
Now when they set limits , I dont know about this.
Any +ve feedbacks except forheads & eyebrows.
DR.123
___________________
ALLAH; guide me to the Path of Success.
|
| doctor123
Forum Senior
Topics: 12
Posts: 250
|
As far as difficult & easy qs concerned, I heard this
But kaplan says - all correct answers carry equal marks.
___________________
ALLAH; guide me to the Path of Success.
|
| deja_vu
Forum Senior
Topics: 4
Posts: 214
|
doctor123 wrote: 99 is a percentile for sure. I read this myself. DR.123
Really??!!
If 99 is percentile, how would you explain the fact that almost 20% test taker score 99?
And that way 75% people should fail the exam (75 is passing right????).
You need to read this..
http://www.usmle.org/General_Information/general_...
Moreover passing is not 189. 3-digit for 99 is 236 in step-1 nowadays. It changes from time to time.
They don't compare anyone to any other. They just compare us with the Qs and their answers.
|
| P4a99
Forum Fanatic
Topics: 34
Posts: 2,272
|
FA, Kaplan Q book and Dr. Daughtery said that it is not percentile nor percent. Please do not open this up again.
___________________
2008 Step 1 Study Plan Discussion ..... I am a moderator not a source for download links. Please do not ask me for any.
|
| doctor123
Forum Senior
Topics: 12
Posts: 250
|
@dejavu
Its about score percentile not people. When mean is 200 (3 digit score).
Anyway I wish u, mine & all others 99 and best of luck.
This will not make a penny diff. to any of us.
DR. 123
___________________
ALLAH; guide me to the Path of Success.
|
| arlete
Intern in 2009!!!!!
Topics: 30
Posts: 2,227
|
Well... whatever... we need to study the most and answer the best we can! 
___________________
When men make the rules, God decides the exceptions.
|
| new_n_lost
Politically InCorrect
Topics: 653
Posts: 6,074
|
Tone it down Guys. Either have this discussion like a proper Debate or if you guys want to thow punches then let me know cos i will start the betting ring for it
___________________
FORUM RULES-- Those who believe in telekinesis, raise my hand. I get enough exercise just by pushing my luck --P4U World.." The pure and simple truth is rarely pure and never simple."
|
| skm95
Forum Newbie
Topics: 0
Posts: 5
|
The 2-digit score USMLE provides is not based on any statistical distribution and it's neither percentile nor percent score. It is simply a linear representation of 3-digit score on a different scale.
On a 3-Digit scale UMSLE has defined two points, a minimum passing score and minimum 99 score. These days the minimum passing is 185, and minimum 99 is 236. USMLE keeps changing these points every few years. My assumption is that 185/236 also represent standard-deviation times some number on a 3-digit statistical distribution graph. For example if SD = 20 and mean = 215, then 185 = mean - 1.5 * SD, and 236 = mean + 1.05 * SD.
But regardless of what they represent, the 2-digit score can be calculated by following simple straight line-equation:
Y = (Y2 - Y1) / (X2 - X1) * (X - X1) - Y1
Where (X1,Y1) and (X2,Y2) are two points of a straight line.
Point 1 = (X1,Y1) = (185,75)
Point 2 = (X2,Y2) = (236,99)
So the equation can be simplified to:
Y = (99 - 75) / (236 - 185) * (X - 185) - 75
Y = (0.4715 * (X - 185)) - 75
Y = (0.4715 * X) - 87.5 - 75
Y = (0.4715 * X) - 12.5
With this equation we can compute a complete 2-digit table.
Anyone who has taken Step-1 recently can plug their number into this formula and verify if formula is correct. For those who want to do this calculation on spread-sheet like Microsoft-Excel, the function is:
=IF(A1 > 236, 99, ROUND((0.4715*A1-12.5),0))
**Cut/Paste function in A2 cell and enter a 3-digit score in A1 cell.
|
| P4a99
Forum Fanatic
Topics: 34
Posts: 2,272
|
Genbot please verify this. 
___________________
2008 Step 1 Study Plan Discussion ..... I am a moderator not a source for download links. Please do not ask me for any.
|
| smartkap
Forum Newbie
Topics: 0
Posts: 1
|
Hi, can anyone please tell me that at least how many questions should be correct to get a score a score above 240 on 3-digit or 99 on 2-digit in usmle. I have understood the relation between 3-digit and 2-digit score, but i'm not getting that how these scores are related to the number of correct questions? e.g. if one has got 200 as 3 digit score, does that mean that 200 out of total 350 questions are correct?
I will be glad if someone can help me. When I practice questions of different subjects at home, I usually get around 70% questions correct. If same thing happens in my exam, what will be my expected 3-digit and 2-digit score.
|
| genbot
Forum Guru
Topics: 13
Posts: 578
|
Hey dint know this thread was abuzz with activity all this while. Here I am after a long time...
skm95,
I would agree with u if the relationship between the two and the three digit scores was linear but i'm not sure it is.
And having checked this on some of the recent score reports it confirms my suspicion that we're not dealing with a linear correlation here...
But i guess its a pretty close approximation and is useful to some extent...
However i dont see the need to calculate one score from the other because u have both ones on ur score report.
The method used to calculate the 3 digit score surely relates to the number of questions u get right on the test, which however remains a lil difficult to estimate.
The best estimate of ur prep i guess would be the NBMEs...dont think there's a better predictor around.
My advice would be to check to ur scores on NBME anytime u need to assess ur level of readiness for the exam...there's no point finding out wat percentage of questions u need to answer correctly in order to get a certain score.
Stick to the basics...prep well...see how u doin on the standard Qbanks...and anytime u r in doubt check out the NBMEs.
deja vu,
I agree with u that everyone with a 260+ score on step1 is very likely to get a similar score on step2, and the same goes for a 99 on step1...again this is one of those non linear relationships. But yes as u mention somebody with 230 on step 1 is much less likely to get a 260 on step 2...so really the probability of u gettin a 260+ on step2 keeps increasing as ur score on step 1 keeps gettin bigger...but exactly how many will is not easy to calculate. But what we do know for sure is that the number cannot be higher than the number of guys gettin 260+ on step1. This is the absolute limit. Which would be not more than about 200 AMGs and about 60 IMGs. But this again I guess is an overestimation. Again instead of lookin at the overall numbers it would probably be best to look at how many such guys apply in each speciality to find out how competitive the speciality really is.
Hey i'm just goin on n on with my ramblings...i think this is it for now...hehehe
___________________
Step1 267/99. Step2 269/99. CS pass. ECFMG certified. Match 2009. MAY THE FORCE BE WITH ME (The one they call the strong nuclear force!)
|
| P4a99
Forum Fanatic
Topics: 34
Posts: 2,272
|
Interesting thanks.
___________________
2008 Step 1 Study Plan Discussion ..... I am a moderator not a source for download links. Please do not ask me for any.
|
| skm95
Forum Newbie
Topics: 0
Posts: 5
|
genbot,
I agree with you that there is no need to calculate 2-digit score as it is already on the score-card. I posted the line-equation to clarify the fact that this 2-digit score is neither percent nor percentile.
But the equation I presented is not an approximation, it is exact. I just picked 5 scores randomly from exam-score thread:
http://www.prep4usmle.com/forum/thread/71240/
http://www.prep4usmle.com/forum/thread/66639/
http://www.prep4usmle.com/forum/thread/67253/
http://www.prep4usmle.com/forum/thread/59842/
http://www.prep4usmle.com/forum/thread/68800/
All fit the straight-line. In mathematics, for a given set of (x,y) data points, the straight-line equation defined by 2 random (x,y) points will always pass through all of the other points.
For approximation, the best method will be to use differential-calculus or polynomial-regression. These two methods can provide with an equation which can predict a curve with very close approximation. But it will never be 100% correct. For example the relationship between NBME score and USMLE score, as defined by NBME here: https://apps.nbme.org/nsasweb/doc/sample_CBSSA.pd... is a curved line. If you pick any 2 numbers and try to compute straight line, it will most likely not pass through many other points.
BTW, one of the equations we can use for NBME-USMLE score relationship is (presented here as an excel spreadsheet function):
=IF(AND(200<=A1,A1<=800),ROUND((70.2-0.0000002067*A1^3+0.0000937*A1 ^2+0.30096*A1),0),"Enter a score in 200-800 range in A1")
This is a 3rd order polynomial. If you draw a graph, you will also notice that it follows Log n curve. Therefore curve equation can also be computed using Monte-Carlo method or inverse Laplace transform (since it is a transformation/mapping of one statistical distribution to another).
If you want to play with this equation by entering RAW NBME-scores to compute you USMLE score then use following:
=IF(AND(50<=A1,A1<=200),ROUND((70.2-0.0000002067*(A1*4)^3+0.0000937 *(A1*4)^2+0.30096*(A1*4)),0),"Enter a score in 50-200 range in A1")
**NOTE: Raw score conversion is little misleading, why? It is a topic for another time!
Enjoy!!
|
| genbot
Forum Guru
Topics: 13
Posts: 578
|
check it for 96/232
___________________
Step1 267/99. Step2 269/99. CS pass. ECFMG certified. Match 2009. MAY THE FORCE BE WITH ME (The one they call the strong nuclear force!)
|
| genbot
Forum Guru
Topics: 13
Posts: 578
|
It sure aint percentile or percent...but we need to have simple explanations for that...i think giving a core mathematical explanation wont help with resolving doubts people around here have...
As u said its a simple correlation of ur performance to a 2 digit score where 75 is the passing mark and corresponds to 185...and 99 corresponds to anything equal to or above 236...
The only way to guage ur percentile is by lookin at ur 3 digit score, which can give a rough approximation. I dont know if its possible to figure out a formula for the USMLE score distribution....I dont know if it would fit one...I dont know enough maths for that but I'm fairly certain it'd be very difficult to devise a formula for that...considering the relative random nature of the score distributions...from the curve that i've seen of the score distributions it seems to me that they do not fit any particular mathematical distribution...I may be wrong but thats my impression...
I had devised a method to check for ur percentiles once u had ur 3 digit score based on the correlations they have on the nbme forms...i have the thread somewhere here...
And tryin to find out how many answers u need to get right is next to impossible...there's so many variables u need to consider...we dont even know how many questions count...everyone seems to come up with a number of his own...
Eventually your performance on the test boils down to ur prep and thats all u should worry bout....but nevertheless this is interesting stuff in itself...atleast for souls like me.
___________________
Step1 267/99. Step2 269/99. CS pass. ECFMG certified. Match 2009. MAY THE FORCE BE WITH ME (The one they call the strong nuclear force!)
|
| skm95
Forum Newbie
Topics: 0
Posts: 5
|
I found your post to compute percentile score:
http://www.prep4usmle.com/forum/thread/68993/
very impressive, considering the fact that most IMGs, or even AMGs don't have deep mathematical background. In fact without knowing, you came very close to discovering how exams like USMLE are designed (concept of mapping one distribution to another).
Now let me try to explain how exams like USMLE are designed. What I am going to describe is based on general theory, and USMLE most likely does things differently.
Designing a good exam is a science by itself. A lot of statistical calculations go into finding the perfect question, answer choices, and score distribution. There are many statisticians who work along side field-professionals to design professional exams.
USMLE uses statistical normalization to compute and calibrate score for each question; yes! each and every question. That is also the reason, why you never know about maximum possible score. How does it work? I don't know how USMLE does it, but here is an example of how it can be done:
Suppose I am to start teaching a class in a college. To test my students I want to develop a question-bank from which I can give the final exam every semester. One of my goals is to keep adding new questions to this bank and keep removing the old ones. When adding the questions, I want to make sure that questions are not very-very easy and they are also not very-very difficult. My other goal is to keep the grading standard consistent. For this I define my own statistical distribution based on either some standard mathematical model or define a new one. My plan is to give 300 questions on final and for this I pick a distribution with mean M = 215 and standard-deviation SD = 20. I will call it SKM95 distribution.
Given all the parameters, now let's work on the problem in a simplest possible way.
I will start by giving a practice test to all my students every week. Let's assume that each question on the test is worth 10 points. After every practice test I will compute mean and standard-deviation for each question. So, let's say that for Question-1 (Q1) the M = 5.2 and SD = 1.5.
Based on M and SD of every practice test question, I can decide which questions are too difficult and which are too easy. I can then remove those from my final question-bank, e.g. if on particular practice test question all students get 10/10 or all get 0/10 then that question will not make to final, I will drop it from question-bank.
At the end of the semester I will have a full question-bank from which I can randomly pick 300 questions and give the final to my students. Let's assume Q1 is on the final. Now pick 3 students A,B, and C randomly. On the final A gets 10/10, B gets 0/10, and C gets 7/10 on Q1. Now I will compute z-value for these 3 students, for Q1.
Formula for z = (Score - M) / SD.
For A, z = (10 - 5.2) / 1.5 = 3.2
For B, z = ( 0 - 5.2) / 1.5 = -3.47
For C, z = ( 7 - 5.2) / 1.5 = 1.2
For a full 300 question final, the M and SD will be different per question and to compute final z-score I have to add z-values for all 300 questions and divide by 300.
For this example let's assume that final exam has only one question. So, now the next step is to compute final score by mapping this z-score to my SKM95 distribution, and that can be computed by using the same z-value formula, except in this case the unknown is Final Score, so the equation will be:
Final Score = M + (SD * z)
For A, Final = 215 + (20 * 3.2 ) = 279
For B, Final = 215 + (20 * -3.47) = 146
For C, Final = 215 + (20 * 1.2 ) = 239
You can see that even with 100% correct the Max score was 279 and not 300, and with 0% correct the minimum was 146 and not zero.
Now you know why no one knows the maximum score on USMLE, and since very question has different z-score therefore no one can answer the question, "how many questions I need to answer to get a 236/99?"
In this imaginary scenario I can also repeat some of the questions in next semester practice tests. For these questions the M and SD will have to be re-computed based on new sample size (previous semester class size plus next semester class size). It will skew the per question distribution curve slightly so before the next semester final I will have re-adjust my SKM95 mean and standard deviation (.e.g. I may have to move M from 215 to 214 and SD from 20 to 21, etc..), to keep the grading standard consistent.
Now you know why USMLE SD and M shifts over period of time.
In my make-believe world I can also add some 50 new practice test questions to final and increase the final to 350 questions and call these practice questions experimental questions. These 50 won't be counted towards final score and only I will know about these experimental questions (sound familiar!)
With this scoring system all the students can theoretically get a perfect 270+ on final exam. But probability of that happening is next to zero, why? Because in practice/experimental questions if all students get 10/10 or 0/10, then that question is thrown away. But on paper I can still claim that every one can get a perfect score ;-)
Now this was a very simple explanation of how a scoring system like SUMLE can be designed. In reality it is much more complex.
USMLE, most likely, repeats same experimental question for 1 year before converting it to a question that counts, therefore sample size is huge/10000+.
Each experimental question then goes through item-analysis. This is where some of the following functions are performed:
Calculate p-value. This is probability of getting this question right. On a five-response multiple choice question, optimum difficulty level is 0.50 for maximum discrimination between high and low achievers.
Calculate point-biserial correlation (PBC). This is how good students did on question compared to their over all test score. A highly discriminating question indicates that the students who had high tests scores got the question correct whereas students who had low test scores got it incorrect. Goal for USMLE like exam is to get PBC of 0.4 or more.
Calculate Reliability coefficient. Using Kuder-Richardson formula compute the degree to which a question measures a single cognitive construct. Goal is to get 0.9 or above for USMLE like exam.
Distracter Analysis. On a multiple choice question if A is correct answer then B,C,D, & E are distracters (wrong answers). Distracters should appeal to low scorers who have not mastered the material whereas high scorers should infrequently select the distracters.
Distribution skew. In most processional exams the distribution is always negatively skewed (to right), including USMLE exam.
OK! that's enough statistics for the day! Back to studying for Step-1!!!!!
|
| genbot
Forum Guru
Topics: 13
Posts: 578
|
Very informative post skm95.
Btw are u a mathematician or a doc??!!
___________________
Step1 267/99. Step2 269/99. CS pass. ECFMG certified. Match 2009. MAY THE FORCE BE WITH ME (The one they call the strong nuclear force!)
|
|
| |
| | | | | | | | | | | | | | | | | | | | | | | | | |
