kafo
Forum Newbie
Topics: 4
Posts: 12
|
Hey everyone
I'm posting for my friend who has just came back from step 1 and you know what?? She is crushed!! She was very good student and has been preparing for step 1 for long time.
She scored on NMBE1: 620 and NMBE2: 640, Qbank 1st/2nd trial: 75%/90%
I guarantee that she was well prepared. The problem is that she was out of time on 4 blocks (didn't answer abou 10-15 questions) because of long long questions, a lot of calculation etc. Excluding the reason why she didn't try the shots on these question I wonder if she still has chances to score high?? Do results depend on the level of difficulty of the test like I feel that she had very difficult and problematic set of questions (so far I thought that everybody has more or less the same level of difficulty??) Is three-digit score just a number of corect answers or something more??? This girl definitively deserve for high scores (even 99 in two-digit scale).Be so kind and answer my post.
Regards
|
| p53
Forum Guru
Topics: 51
Posts: 804
|
http://www.prep4usmle.com/forum/comments.php?id=3...
___________________
"Everything should be made as simple as possible, but not simpler." - Albert Einstein
|
| adeelmd
Forum Elite
Topics: 40
Posts: 399
|
Harder questions are not given higher points on the test. There are different tests and each test is graded according to it's relative difficulty. On some tests, getting 60% of the questions right is passing and on others getting 70% of the questions right is passing. All questions are equal. That's why a lot of people who cry their test was hard end up getting >250 because they had to get a fewer percentage of questions right to get such a high score. I called up NBME and confirmed this back when I gave step 1, so I am sure.
Edited by adeelmd on 04/30/06 - 12:25 AM
___________________
where i lay my head is home.
|
| p53
Forum Guru
Topics: 51
Posts: 804
|
Maybe you are right - but at the same time this means that even though for each person all given questions are equal and worth the same, there is difference in how much each question counts between diferrent tests, i.e. for different persons.
This means that they have taken each test, counted the number of difficult, in the middle and easy questions, giving to each different points, then have calculated the mean point for each question, i.e. have equalized each question in the given test and now each question worth the same for 2 persons with the same percent of correctly answered questions needed to pass, let's say, for example, 63%, but the questions worth different for 2 persons in case for one 61% is enough and for other 69% is needed - because in the first test the number of difficult questions is more than in the second test, no matter examinee has answered these difficult questions or not; am I right?
I think this scoring method is much more unfair - if someone has answered correctly mostly easy questions in his/her test, he/she will get the same number of points (i.e. score) as someone who has correctly answered the same percent, but mostly much more difficult questions, given that the mean points for these 2 tests are equal (questions worth the same on these 2 tests), i.e. they need the same percent of the correctly answered questions to get the same score.
and people from ECFMG confirmed that they are working using this system of scoring?
I guess I'll also call them or send an e-mail to them - let's see what they'll say.
anyway, thank you, adeelmd - let's discuss this subject more to find where is the truth - it's important!
Edited by p53 on 04/30/06 - 04:35 AM
___________________
"Everything should be made as simple as possible, but not simpler." - Albert Einstein
|
| kafo
Forum Newbie
Topics: 4
Posts: 12
|
Thanks guys for trying to find the clue. p53 is right-it's important.
|
| adeelmd
Forum Elite
Topics: 40
Posts: 399
|
ecfmg doesn't have any jurisdiction over this. Call up the nbme and they will let you know. I think the beginning of kaplan qbank hints to the fact that all questions are equal.
I think your analyses is a bit too complicated. Instead of thinking that usmle is a qbank that randomly pulls questions, I think they probably have 200 or so different tests. Some questions will overlap in those tests, but the relative amount of questions that have been gotten right has been predetermined through prior test-taking perfomance. The relative performance of a test (or form as they like to call it) can easily be judged by comparing to a series of cohorts (I think this is partly why we have a month delay in results). This is why you need to get 60-70% of questions right to pass, a set percentage is not given because there is no universal set percentage.
It is unfair... It is very hard to get a great score on an "easy" form, and a "hard" form can freak out someone during the test. Knowing this will actually make you happy when you see impossible hard questions on transgenic mice. I actually started freaking out during my test because I thought the questions were too "easy"... but it worked out fine. In the end, go into the test trusting your correlations...people's feelings are often wrong.
___________________
where i lay my head is home.
|
| kafo
Forum Newbie
Topics: 4
Posts: 12
|
Hi Everybody
Today my friend got her score : 234/94 it's not 99 but still not bad. What do you think?? Is it true that for program directors step 2 scores are more important ??
|
| tolito
Forum Fanatic
Topics: 119
Posts: 2,158
|
from the little i know, the exam is going to be adaptable. they say they will not let us know when they change to adaptable. i wonder why. but it is going to happen soon if it has not yet happened.
___________________
It has been a looooong hard journey but I am inches away from my destination...
|
| kafo
Forum Newbie
Topics: 4
Posts: 12
|
Tolito, what do you mean?? 'adaptable" like after correct answer next question will be more difficult etc. ?? I hope you're wrong...
|
| kulmur
Forum Senior
Topics: 1
Posts: 28
|
i think the installation of the FRED software was the test going adaptable......
|
| p53
Forum Guru
Topics: 51
Posts: 804
|
guys, why do you think that the test is already adaptive? have recent examinees had clear feeling that it's adapting after each block?
___________________
"Everything should be made as simple as possible, but not simpler." - Albert Einstein
|
| tolito
Forum Fanatic
Topics: 119
Posts: 2,158
|
KAFO, exactly. ie the first block determines your level of competence. it is advised that one should do his/her best in the first block so you can get 'difficult' q which would ensure a high score. however, through out the test, the computer keeps on assessing you. the problem is if you are started up on a low skill level, you may still get taken a step or two higher but you run the risk of not gettting your best marks. that is why they advise that the first block is the most important block. unfortunately, i cannot remember the source but it was from usmle or ecfmg. they say they will not tell us when the tests becomes adaptive.
___________________
It has been a looooong hard journey but I am inches away from my destination...
|
| sachida
Forum Guru
Topics: 54
Posts: 438
|
All, the test is NOT adaptive at this point. I have taken one in form of GRE.
It feels different, you can feel the change so vividly.
At present, the whole exam is created at the begining and does not change.
There was a student at my test center who entered his id number and computer failed to create test. they asked him to schedule for another date.
Also, majority of student just like me also felt the block 4-5-6 are usually most difficult. No 8 is not. if it was truely an adaptive test- last blocks are most difficult.
To me, this exam is not adaptive at present.
___________________
Doubt is a luxury we cant afford. Future is predetermined by the character of the people who shape it
|
| adeelmd
Forum Elite
Topics: 40
Posts: 399
|
The test is not adaptable, it will never be adaptable. there are two simple reasons for this...
1) The USMLE is meant to be a standard... ie a 99 scored today is equal to a 99 scored 10 years ago.... changing the scoring method will change the standard
2) It will create a scoring nightmare and become too difficult to grade... Let alone the amount of code you would have to write, the grading system, etc...
all questions count equally, and they will never move to an adaptable test. People that have taken the test can tell you that after a block, the next one starts immediately.... if the test were adaptive, it would at least take the time to upload your results, and then create a test (like it does in the beginning of the test and after the last block)
It's when you start thinking that the test is adaptive, or when you think its impossibly hard... that you start to freak out and loose valuable points and time. Like I said before, be sure of the correlations with nbme/qbank and forget how you feel--- "feelings" are often VERY wrong on step 1, and can actually decrease your performance.
___________________
where i lay my head is home.
|
| p53
Forum Guru
Topics: 51
Posts: 804
|
adeelmd, can't agree with you:
1. first of all, I don't think that changing the test to adaptive will change the standard - test isn't going to be more difficult or more easy with this change, it will be just adapting to every individual examinee; furthermore, even if standard will change, I can't see the reason why they would refuse to do so - USMLE is already making modification to the performance levels required to pass one or more of the CS subcomponents, i.e. they are changing the standard for step 2 CS - see http://www.prep4usmle.com/forum/thread/34844
2. I don't think that with this change test will become too difficult to grade - it's computer software, the program which is making this process, not the human being.
again, I can't agree with you that all questions count equally now - you should provide a reliable source to prove this claim - I know that the major reason why there are experimental questions on the USMLE is that they are grading the level of difficulty of these questions according to the performance status of examinees, and are using these graded questions for the assessment of future examinees; another reason to think that questions don't count equally is the range of the percent of correctly answered questions necessary to pass the exam - 60-70% and not the fixed percent, for example, 65%.
I also think that USMLE isn't adaptive yet, but I can't agree that it will take much time to upload your results, and then create a test - I have taken TOEFL and GRE (both are adaptive tests) and haven't noticed any delay.
to conclude, I can't agree with the statement 'it will never be adaptable' - I'm quoting from 'First Aid for USMLE step 1':
'Eventually the NBME plans to implement computer-adaptive sequential testing (CAST) in order to customise "the difficulty of the USMLE to the proficiency of each examinee across various stages of the examination." Essentially, this means that an "adaptive" boards exam will use the examinee's performance on a block of questions to determine the difficulty of the block that is presented to him or her next. This will allow the exam to assess proficiency accurately using fewer questions.
The NBME adds, however, that although some examinees may receive more difficult blocks of questions, "...the score you achieve will not be affected by the difficulty of the question blocks selected for you." The NBME goes on to claim that "every examinee will be tested on equivalent content."
After an initial CBT trial period, the NBME plans to change the exam to a CAST format, but implementation has been delayed indefinitely.'
Edited by p53 on 06/05/06 - 09:48 PM
___________________
"Everything should be made as simple as possible, but not simpler." - Albert Einstein
|
| adeelmd
Forum Elite
Topics: 40
Posts: 399
|
All this information is from 2004, so it might just be out of date but my sources were....
well all questions counting equally and just different tests being marked differently I verified by calling up the nbme ... you could call and ask them that; i think the number is on the website
Experimental questions are either completely thrown out or kept in depending on how many people answered it wrong/right. if it is in a certain range, (ie. 20-80% then it is sent to the question pool) if not, the question is deleted.
As far as it never becoming adaptive is what the director of medical testing for Kaplan, Dr. Daugherty told me when I asked him. He's the one who wrote the test-taking strategies portion of the qbook and the behavioural/biostats book. He gave me the same reasons I said above. He was quite adamant about the fact that "Every year there is a rumor that the test is adaptive, and I can assure you it isn't adaptive, and it never will be".
step 2 cs is a new test, and therefore the standard is still being set.
I don't get the first aid quote... they say there are implementing cast, but it wont effect your score... so then what is the point? Moreover, it would be impossible to make a test adaptive and not have it affect outcome.
___________________
where i lay my head is home.
|
| p53
Forum Guru
Topics: 51
Posts: 804
|
adeelmd
'calling up the nbme ...'
- you know that it's not reliable source - surely, the person you talked with may not be in charge to give the information that isn't otherwise publicly and officially available.
'Experimental questions are either completely thrown out or kept in depending on how many people answered it wrong/right. if it is in a certain range, (ie. 20-80% then it is sent to the question pool) if not, the question is deleted.'
- I guess that's only your assumption - again, could you provide the reliable source to prove it?
'I can assure you it isn't adaptive, and it never will be.'
- I think that the director of medical testing for Kaplan isn't in charge to make such claims - Kaplan's commercial course has no power to define or predict the future policy of the NBME/USMLE.
I can't agree that step 2 cs is a new test - it was given for many years under the name CSA and in 2004 they only changed it's name, not the content.
'so then what is the point?'
- the point here is simple - I'm quoting FA again:
'This will allow the exam to assess proficiency accurately using fewer questions.'
'The NBME claims that "this process of customizing the test difficulty to each examinee's proficiency increases the overall accuracy of the scores and any related pass/fail decisions."'
So, CAST will allow the test to be:
1. shorter (did you like taking the test for 8 hours? I didn't);
2. more accurate.
Edited by p53 on 06/06/06 - 03:46 AM
___________________
"Everything should be made as simple as possible, but not simpler." - Albert Einstein
|
| luckyluck
Forum Newbie
Topics: 1
Posts: 15
|
hi ..thanx 4 the info...but can u just clarify 4 me that is it the absolute scoe or the relative score...n does towards the passing score each correctly answered q will amount to 1.5 times to the 3 digit score(as said by FA) n does my performance is influenced by how others perform...plz reply
|
| p53
Forum Guru
Topics: 51
Posts: 804
|
what do you mean under the absolute score or the relative score?
FA says the opposite:
'near the failure threshold, each three-digit scale point is equivalent to about 1.5 questions answered correctly.'
your score isn't influenced by how others perform, but your percentile, calculated from your score is influenced.
___________________
"Everything should be made as simple as possible, but not simpler." - Albert Einstein
|
| luckyluck
Forum Newbie
Topics: 1
Posts: 15
|
absolute score is the score is purely on the exam content ...n relative score is how did i preform while compared to others n does my 3 digit score reflects as to no. of correct answers on 350..plz reply
|
| p53
Forum Guru
Topics: 51
Posts: 804
|
USMLE will povide 2-digit score and 3-digit score - neither of them are relative scores - you can judge about how did you perform compared to others only by percentile - this can be calculated from your 3-digit score using mean and SD - see http://www.prep4usmle.com/forum/thread/32973
___________________
"Everything should be made as simple as possible, but not simpler." - Albert Einstein
|
| wey2002
Forum Senior
Topics: 4
Posts: 123
|
I took test on June,2
I feel all blocks are of the same difficulty level.
I surely got a mixture of easy and tough Qs in a given block.
The level of difficulty of all blocks is the same to me.
I felt really scary when I read this post before my test.
Later I told myself I have to got this test good, what matters is my attitude and hard work, it is not the format of the test.
However, this kind of information is not good for most test-takers, this serves as a distraction that will lead them to focus on the format instead of study.
Unless anyone of us gets true information from NBME or ECFMG, we should not draw our own conclusions based on estimations to scare others.
Will do good!
|
|
| |
| | | | | | | | | | | | | | | | | | | | | | |
