Exam Results
Why Exam Results Take Weeks to Release
After months of preparation and studying, you most likely breathed a sigh of relief when you walked out of the testing center following your ABPMR exam. But after that moment you still wait several more weeks before you find out whether you passed or not.
If the board receives testing data from the exam administration company just days after the exam, why do official exam results still take weeks to come out? It's all about ensuring exam reliability, validity, and security.
Learn more about the process below.
Step 1: Raw Data
It's true — the board office receives raw data (a long string of As, Bs, Cs, and Ds) from our computer-based testing vendor (or our scantron vendor) within a week or so of an exam administration. But what then?
After downloading the data, our psychometrician (a doctor in the science of measurement) first formats the data to work in her software. Then she checks to ensure all candidates are accounted for, ensures the correct exam key is being used, and checks through the statistics on all exam items (questions).
Step 2: Key Validation
During that initial data review, the psychometrician flags individual items in the exam whose statistics are slightly "off." This can mean a variety of things, such as when very few candidates select the correct answer, the item does not discriminate between high- and low-ability candidates, or some other indicator that the key may be incorrect.
The key validation report is then sent to members of the exam committee to review those individual items. Those members review and provide direction for each flagged item, for example, to make changes to the exam key, delete items, or sometimes double key an item (if two answers are correct, for example). These changes are only rarely because of an actual error on the item, thanks to the rigorous item writing and review process; it's more likely that a change to the key is necessary because of a change in medicine or an updated reference since the item was last reviewed. Of course, sometimes the item and key are correct and it's just a difficult item.
Step 3: Scoring
After the exam key is validated, the ABPMR psychometrician reviews the pass/fail results and looks at individual results from an exam security standpoint (to evaluate for possible cheating).
Then, she writes the summary report including candidate statistics, exam statistics, pass rate, and exam validity and reliability, which includes considerations such as controlling for exam difficulty, or in the case of the Part II Examination, consistency of rating patterns for individual examiners and vignettes.
The summary report is sent to the entire exam committee for review and approval.
Step 4: Final Scoring
After the exam committee approves the examination summary report, the psychometrician performs final security checks, including comparing scores of candidates from the same residency training program, candidates who took the exam at the same testing facility, and other response pattern analysis designed to flag irregular behavior.
Following these final checks, results and score reports are uploaded to the ABPMR website, which candidates can access by signing in to the Exam Center.
Ensuring Fairness and Reliability —Several Times Over
At the ABPMR, certification is the main way we accomplish our mission of protecting the public. In addition to many steps taken prior to exam administration (in item writing and review, exam assembly, and more), there are necessary tasks after an exam is taken to ensure the exam was fair, valid, and reliable for assessing knowledge and skills in the specialty or subspecialty area being tested.
In addition to the above steps, multiple rounds of quality assurance checks are performed at each point in the scoring process. All of the analyses, reports, reviews, and approvals take time and careful consideration. By performing key validation, multiple rounds of scoring, and numerous quality checks, candidates can be assured: when you get your score report, it is a valid and accurate measurement of your knowledge and skills in the field of PM&R or one of our subspecialties.
Any questions about this process? Please let us know.
