LA-PMR
Longitudinal Assessment Pilot User Satisfaction Results Are In
When starting the longitudinal assessment pilot, the ABPMR set out to find out whether it could be a replacement for the MOC Examination and improve the assessment process based on several outcomes, including comparison of performance on the 10-year exam, performance on CertLink questions including clone questions, and user satisfaction.
As you can imagine, we have received a lot of feedback over the past several years on the secure 10-year MOC Examination; in fact, we began the pilot because we heard your concerns and set out to find a method of assessment that would be more valuable, relevant, and meaningful while reducing burden.
Just over 400 randomly selected diplomates agreed to participate in the CertLink pilot, which began last spring. Now, following a successful acceleration of the timeline, we have paused the pilot for data analysis. Today, we are releasing results for one outcome: User satisfaction.
Take a look at the highlights in the graphs below, and the bulleted list for more results.
99 percent of participants agreed or strongly agreed that the platform is easy to learn and use.
94 percent of participants agreed or strongly agreed that longitudinal assessment helped them identify gaps in their knowledge.
97 percent of participants agreed or strongly agreed that the critiques and references presented after each question (explanation of why the correct answer is correct, and citation) helped them learn.
92 percent of participants agreed or strongly agreed that longitudinal assessment will help them keep current.
93 percent of participants agreed or strongly agreed that using a longitudinal assessment process like CertLink adds value to their board certification.
"We've been optimistic about longitudinal assessment since before the pilot began, so to see the CertLink pilot participants affirm its value is very exciting," says Carolyn Kinney MD, Executive Director of the ABPMR. "We are still evaluating whether this method can match or improve upon the MOC Examination as an assessment of knowledge as well as a learning tool, but these user satisfaction results are very promising."
Additional results from the user satisfaction survey include:
- 84% agreed or strongly agreed that two minutes per question is an adequate amount of time (The average time users are taking to answer one question is 50 seconds)
- 98% agreed or strongly agreed that twenty questions per quarter is a reasonable number of questions (some participants requested more!)
- 87% agreed or strongly agreed that the question history and categories (features of the platform) are useful for learning
- 84% agreed or strongly agreed that the current customization system allows them to get questions that adequately reflect their practice
- 83% of participants estimate their total time spent on CertLink per quarter, including answering questions, reading critiques, etc., to be two hours or less (that's 8 hours per year)
The ABPMR Board of Directors is happy to see this vote of confidence in longitudinal assessment from the pilot participants who have used the platform for a full year. As the other data are analyzed to determine the success of the pilot, we will post more information here. Stay tuned!
