November 09, 2020 - Research

Published: Influence of Sex and Age on Ratings of Confidence and Relevance in Continuing Certification Longitudinal Assessment - A Pilot Study

Summary by Carolyn L. Kinney, MD

In 2020, the American Board of Physical Medicine and Rehabilitation (ABPMR) introduced a longitudinal knowledge assessment (Longitudinal Assessment for PM&R, or LA-PM&R) for continuing certification. This assessment replaces the Maintenance of Certification (MOC) Examination, which had its final administration in October 2020.

LA-PM&R was designed to emphasize educational or formative objectives—an assessment for learning—based on what is known from research on adult learning. Shorter, more frequent assessments are offered, with repetition of targeted content to reinforce learning. Each question is coupled with learning material related to the relevant educational concepts.

Embedded in the LA-PM&R design is an algorithm to prioritize content to be delivered to learners, including question reattempts and clone questions. An obvious target for repetition is whether or not a learner answered the question correctly. Other ratings, such as whether the material is relevant to a physician’s practice, or whether the physician is confident in their answer, could also be considered.

The ABPMR performed a prospective quality improvement pilot study of 403 ABPMR diplomates prior to deciding to implement LA-PM&R for all ABPMR diplomates (see Robinson, et al. AJPMR 2020;99(5):420-23 for further description and results). In addition to answering the questions, participants rated each question for both relevance to their own medical practices and their confidence in their answer.

The pilot revealed a significant difference based on age and gender on how pilot participants rated their item-specific response confidence and the relevance of the question to their clinical practice. Women were less confident than men in their responses overall, regardless of whether or not their answer was correct. Younger physicians were more confident in their responses and rated items as more relevant than did older pilot participants.

Given the learning objectives of LA-PM&R, the results of this pilot study supported using item correctness and practice relevance, rather than ratings of answer confidence, in prioritizing delivery of content for spaced repetition in LA-PM&R. This algorithm has been incorporated into the current LA-PM&R design.

Read the abstract or download the full article, which has been published online ahead of print: Influence of Sex and Age on Ratings of Confidence and Relevance in Continuing Certification Longitudinal Assessment - A Pilot Study

Kinney, Carolyn L. MD; Raddatz, Mikaela M. PhD; Robinson, Lawrence R. MD Influence of Sex and Age on Ratings of Confidence and Relevance in Continuing Certification Longitudinal Assessment, American Journal of Physical Medicine & Rehabilitation: February 2021 - Volume 100 - Issue 2S - p S3-S6 doi: 10.1097/PHM.0000000000001635