Boston-based John Hopkins research found a combination of step-by-step checklists and a rigorous error tracking system may serve as the best tool for assessing orthopedic residents' surgical abilities, according to Health Leaders Media.
In the study, three John Hopkins orthopedic specialists assessed 23 Johns Hopkins medical residents using the grading system as they tested their skills on cadavers. The residents ranged from their first year of residency to their fifth year.
Researchers used the Objective Structured Assessment of Technical Skills to evaluate skills, and modified the checklist to apply to shoulder surgical procedures. Under OSAT, residents received a zero if they performed steps out of order, and received one point if they successfully completed a step. Additionally, researchers analyzed residents' skills based on Global Rating Scale, which takes a comprehensive view of the surgical process and gives residents comments on areas, such as whether they used instruments correctly.
Comparatively, researchers tested skills based on a pass/fail system in which residents obtained a failing grade if they made a huge error, such as severing a blood vessel.
Here are seven things to know:
1. While procedural checklists may serve as a sufficient tool to asses residents' technical skills, they often fall short in they don't evaluate quality, said Dawn LaPorte, MD, orthopedic surgery professor at the Johns Hopkins University School of Medicine and a study author.
2. In the existing system, formal feedback for orthopedic surgeon residents on their motor skills or errors is few and far between. The lack of formal feedback allows mistakes to consistently occur. The error tracking system analyzes residents' skills when they are performing procedures on cadavers, which allows them to improve upon their mistakes.
3. OSATS and the Global Rating Scale provided residents good and objective ways to evaluate resident performance, the study found.
4. On the other hand, the pass/fail system provided "unambiguous feedback," researchers concluded.
5. Residents with more training obtained higher OSATS and Global Rating Scale scores than residents in their first year.
6. In the study, researchers noted 11 cases where residents damaged the nerves or veins, with first- and second-year residents being accountable for nine of the 11 mistakes.
7. However, investigators determined none of the three evaluations sufficiently captured those 11 mistakes, although the pass/fail evaluation did come closest as residents received a zero if they cut a nerve or major vessel. However, the pass/fail test does not tell residents the specific nature of their mistakes, the researchers noted.
More articles on practice management:
Will the 2017 CMS payment update mean more independent practices?
How 4 possible Medicare modifications affect beneficiaries: Spending increases & reductions
Colorado Springs Orthopaedic Group, OBERD partner for registries, data collection & PQRS reporting — 4 observations