See One, Do One, Forget One: Early Skill Decay After Paracentesis Training

Abstract

INTRODUCTION:Internal medicine residents perform paracentesis, but programs lack standard methods for assessing competence or maintenance of competence and instead rely on number of procedures completed. This study describes differences in resident competence in paracentesis over time. METHODS: From 2016 to 2017, internal medicine residents (n = 118) underwent paracentesis simulation training. Competence was assessed using the Paracentesis Competency Assessment Tool (PCAT), which combines a checklist, global scale, and entrustment score. The PCAT also delineates two categorical cut-point scores: the Minimum Passing Standard (MPS) and the Unsupervised Practice Standard (UPS). Residents were randomized to return to the simulation lab at 3 and 6 months (group A, n = 60) or only 6 months (group B, n = 58). At each session, faculty raters assessed resident performance. Data were analyzed to compare resident performance at each session compared with initial training scores, and performance between groups at 6 months. RESULTS: After initial training, all residents met the MPS. The number achieving UPS did not differ between groups: group A = 24 (40%), group B = 20 (34.5%), p = 0.67. When group A was retested at 3 months, performance on each PCAT component significantly declined, as did the proportion of residents meeting the MPS and UPS. At the 6-month test, residents in group A performed significantly better than residents in group B, with 52 (89.7%) and 20 (34.5%) achieving the MPS and UPS, respectively, in group A compared with 25 (46.3%) and 2 (3.70%) in group B (p < .001 for both comparison). DISCUSSION: Skill in paracentesis declines as early as 3 months after training. However, retraining may help interrupt skill decay. Only a small proportion of residents met the UPS 6 months after training. This suggests using the PCAT to objectively measure competence would reclassify residents from being permitted to perform paracentesis independently to needing further supervision.

Cohort study of hospitalists’ procedural skills: baseline competence and durability after simulation-based training

“…Most recently, a study of internal medicine residents trained in paracentesis skills showed that skill decay was mitigated if booster training occurred at 3 months rather than 6 months. 42 These two investigations Open access shed light on the impact of booster training on procedural skills in nonsurgeons but are imperfect as proxies for hospitalists, who are no longer trainees and have accumulated more experience. Insights from studies on monitoring hospitalist performance in non-procedural skills are also germane in concept, as they focus on collecting and reporting objective individual-level 16 and group-level metrics, 13 approaches applicable to procedural competence tracking.…”

“…Twenty-two of 38 eligible hospitalists participated. The average age was 36.5 (31)(32)(33)(34)(35)(36)(37)(38)(39)(40)(41)(42)(43)(44)(45)(46) years. Participants reported an average of 5.9 (range 1-14) years in practice.…”

Who Trains the Trainers?

“…Regardless of how rarely these cases present in the clinical setting, we know that (1) faculty must be prepared to handle them, especially for those practicing in community affiliated centers, and (2) if they do not continually practice these skills, they become vulnerable to procedural decay. 11,12 Strategies exist to help trainees achieve, maintain, and assess their procedural competency. These include procedure training sessions, just-in-time teaching sessions, and competency-based skills assessments before allowing performance in the clinical arena.…”

“…Faculty may go months to years between performing these procedures, as opportunities are infrequent and trainees often make the first attempt. Regardless of how rarely these cases present in the clinical setting, we know that (1) faculty must be prepared to handle them, especially for those practicing in community affiliated centers, and (2) if they do not continually practice these skills, they become vulnerable to procedural decay 11,12 . Strategies exist to help trainees achieve, maintain, and assess their procedural competency.…”

A Dynamic Risk Management Approach for Reducing Harm From Invasive Bedside Procedures Performed During Residency

“…4,5 Internal medicine (IM) program directors and residents commonly rely on counting a small number of procedures as a proxy for competence, as no reliable method exists for determining competence in procedural training. 6,7…”