A square peg in a round hole? Defending the CUSUM and LC‐CUSUM tests

Quantifying Trainees' Competence in Upper Endoscopy

“…In response to an analysis of LC-CUSUM, Biau stated that ''to ensure the validity of CUSUM assessment of competence, it will be important to come to an agreed, expert review as to what constitutes success and failure at any specific skill'' (11). Critical to LC-CUSUM application is the setting of thresholds of acceptable success rates.…”

“…As it derived from a continuous monitoring foundation, it is probable that repeated evaluations may be needed. Biau further stated that ''if the CUSUM assessment is to provide a measurable impact, trainees must be observed, guided, and formally assessed by a senior colleague to ensure that performance flaws are accurately identified'' (11).…”

A Fellow Assessment Tool for The Pediatric Gastroenterology Entrustable Professional Activities

“…

impact, trainees must be observed, guided, and formally assessed by a senior colleague to ensure that performance flaws are accurately identified'' (11).In a separate analysis, Barraclough et al found that age of patient, outpatient versus inpatient, and attendance at a skills course were independently associated with procedural completion, suggesting that more complex analysis may be fruitful. It may be of value to know whether instructor variables had any influence on trainee competency.The small proportion of the subjects who achieved competency at the proposed threshold (42/79), the self-reported nature of the database and the validity of D2 intubation alone as a marker of competency limit the applicability of the numerical results from this study for widespread use in Pediatric Gastroenterology training programs.

…”

The educational value of using cumulative sum charts

“…It also seems plausible that a chart that encourages help-seeking behaviour or further practice in a skill might improve performance. The use of CUSUM charts as a tool for learning has been the subject of recent debate [10][11][12][13][14] and, until research in this area has been conducted, the educational community is likely to remain appropriately sceptical.The aim of this study was to assess the impact of using CUSUM charts during learning on performance of a procedural skill. Specifically, we aimed to: 1 test the hypothesis that final year medical students who are given access to CUSUM charts perform better than those without such access in a test of automaticity of peripheral venous cannulation skill; 2 estimate the effect size of using CUSUM charts on final year medical students' performance in a test of automaticity of peripheral venous cannulation skill.…”

“…It also seems plausible that a chart that encourages help‐seeking behaviour or further practice in a skill might improve performance. The use of CUSUM charts as a tool for learning has been the subject of recent debate [10–14] and, until research in this area has been conducted, the educational community is likely to remain appropriately sceptical.…”