Consensus on Training and Assessment of Competence in Performing Chorionic Villus Sampling and Amniocentesis: An International Delphi Survey

Abstract

<b><i>Introduction:</i></b> The aim of this study was to obtain expert consensus on the content of a curriculum for learning chorionic villus sampling (CVS) and amniocentesis (AC) and the items of an assessment tool to evaluate CVS and AC competence. <b><i>Methods:</i></b> We used a 3-round iterative Delphi process. A steering committee supervised all processes. Seven international collaborators were identified to expand the breadth of the study internationally. The collaborators invited fetal medicine experts to participate as panelists. In the first round, the panelists suggested content for a CVS/AC curriculum and an assessment tool. The steering committee organized and condensed the suggested items and presented them to the panelists in round 2. In the second round, the panelists rated and commented on the suggested items. The results were processed by the steering committee and presented to the panelists in the third round, where final consensus was obtained. Consensus was defined as support by more than 80% of the panelists for an item. <b><i>Results:</i></b> Eighty-six experts agreed to participate in the study. The panelists represented 16 countries across 4 continents. The final list of curricular content included 12 theoretical and practical items. The final assessment tool included 11 items, systematically divided into 5 categories: pre-procedure, procedure, post-procedure, nontechnical skills, and overall performance. These items were provided with behavioral scale anchors to rate performance, and an entrustment scale was used for the final overall assessment. <b><i>Conclusion:</i></b> We established consensus among international fetal medicine experts on content to be included in a CVS/AC curriculum and on an assessment tool to evaluate CVS/AC skills. These results are important to help transition current training and assessment methods from a time- and volume-based approach to a competency-based approach which is a key step in improving patient safety and outcomes for the 2 most common invasive procedures in fetal medicine.

Validity evidence supporting clinical skills assessment by artificial intelligence compared with trained clinician raters

“…The clinicians were instructed and trained in using an assessment instrument that had been developed in a previous Delphi study involving international CVS experts. 24 The final assessment instrument included 11 items that were scored using five-point Likert scales. In the context of our simulated setting, two of the items were omitted as they were irrelevant to the test set-up (sample preparation and handling, and documentation).…”

“…After each rating, they discussed their scores until obtaining consensus for each item. Five items were selected based on 24 The training took place in Lobe (Microsoft). Thereafter, the models were exported and applied for each participant using Python (3.7.9) with libraries in OpenCV and NumPy.…”

“…Items and scale anchors were selected and agreed on by a large expert panel using the Delphi method. 24 Three out of nine items failed to discriminate between novices and experts (general preparations, F [2, 39] = 0.9, p = 0.44; identification and consent, F [2, 39] = 0.2, p = 0.81; and preparation of site and instruments, F [2, 39] = 1.3, p = 0.29) (Table 3). Internal consistency for the remaining six items was high (Cronbach's α = 0.89).…”

“…Previous studies using similar assessments of ultrasound skills have shown good generalisation between contexts with reliability estimates that correspond to those obtained in our study. 24,26,[32][33][34]…”

“…To assess if our simulation-based test set-up was sufficient to establish a reliable measure of participants' skills in performing the CVS procedure, we compared the content of our test set-up with curricular content deemed relevant by an international panel of CVS experts. 24 Our test set-up represented three out of three major topics listed under 'Technical skills: ultrasound' and three out of six major topics listed under 'Technical skills: sampling'. Technical skills, such…”