Reliable and Valid Assessment of Clinical Bronchoscopy Performance

Konge, Lars; Larsen, Klaus; Clementsen, Paul Frost; Arendrup, Henrik; Buchwald, Christian von; Ringsted, Charlotte

doi:10.1159/000330061

Cited by 49 publications

(47 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The variance due to performance of different cases (test-retest reliability) was considerably larger; this was expected, as we used consecutive patients of differing difficulty. This finding underlines the importance of assessing multiple procedures to reach a reliable judgment regarding the competence of a trainee [13]. The decision study showed that several feasible combinations of the number of assessors and the number of procedures resulted in acceptable generalisability coefficients (figure 3).…”

Section: Validity Evidence For the Assessment Toolmentioning

confidence: 74%

“…The assessment tool was developed by a group consisting of two respiratory physicians, a thoracic surgeon, and a professor of medical education, all with considerable experience in performance, teaching, and validation of endoscopic ultrasound and other procedures [13][14][15]. The tool was designed according to the original format for "objective structured assessment of technical skills", in which each item is rated on a scale from 1 to 5, with descriptive anchors in the middle and at the ends, and re-coded into a score from 0 to 4 points [16].…”

Section: Development Of the Assessment Toolmentioning

confidence: 99%

See 1 more Smart Citation

Simulator training for endobronchial ultrasound: a randomised controlled trial

et al. 2015

Self Cite

View full text Add to dashboard Cite

Endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA) is very operator dependent and has a long learning curve. Simulation-based training might shorten the learning curve, and an assessment tool with solid validity evidence could ensure basic competency before unsupervised performance.A total of 16 respiratory physicians, without EBUS experience, were randomised to either virtual-reality simulator training or traditional apprenticeship training on patients, and then each physician performed EBUS-TBNA procedures on three patients. Three blinded, independent assessor assessed the video recordings of the procedures using a newly developed EBUS assessment tool (EBUSAT).The internal consistency was high (Cronbach's α=0.95); the generalisability coefficient was good (0.86), and the tool had discriminatory ability ( p<0.001). Procedures performed by simulator-trained novices were rated higher than procedures performed by apprenticeship-trained novices: mean±SD are 24.2±7.9 points and 20.2±9.4 points, respectively; p=0.006. A pass/fail standard of 28.9 points was established using the contrasting groups method, resulting in 16 (67%) and 20 (83%) procedures performed by simulatortrained novices and apprenticeship-trained novices failing the test, respectively; p<0.001.The endobronchial ultrasound assessment tool could be used to provide reliable and valid assessment of competence in EBUS-TBNA, and act as an aid in certification. Virtual-reality simulator training was shown to be more effective than traditional apprenticeship training. @ERSpublications Virtual-reality simulation-based training shortens the learning curve for endobronchial ultrasound (EBUS)

show abstract

Section: Validity Evidence For the Assessment Toolmentioning

confidence: 74%

Section: Development Of the Assessment Toolmentioning

confidence: 99%

Simulator training for endobronchial ultrasound: a randomised controlled trial

et al. 2015

Self Cite

View full text Add to dashboard Cite

show abstract

“…The WABIP Bronchoscopy Education Project [25] implemented in 2010 has successfully expanded a multidimensional educational program that includes assessment tools, training videos, a BronchAtlas video series, a collection of Fundamentals of Bronchoscopy PowerPoint lectures, patient-centered practical approach exercises, and The Essential Bronchoscopist series of books and training manuals 8 developed by Bronchoscopy International. Courses and faculty development programs have been held all over the world with the assistance of certified 8 Bronchoscopy Education Project by Bronchoscopy International is licensed under a Creative Commons Attribution-NoncommercialShare Alike 3.0 United States License.…”

Section: New Developments In Education Simulation and Computer Gamingmentioning

confidence: 99%

“…Courses and faculty development programs have been held all over the world with the assistance of certified 8 Bronchoscopy Education Project by Bronchoscopy International is licensed under a Creative Commons Attribution-NoncommercialShare Alike 3.0 United States License. Fig.…”

Section: New Developments In Education Simulation and Computer Gamingmentioning

confidence: 99%

Simulation in bronchoscopy training: are we there yet?

Colt

2012

Curr Respir Care Rep

View full text Add to dashboard Cite

The time has come for bronchoscopists around the world to recognize that simulation is the best alternative to learning on patients. Only in this way will patients no longer suffer the burden of procedure-related learning. Technical skills can be acquired, practiced, and improved by use of inanimate models and combinations of low and high-fidelity simulation, including increasingly portable computer-based trainers. Validated assessment tools and checklists can be used to measure a learner's progress towards competence, complementing assessments during direct observation and real patient experience. Simulation scenarios should include more than technical skill exercises, to encompass the full range of procedural practice. As simulation is increasingly incorporated into training programs, there is a need to use business ethics to enhance innovation and democratize new technology. Simulation in bronchoscopy training: we are not there yet, but we are certainly moving in the right direction.

show abstract

“…Our group has developed and validated two assessment tools for assessing competence in bronchoscopy [20,21]. The first tool is designed to be administered on a virtual-reality bronchoscopy simulator and the second to be used in a clinical setting.…”

Section: Introductionmentioning

confidence: 99%

Establishing Pass/Fail Criteria for Bronchoscopy Performance

et al. 2011

Self Cite

View full text Add to dashboard Cite

Background: Several tools have been created to assess competence in bronchoscopy. However, educational guidelines still use an arbitrary number of performed procedures to decide when basic competency is acquired. Objectives: The purpose of this study was to define pass/fail scores for two bronchoscopy assessment tools, and investigate how these scores relate to physicians’ experience regarding the number of bronchoscopy procedures performed. Methods: We studied two assessment tools and used two standard setting methods to create cut scores: the contrasting-groups method and the extended Angoff method. In the first we compared bronchoscopy performance scores of 14 novices with the scores of 14 experienced consultants to find the score that best discriminated between the two groups. In the second we asked an expert group of 7 experienced bronchoscopists to judge how a borderline trainee would perform on each item of the test. Results: Using the contrasting-groups method we found a standard that would fail all novices and pass all consultants. A clear pass related to prior experience of 75 procedures. The consequences of using the extended Angoff method were also acceptable: all trainees who had performed less than 50 bronchoscopies failed the test and all consultants passed. A clear pass related to 80 procedures. Conclusions: Our proposed pass/fail scores for these two methods seem appropriate in terms of consequences. Prior experience with the performance of 75 and 80 bronchoscopies, respectively, seemed to ensure basic competency. In the future objective assessment tools could become an important aid in the certification of physicians performing bronchoscopies.

show abstract

Reliable and Valid Assessment of Clinical Bronchoscopy Performance

Cited by 49 publications

References 60 publications

Simulator training for endobronchial ultrasound: a randomised controlled trial

Simulator training for endobronchial ultrasound: a randomised controlled trial

Simulation in bronchoscopy training: are we there yet?

Establishing Pass/Fail Criteria for Bronchoscopy Performance

Contact Info

Product

Resources

About