Utility of Interobserver Agreement Statistics in Establishing Radiology Resident Learning Curves During Self-directed Radiologic Anatomy Training

Türeli, Derya; Altaş, Hilal; Cengic, Ismet; Ekinci, Gazanfer; Baltacıoğlu, Feyyaz

doi:10.1016/j.acra.2015.07.004

Cited by 10 publications

(15 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, with the rapid technical development within the field of radiology it must be argued that continuous learning throughout the career is required. However, a problem with radiology education is the scarcity of measurable data with regard to the learning process during residency (14).…”

Section: Pedagogical Aspectsmentioning

confidence: 99%

“…Studies on learning curve effects in radiology usually focus on identifying a specific diagnosis or anatomical structure on certain examinations (14,77,78,80). Most studies include either relatively few reviewers or training examinations.…”

Section: Pedagogical Aspectsmentioning

confidence: 99%

“…In addition, different outcome measures are used to assess improvements in diagnostic ability over time, including sensitivity/specificity, ROC-analysis and interreader agreement, making direct comparison among studies more difficult. There appears to be a general agreement that the amount of training is an important factor to achieve a high diagnostic accuracy (77,78,81), while the effects of previous radiologic experience on the ability to learn new radiologic methods have not been established (14,80). A few studies have shown a positive effect on the review time, but not on diagnostic accuracy (78,80).…”

Section: Pedagogical Aspectsmentioning

confidence: 99%

“…The rapid development in radiology in response to the current technical development requires a continuous training throughout the career among radiologists. However, the expertise on how to train radiology residents is limited (14) and the availability of continuous learning programs for radiologists even more so. These aspects are of great importance when introducing new diagnostic methods.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Do radiologists detect chronic thromboembolic disease on computed tomography?

et al. 2019

View full text Add to dashboard Cite

Background: Acute pulmonary embolism (APE) is a potentially severe medical condition with blood clots obstructing the pulmonary arterial vasculature. In most cases the APE resolves without any sequelae after anticoagulation therapy. In some patients, however, the emboli do not resolve upon treatment and the remnants cause increased vascular resistance, a condition known as chronic thromboembolic pulmonary hypertension (CTEPH). Both APE and CTEPH have a non-specific clinical presentation and imaging is an important part of the diagnosis. In APE computed tomography pulmonary angiography (CTPA) is the diagnostic gold standard, although the method is not suitable for all patients. CTPA has a high specificity for CTEPH, but the sensitivity remains under debate. At present CTPA is not recommended as a first line test among patients with a clinical suspicion of CTEPH. Purpose: To investigate unestablished imaging modalities in the diagnosis of APE (Study I) and CTEPH (Study III) including learning aspects (Study II) and knowledge (Study IV) of theses among radiologists. Regarding APE we studied magnetic resonance imaging (MRI) and in CTEPH we studied CTPA. Material and methods: Studies I-II were based on a prospective collection of 70 unenhanced MRI exams with CTPA as the gold standard. In Studies III-IV we used a retrospective material based on 43 CTPA exams from patients with confirmed CTEPH referred for presurgical assessment at a specialist centre, with a matched control with suspected APE. Results: All MRI exams were of diagnostic quality. Specificity was 100% for both readers and sensitivity 90% and 93% respectively with a nearly perfect inter-reader agreement (kappa 0.97) (Study I). Residents interpreting the MRI exams within the training program reached a clinically acceptable level after approximately 50 examinations and review time was halved during the training program (Study II). The sensitivity for CTEPH on CTPA reviewed by two experts was 100% and the specificity 100% (Study III), while the sensitivity based on the original reports from the same cases was 26% (Study IV). Conclusions: Unenhanced MRI has a high sensitivity and specificity for APE (Study I) and residents can learn to interpret such exams by using a self-directed training program (Study II). Enhanced CTPA has a high sensitivity when reviewed by experienced radiologists (Study III), but among radiologists in general the sensitivity is low (Study IV). LIST OF SCIENTIFIC PAPERS I. Detection of pulmonary embolism using repeated MRI acquisitions without respiratory gating: A preliminary study.

show abstract

Section: Pedagogical Aspectsmentioning

confidence: 99%

Section: Pedagogical Aspectsmentioning

confidence: 99%

Section: Pedagogical Aspectsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Do radiologists detect chronic thromboembolic disease on computed tomography?

et al. 2019

View full text Add to dashboard Cite

show abstract

“…[19][20][21] However, diagnostic skill, defined as a high level of accuracy with high Kappa values when compared to an expert, requires both theoretical knowledge and practical training with calibration. 22,23 We found only one study evaluating training for GDPs in diagnosing CCAAs in PRs and it included only one participant. The study showed that after completing a training package sponsored by the American Academy of Oral and Maxillofacial Radiology, the GDP had a high sensitivity but low specificity when diagnosing CCAA.…”

mentioning

confidence: 99%

Long‐term skill improvement among general dental practitioners after a short training programme in diagnosing calcified carotid artery atheromas on panoramic radiographs

Gustafsson

Ahlqvist

Jäghagen

2018

Eur J Dental Education

View full text Add to dashboard Cite

Purpose To study general dental practitioners (GDPs) ability to detect calcified carotid artery atheromas (CCAAs) in panoramic radiographs (PRs) and if their diagnostic accuracy in long term is improved after a short training programme. Methods Fourteen GDPs had their diagnostic accuracy regarding CCAA in PR assessed at baseline, 2 weeks and 1 year after training. Comparison was made with a reference standard based on consensus results from two experienced oral and maxillofacial radiologists. At each session, 100 radiographs were assessed individually by the GDPs. After the baseline assessment, the GDPs participated in a 2‐hour training programme comprising a lecture and diagnostic training by calibration. The GDPs results before and after training were compared, as well as between follow‐up sessions. Results A significant improvement in diagnostic accuracy was observed with increased sensitivity (from 41.8% to 55.7%, P = 0.02) without a significant decrease in specificity (from 87.2% to 86.7%, P = 0.87). The Kappa values also increased (from 0.66 to 0.71, P = 0.04). At 1‐year follow‐up, the improvement compared to baseline remained significant. There were no significant changes between the 2‐week and 1‐year follow‐up assessment. Conclusion A short training programme can significantly and sustainable improve GDPs diagnostic accuracy regarding CCAA.

show abstract

The learning curve in bladder MRI using VI-RADS assessment score during an interactive dedicated training program

et al. 2022

View full text Add to dashboard Cite

Objective The purpose of the study was to evaluate the effect of an interactive training program on the learning curve of radiology residents for bladder MRI interpretation using the VI-RADS score. Methods Three radiology residents with minimal experience in bladder MRI served as readers. They blindly evaluated 200 studies divided into 4 subsets of 50 cases over a 3-month period. After 2 months, the first subset was reassessed, resulting in a total of 250 evaluations. An interactive training program was provided and included educational lessons and case-based practice. The learning curve was constructed by plotting mean agreement as the ratio of correct evaluations per batch. Inter-reader agreement and diagnostic performance analysis were performed with kappa statistics and ROC analysis. Results As for the VI-RADS scoring agreement, the kappa differences between pre-training and post-training evaluation of the same group of cases were 0.555 to 0.852 for reader 1, 0.522 to 0.695 for reader 2, and 0.481 to 0.794 for reader 3. Using VI-RADS ≥ 3 as cut-off for muscle invasion, sensitivity ranged from 84 to 89% and specificity from 91 to 94%, while the AUCs from 0.89 (95% CI:0.84, 0.94) to 0.90 (95% CI:0.86, 0.95). Mean evaluation time decreased from 5.21 ± 1.12 to 3.52 ± 0.69 min in subsets 1 and 5. Mean grade of confidence improved from 3.31 ± 0.93 to 4.21 ± 0.69, in subsets 1 and 5. Conclusion An interactive dedicated education program on bladder MRI and the VI-RADS score led to a significant increase in readers’ diagnostic performance over time, with a general improvement observed after 100–150 cases. Key Points • After the first educational lesson and 100 cases were interpreted, the concordance on VI-RADS scoring between the residents and the experienced radiologist was significantly higher. • An increase in the grade of confidence was experienced after 100 cases. • We found a decrease in the evaluation time after 150 cases.

show abstract

Utility of Interobserver Agreement Statistics in Establishing Radiology Resident Learning Curves During Self-directed Radiologic Anatomy Training

Cited by 10 publications

References 24 publications

Do radiologists detect chronic thromboembolic disease on computed tomography?

Do radiologists detect chronic thromboembolic disease on computed tomography?

Long‐term skill improvement among general dental practitioners after a short training programme in diagnosing calcified carotid artery atheromas on panoramic radiographs

The learning curve in bladder MRI using VI-RADS assessment score during an interactive dedicated training program

Contact Info

Product

Resources

About