Categorisation of lumbar spine MRI referrals in Denmark as compliant or non-compliant to international imaging guidelines: an inter-rater reliability study

Abstract

Background Managing low back pain (LBP) often involves MRI despite the fact that international guidelines do not recommend routine imaging. To allow us to explore the topic and use this knowledge in further research, a reliable method to review the MRI referrals is needed. Consequently, this study aimed to assess the inter-rater reliability of a method evaluating lumbar spine MRI referrals’ appropriateness. Methods Four inexperienced students (chiropractic master’s students) and a senior clinician (chiropractor) were included as independent raters in this inter-rater reliability study. Lumbar spine MRI referrals from primary care on patients (> 18 years) with LBP with or without leg pain were included. The referrals were classified using a modified version of the American College of Radiology (ACR) imaging appropriateness criteria for LBP. Categories of appropriate referrals included; fractures, cancer, previous surgery, candidate for surgery or suspicion of cauda equina. Inappropriate referrals included lacking information on previous non-surgical treatment, no word on non-surgical treatment duration, or “other reasons” for inappropriate referrals. After two rounds of training and consensus sessions, 50 lumbar spine MRI referrals were reviewed independently by the five raters. Inter-rater reliability was quantified using unweighted Kappa statistics, and the observed agreement was calculated with both a pairwise comparison and an overall five-rater comparison. Results Inter-rater reliability was substantial, with a Kappa value for appropriate vs. inappropriate referrals of 0.76 (95% CI: 0.55–0.89). When six and eight subcategories were evaluated, the Kappa values were 0.77 (95% CI: 0.58–0.91) and 0.82 (95% CI: 0.72–0.92), respectively. The overall percentage of agreement for appropriate and inappropriate referrals was 92% and ranged from 88 to 98% for the pairwise comparisons of the five raters’ results. For the six and eight subcategories, the overall agreement was 92 and 88%, respectively, ranging from 88 to 98% and 84–92%, respectively, for the pairwise comparisons. Conclusion The inter-rater reliability of the evaluation of the appropriateness of lumbar spine MRI referrals, according to the modified ACR-appropriateness criteria, was found to range from substantial to almost perfect and can be used for research and quality assurance purposes.

Measuring appropriateness of diagnostic imaging: a scoping review

“…1). The most frequent reasons for exclusion at full-text level (Additional file 1: Supplementary material 3) were conference abstracts (n = 26) and studies exclusively investigating the appropriateness of requests for diagnostic imaging (n = 13) [19,[50][51][52][53][54][55][56][57][58][59][60][61]. Finally, 50 studies met the predefined inclusion criteria and were included in this scoping review [20][21][22][23].…”

Appropriateness of referrals from primary care for lumbar MRI

“…1 and designed to allow only one category per referral to be entered. Details of this process have been described elsewhere [ 17 ] The full dataset was randomly divided into five datasets of equal size by administrative staff in the radiology department who were not part of the study. Due to logistical and time constraints, only a sample representing approximately half of the referrals could be assessed.…”

Appropriateness of Referrals From Primary Care for Lumbar MRI

“…Prior to data collection, the inter-rater reliability was tested between the ve raters. Inter-tester reliability was 'Substantial' to 'Almost perfect' with Kappa values ranging from 0.76 (95% CI: 0.55-0.89) to 0.82 (95% CI: 0.72-0.92) depending on the number of subcategories [14].…”

“…elsewhere [14]. The full dataset was randomly divided into ve datasets of equal size by administrative staff in the radiology department who were not part of the study.…”