Reliability Analysis of Deep Learning Algorithms for Reporting of Routine Lumbar MRI Scans

Abstract

Background: Artificial intelligence could provide more accurate magnetic resonance imaging (MRI) predictors of successful clinical outcomes in targeted spine care.Objective: To analyze the level of agreement between lumbar MRI reports created by a deep learning neural network (RadBot) and the radiologists' MRI reading.Methods: The compressive pathology definitions were extracted from the radiologist lumbar MRI reports from 65 patients with a total of 383 levels for the central canal: (0) no disc bulge/protrusion/canal stenosis, (1) disc bulge without canal stenosis, (2) disc bulge resulting in canal stenosis, and (3) disc herniation/protrusion/extrusion resulting in canal stenosis. For both, neural foramina were assessed with either (0) neural foraminal stenosis absent or (1) neural foramina stenosis present. Reporting criteria for the pathologies at each disc level and, when available, the grading of severity were extracted, and the Natural Language Processing model was used to generate a verbal and written report. The RadBot report was analyzed similarly as the MRI report by the radiologist. MRI reports were investigated by dichotomizing the data into 2 categories: normal and stenosis. The quality of the RadBot test was assessed by determining its sensitivity, specificity, and positive and negative predictive value as well as its reliability with the calculation of the Cronbach alpha and Cohen kappa using the radiologist MRI report as a gold standard.Results: The authors found a RadBot sensitivity of 73.3%, a specificity of 88.4%, a positive predictive value of 80.3%, and a negative predictive value of 83.7%. The reliability analysis revealed the Cronbach alpha as 0.772. The highest individual values of the Cronbach alpha were 0.629 and 0.681 when compared to the MRI report by the radiologist, rending values of 0.566 and 0.688, respectively. Analysis of interobserver reliability rendered an overall kappa for the RadBot of 0.627. Analysis of receiver operating characteristics (ROC) showed a value of 0.808 for the area under the ROC curve.Conclusions: Deep learning algorithms, when used for routine reporting in lumbar spine MRI, showed excellent quality as a diagnostic test that can distinguish the presence of neural element compression (stenosis) at a statistically significant level (P , .0001) from a random event distribution. This research should be extended to validated and directly visualized pain generators to improve the accuracy and prognostic value of the routine lumbar MRI scan for favorable clinical outcomes with intervention and surgery.

A deep learning model for detection of cervical spinal cord compression in MRI scans

“…This model performed well on the holdout dataset with an AUC of 0.94 and F1-score of 0.82. These performance metrics are similar to the best performing lumbar spine models that have been previously published 39 , 41 – 43 .…”

“…This model performed well on the holdout dataset with an AUC of 0.94 and F1-score of 0.82. These performance metrics are similar to the best performing lumbar spine models that have been previously published 39,[41][42][43] . We generated CAMs using example images from the testing dataset that were classified correctly (true positives) and incorrectly (false negatives).…”

“…Image segmentation can potentially allow for automated localization of pathologic findings, which was not possible with the simpler approach that we took. Another study by Lewandrowski et al used a dataset of lumbar spine MRI scans from 3560 patients and developed a model to grade lumbar disc herniation and canal stenosis 42 , 43 . Similar to the DeepSpine framework, their approach consisted of a segmentation step in which intervertebral disks were segmented.…”

The Clinical Diagnostic Value of Lumbar Intervertebral Disc Herniation Based on MRI Images

“…Weakness of intervertebral disc's own anatomical factors: human intervertebral disc blood circulation gradually lacks in adulthood, and its self-repair ability becomes worse. Genetic factors: there are reports of familial incidence of lumbar disc herniation [ 17 ]. Congenital abnormalities of the lumbosacral: sacralization of the lumbar spine, hemivertebral deformity, asymmetry of the articular process, etc., change the stress on the lower lumbar spine, which is prone to degeneration and injury.…”

“…Cheung and Luk [ 16 ] reported that the accuracy of MRI in the diagnosis of intervertebral disc herniation is significantly higher than that of CT, and the accuracy of the diagnosis of intervertebral disc herniation and free intervertebral disc nucleus pulposus is 100%, which is significantly higher than that of CT examination of 88% and 50% LDH. The total detection rate of LDH by MRI is 96.67%, which is significantly higher than 71.67% of CT. For the diagnosis of lumbar facet joints in the comparison of CT and MRI, LewandrowskI et al [ 17 ] believe that MRI is an indispensable tool. The evaluation accuracy of FJ degeneration degree is 94% of CT. Osteophyte hyperplasia, gas accumulation in facet joints, and vacuum phenomenon will affect the accuracy of articular cartilage thickness measurement.…”