Deep-learning-assisted diagnosis for knee magnetic resonance imaging: Development and retrospective validation of MRNet

Abstract

BackgroundMagnetic resonance imaging (MRI) of the knee is the preferred method for diagnosing knee injuries. However, interpretation of knee MRI is time-intensive and subject to diagnostic error and variability. An automated system for interpreting knee MRI could prioritize high-risk patients and assist clinicians in making diagnoses. Deep learning methods, in being able to automatically learn layers of features, are well suited for modeling the complex relationships between medical images and their interpretations. In this study we developed a deep learning model for detecting general abnormalities and specific diagnoses (anterior cruciate ligament [ACL] tears and meniscal tears) on knee MRI exams. We then measured the effect of providing the model’s predictions to clinical experts during interpretation.Methods and findingsOur dataset consisted of 1,370 knee MRI exams performed at Stanford University Medical Center between January 1, 2001, and December 31, 2012 (mean age 38.0 years; 569 [41.5%] female patients). The majority vote of 3 musculoskeletal radiologists established reference standard labels on an internal validation set of 120 exams. We developed MRNet, a convolutional neural network for classifying MRI series and combined predictions from 3 series per exam using logistic regression. In detecting abnormalities, ACL tears, and meniscal tears, this model achieved area under the receiver operating characteristic curve (AUC) values of 0.937 (95% CI 0.895, 0.980), 0.965 (95% CI 0.938, 0.993), and 0.847 (95% CI 0.780, 0.914), respectively, on the internal validation set. We also obtained a public dataset of 917 exams with sagittal T1-weighted series and labels for ACL injury from Clinical Hospital Centre Rijeka, Croatia. On the external validation set of 183 exams, the MRNet trained on Stanford sagittal T2-weighted series achieved an AUC of 0.824 (95% CI 0.757, 0.892) in the detection of ACL injuries with no additional training, while an MRNet trained on the rest of the external data achieved an AUC of 0.911 (95% CI 0.864, 0.958). We additionally measured the specificity, sensitivity, and accuracy of 9 clinical experts (7 board-certified general radiologists and 2 orthopedic surgeons) on the internal validation set both with and without model assistance. Using a 2-sided Pearson’s chi-squared test with adjustment for multiple comparisons, we found no significant differences between the performance of the model and that of unassisted general radiologists in detecting abnormalities. General radiologists achieved significantly higher sensitivity in detecting ACL tears (p-value = 0.002; q-value = 0.019) and significantly higher specificity in detecting meniscal tears (p-value = 0.003; q-value = 0.019). Using a 1-tailed t test on the change in performance metrics, we found that providing model predictions significantly increased clinical experts’ specificity in identifying ACL tears (p-value < 0.001; q-value = 0.006). The primary limitations of our study include lack of surgical ground truth and the small size of ...

Model-Based and Data-Driven Strategies in Medical Image Computing

“…More recently, machine-learning (in particular deep learning) approaches have dominated attempts to perform classification of images with the purpose of disease diagnosis [89,16]. Many of these approaches use the image intensities directly without any explicit feature extraction.…”

Artificial Intelligence in Musculoskeletal Imaging: A Paradigm Shift

“…(49,50) Automated detection of the fascia lata in the thigh was used to assess for fatty replacement of muscle in patients with muscular dystrophy. (52) In the next sections, we discuss specific applications in more detail. (52) In the next sections, we discuss specific applications in more detail.…”

“…(51) Knee cartilage defects and meniscal tears have been automatically assessed on MRI. (52) In the next sections, we discuss specific applications in more detail.…”

Deep Learning for Musculoskeletal Image Analysis

“…We have used publicly available challenge dataset, named MRNet, comprising of multi-view knee MRIs [3]. The dataset consists of 1370 MRI exams from Stanford University medical center, MRI scans were acquired between 2001 -2012.…”

“…In this regard, to facilitate development of deep learning models for detecting abnormalities from knee MRI scans, one important milestone was the release of a collected set of knee MRI scans and their annotations by Stanford scientists [3]. We utilized this data to conduct our comparative evaluations.…”