Automatic Detection of Knee Joints and Quantification of Knee Osteoarthritis Severity Using Convolutional Neural Networks

Abstract

Abstract. This paper introduces a new approach to automatically quantify the severity of knee OA using X-ray images. Automatically quantifying knee OA severity involves two steps: first, automatically localizing the knee joints; next, classifying the localized knee joint images. We introduce a new approach to automatically detect the knee joints using a fully convolutional neural network (FCN). We train convolutional neural networks (CNN) from scratch to automatically quantify the knee OA severity optimizing a weighted ratio of two loss functions: categorical cross-entropy and mean-squared loss. This joint training further improves the overall quantification of knee OA severity, with the added benefit of naturally producing simultaneous multi-class classification and regression outputs. Two public datasets are used to evaluate our approach, the Osteoarthritis Initiative (OAI) and the Multicenter Osteoarthritis Study (MOST), with extremely promising results that outperform existing approaches.

Predicting knee osteoarthritis severity: comparative modeling based on patient’s data and plain X-ray images

“…We have demonstrated that statistical models, using patients' questionnaire data, could predict KOA severity level with a good level of accuracy (RMSE: 0.974 & 0.943). The prediction performance of the statistical models presented in this paper are comparable to models using X-ray image data based on model performance as assessed by RMSE measures [26][27][28] . In particular we have demonstrated that functional impairment at severity levels 1 and 2 can be predicted by our statistical models (Elastic Net & Random Forest and LMM) trained from the patients' assessment data to a level of accuracy similar to the accuracy achieved on the basis of CNN model trained on X-ray images.…”

“…To train and validate the predictive models we used a training and validation data split as shown in Table 1 (roughly a 70% -30% split). To make valid comparisons, we used the same validation set in the models developed on patient's questionnaire data and the model developed using X-ray images 26,27 . The validation set contained data for both knees for 846 patients, i.e.…”

“…The dataset was split into two parts, training and validation sets, by taking a random sample of 70% of the data for training and the remaining 30% for validation. To make valid comparisons, we used the same validation set in the models developed on patient's questionnaire data and the model developed using X-ray images 26,27 . The data pre-processing steps are summarized in ( Figure 5).…”

“…The Osteoarthritis Initiative (OAI) 23 and the Multi-centre Osteoarthritis Study (MOST) 24 mitigated this small sample size limitation by making thousands of patients data and X-ray images available. Recently, several researchers have used these resources to develop an automatic approach for quantifying KOA severity by analyzing X-ray images 21,[25][26][27][28] . Although there have been multiple attempts to quantify KOA severity based on an automated analysis of X-ray images, so far there has been no attempt to build a predictive model on a patient's assessment data such as signs, symptoms, medication and other characteristics about a patient (later on referred as patient's questionnaire data) and to compare this approach against the X-ray based prediction.…”

Deep learning for prediction of cardiac indices from photoplethysmographic waveform: A virtual database approach

“…Second, we extend the approach by employing deep convolutional neural networks (CNNs). [29][30][31] Originally, CNNs were developed for image recognition task, 31,32 but recently, CNNs are also been applied to several medical applications, for example, diagnostic chest X-Rays, 33,34 fracture detection, [35][36][37] mammography, 38 low-dose X-ray tomography, [39][40][41] detection of osteoarthritis, 42,43 diagnosis of retinal diseases, 44 Alzheimer's disease diagnostics, [45][46][47] and MRI segmentation. 48 CNNs are also applied to the risk stratification of hypertension from PPG waveform data.…”