Evaluation Method of Rheumatoid Arthritis by the X-ray Photograph using Deep Learning

Hioki, Yuri; Makino, Koji; Koyama, Kikuhiko; Haro, Hirotaka; Terada, Hidetsugu

doi:10.1109/lifetech52111.2021.9391953

Cited by 8 publications

(9 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The detection of wrist joints with progressive destruction is a particularly challenging task,[3] and most reports on AI scoring models are limited to finger score predictions. [4,22,23] ResNet34, used in a previous report,[2] could not achieve highly accurate detection in our hand images, possibly because our dataset contains more severe joint destruction. In addition, a CNN that directly regresses landmarks, such as joint coordinates, is considered a model that performs a highly nonlinear transformation from images to coordinates; thus, accurately estimating coordinates is difficult.…”

Section: Discussionmentioning

confidence: 99%

“…Several CNN-based scoring systems have been reported for joint destruction in RA. [2][3][4] However, to the best of our knowledge, automated scoring systems applicable to clinical or research settings have not been constructed owing to the following reasons. First, the previous image dataset in RA was mostly dominated by intact joint images.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Development of a scoring model for the Sharp/van der Heijde score using convolutional neural networks and its clinical application in predicting radiographic progression using a graph convolutional network

Honda

Yano

Tanaka

et al. 2022

Preprint

View full text Add to dashboard Cite

ObjectiveTo construct a predictive model for the Sharp/van der Heijde score (SHS) and assess its applicability in clinical research settings.MethodsFirst, we built a rheumatoid arthritis (RA) image database linked to clinical information. We then constructed the prediction model in three steps: orientation, detection, and damage prediction. We assessed whether the SHS generated by our model replicated known findings on the association between radiographic progression and serological markers. Finally, after characterizing the SHSs of 4,264 patients using hierarchical clustering, we constructed a predictive model for joint destruction using a graph convolutional network (GCN).ResultsWe built a model with an accuracy of 100% in the correction of image orientation using EfficientNet and constructed a model with all predicted joint coordinates within 10 pixels of the correct coordinates using U-Net. In the damage prediction phase, the EfficientNet-based model combined with the modules achieved correlation coefficients of 0.879 for erosion and 0.868 for joint space narrowing between the model and expert, exceeding that of the previous best model. Our model replicated the known finding of erosion progression’s association with serologically positive patients. The areas under the curve for predicting finger and wrist erosion in the GCN model were 0.800 and 0.748, respectively. We observed that clusters generated by hierarchical clustering ranking in the top 10 were important features in the GCN for predicting erosion.ConclusionWe constructed a high-performance scoring model for SHSs applicable to clinical research. Our analysis revealed that clusters are important for predicting erosion using the GCN.Key messagesWhat is already known about this subject?►Several deep learning models that automatically predict Sharp/van der Heijde scores (SHSs) have been reported. However, the accuracy of their joint detection and erosion prediction was insufficient, and more importantly their clinical applicability was unclear.►Models for predicting joint destruction using clinical factors have been constructed with arbitrary factor selection by humans. No report has demonstrated the usefulness of deep learning models in predicting joint destruction using large SHS datasets.What does this study add?►Our deep learning model showed a high performance in both joint space narrowing and erosion, replicated previous findings on association between joint destruction and serological markers, thereby demonstrating, for the first time, that deep learning models could be clinically applicable in estimating SHSs.►We also demonstrated that a graph convolutional network (GCN) is a high performance model in predicting radiographic progression.How might this impact clinical practice or future developments?►We believe our model will be an essential tool for future studies, such as in genome-wide association studies (GWAS) for joint destruction on a scale of thousands to millions, which is difficult to achieve with human scoring. Ultimately, data from large-scale GWAS will be integrated into the GCN to build a powerful model for precision medicine.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Development of a scoring model for the Sharp/van der Heijde score using convolutional neural networks and its clinical application in predicting radiographic progression using a graph convolutional network

Honda

Yano

Tanaka

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In our literature review, we found that many scholars encountered the problem of an insufficient sample size [ 7 , 8 , 10 , 11 , 19 ]. Our method explored the feasibility by standardizing data annotation files to reduce the computing power requirements for the CNN model and improve the positioning accuracy with fewer than 1000 images.…”

Section: Discussionmentioning

confidence: 99%

“…They used 50 X-ray images of RA and diagnostic markers of finger joints to train and validate YOLOv3 to evaluate each of the diagnostic markers. The Sharp/van der Heijde score index of finger joint surface reduction and erosion was employed to categorize the 50 images; the prediction accuracy of each category reached approximately 80% on average, which verified the applicability of the DL technology in RA diagnosis [ 19 ]. However, they did not explicitly mention the data of commonly used evaluation indicators for object detection algorithms, such as the mean average precision (mAP).…”

Section: Introductionmentioning

confidence: 90%

Deep Learning-Based Computer-Aided Diagnosis of Rheumatoid Arthritis with Hand X-ray Images Conforming to Modified Total Sharp/van der Heijde Score

Wang

Lai

et al. 2022

Biomedicines

View full text Add to dashboard Cite

Introduction: Rheumatoid arthritis (RA) is a systemic autoimmune disease; early diagnosis and treatment are crucial for its management. Currently, the modified total Sharp score (mTSS) is widely used as a scoring system for RA. The standard screening process for assessing mTSS is tedious and time-consuming. Therefore, developing an efficient mTSS automatic localization and classification system is of urgent need for RA diagnosis. Current research mostly focuses on the classification of finger joints. Due to the insufficient detection ability of the carpal part, these methods cannot cover all the diagnostic needs of mTSS. Method: We propose not only an automatic label system leveraging the You Only Look Once (YOLO) model to detect the regions of joints of the two hands in hand X-ray images for preprocessing of joint space narrowing in mTSS, but also a joint classification model depending on the severity of the mTSS-based disease. In the image processing of the data, the window level is used to simulate the processing method of the clinician, the training data of the different carpal and finger bones of human vision are separated and integrated, and the resolution is increased or decreased to observe the changes in the accuracy of the model. Results: Integrated data proved to be beneficial. The mean average precision of the proposed model in joint detection of joint space narrowing reached 0.92, and the precision, recall, and F1 score all reached 0.94 to 0.95. For the joint classification, the average accuracy was 0.88, and the accuracy of severe, mild, and healthy reached 0.91, 0.79, and 0.9, respectively. Conclusions: The proposed model is feasible and efficient. It could be helpful for subsequent research on computer-aided diagnosis in RA. We suggest that applying the one-hand X-ray imaging protocol can improve the accuracy of mTSS classification model in determining mild disease if it is used in clinical practice.

show abstract

“…In rheumatology, and more specifically, in the scope of RA, several deep learning researchers over the past years have presented their work as they tried to achieve a method for automatic classification of medical images for RA diagnosis. According to [10], it is very difficult for a doctor not majored in RA to evaluate precisely the condition of RA as its diagnosis is affiliated with implicit knowledge. The authors in [11] also agree that physicians have to rely on a manual and subjective examination of radiographs.…”

Section: Deep Learning On Ramentioning

confidence: 99%

Rheumatoid Arthritis Diagnosis: Deep Learning vs. Humane

Avramidis

Avramidou²,

Papakostas

2021

Applied Sciences

View full text Add to dashboard Cite

Rheumatoid arthritis (RA) is a systemic autoimmune disease that preferably affects small joints. As the well-timed diagnosis of the disease is essential for the treatment of the patient, several works have been conducted in the field of deep learning to develop fast and accurate automatic methods for RA diagnosis. These works mainly focus on medical images as they use X-ray and ultrasound images as input for their models. In this study, we review the conducted works and compare the methods that use deep learning with the procedure that is commonly followed by a medical doctor for the RA diagnosis. The results show that 93% of the works use only image modalities as input for the models as distinct from the medical procedure where more patient medical data are taken into account. Moreover, only 15% of the works use direct explainability methods, meaning that the efforts for solving the trustworthiness issue of deep learning models were limited. In this context, this work reveals the gap between the deep learning approaches and the medical doctors’ practices traditionally applied and brings to light the weaknesses of the current deep learning technology to be integrated into a trustworthy context inside the existed medical infrastructures.

show abstract

Evaluation Method of Rheumatoid Arthritis by the X-ray Photograph using Deep Learning

Cited by 8 publications

References 16 publications

Development of a scoring model for the Sharp/van der Heijde score using convolutional neural networks and its clinical application in predicting radiographic progression using a graph convolutional network

Development of a scoring model for the Sharp/van der Heijde score using convolutional neural networks and its clinical application in predicting radiographic progression using a graph convolutional network

Deep Learning-Based Computer-Aided Diagnosis of Rheumatoid Arthritis with Hand X-ray Images Conforming to Modified Total Sharp/van der Heijde Score

Rheumatoid Arthritis Diagnosis: Deep Learning vs. Humane

Contact Info

Product

Resources

About