Development and validation of deep learning algorithms for scoliosis screening using back images

Yang, Junlin; Zhang, Kai; Fan, Hengwei; Huang, Zifang; Xiang, Yifan; Yang, Jing; He, Lin; Zhang, Lei; Yang, Yahan; Li, Ruiyang; Zhu, Yi; Chen, Chuan; Liu, Fan; Yang, Haoqing; Deng, Yong; Tan, Weiqing; Deng, Nali; Xiang, Yu; Xuan, Xiaoling; Xie, Xiaofeng; Liu, Xiyang; Lin, Haotian

doi:10.1038/s42003-019-0635-8

Cited by 101 publications

(79 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The accuracy of the algorithm for detecting scoliosis was 0.75 and 0.87 with curves of ≥10˚and ≥20˚, respectively [15]. However, the greatest limitation of this system was that their DLAs could not predict the Cobb angle, because they were based on the classification method [15]. In our study, the correlation coefficient was 0.91, indicating that CNN for regression improved our previous system (r=0.85) [10].…”

Section: Discussionmentioning

confidence: 57%

“…In addition, the correlation coefficient was unclear [16−18]. Yang et al [15] developed DLAs for automated scoliosis detection using unclothed 2D back photographs. The accuracy of the algorithm for detecting scoliosis was 0.75 and 0.87 with curves of ≥10˚and ≥20˚, respectively [15].…”

Section: Discussionmentioning

confidence: 99%

“…Yang et al [15] developed DLAs for automated scoliosis detection using unclothed 2D back photographs. The accuracy of the algorithm for detecting scoliosis was 0.75 and 0.87 with curves of ≥10˚and ≥20˚, respectively [15]. However, the greatest limitation of this system was that their DLAs could not predict the Cobb angle, because they were based on the classification method [15].…”

Section: Discussionmentioning

confidence: 99%

“…In the spine field, CNNs have been applied for detection, classification or segmentation of diseases such as differentiation of intradural extramedullary tumors [13] and spinal metastases [14]. Using deep learning algorithms (DLAs) with CNNs to detect AIS, some systems have been developed using 2D photographs [15] or Moir e topography shot from human back [16−18]. Although these systems can classify scoliosis or estimate the position of vertebrae, there are still significant limitations with predicting the Cobb angle.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

An algorithm for using deep learning convolutional neural networks with three dimensional depth sensor imaging in scoliosis detection

et al. 2021

View full text Add to dashboard Cite

BACKGROUND CONTEXT: Timely intervention in growing individuals, such as brace treatment, relies on early detection of adolescent idiopathic scoliosis (AIS). To this end, several screening methods have been implemented. However, these methods have limitations in predicting the Cobb angle. PURPOSE: This study aimed to evaluate the performance of a three-dimensional depth sensor imaging system with a deep learning algorithm, in predicting the Cobb angle in AIS. STUDY DESIGN: Retrospective analysis of prospectively collected, consecutive, nonrandomized series of patients at five scoliosis centers in Japan. PATIENT SAMPLE: One hundred and-sixty human subjects suspected to have AIS were included. OUTCOME MEASURES: Patient demographics, radiographic measurements, and predicted Cobb angle derived from the deep learning algorithm were the outcome measures for this study. METHODS: One hundred and sixty data files were shuffled into five datasets with 32 data files at random (dataset 1, 2, 3, 4, and 5) and five-fold cross validation was performed. The relationships between the actual and predicted Cobb angles were calculated using Pearson's correlation coefficient analyses. The prediction performances of the network models were evaluated using mean absolute error and root mean square error between the actual and predicted Cobb angles. The shuffling into five datasets and five-fold cross validation was conducted ten times. There were no studyspecific biases related to conflicts of interest. RESULTS: The correlation between the actual and the mean predicted Cobb angles was 0.91. The mean absolute error and root mean square error were 4.0˚and 5.4˚, respectively. The accuracy of the mean predicted Cobb angle was 94% for identifying a Cobb angle of ≥10˚and 89% for that of ≥20˚. FDA device/drug status: Not approved (SCOLIOMAP).

show abstract

Section: Discussionmentioning

confidence: 57%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An algorithm for using deep learning convolutional neural networks with three dimensional depth sensor imaging in scoliosis detection

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Any feature of an object can be detected using a CNN. Recently, Yang et al 11 created a system that can detect scoliosis from unclothed back photos using a CNN. Although the accuracy of this system was high, with an AUC of 0.929, it could not estimate the definite value of the Cobb angle.…”

Section: School Scoliosis Screening System Using Moiré Imagesmentioning

confidence: 99%

An Application of Artificial Intelligence to Diagnostic Imaging of Spine Disease: Estimating Spinal Alignment From Moiré Images

2019

View full text Add to dashboard Cite

The use of artificial intelligence (AI) as a tool supporting the diagnosis and treatment of spinal diseases is eagerly anticipated. In the field of diagnostic imaging, the possible application of AI includes diagnostic support for diseases requiring highly specialized expertise, such as trauma in children, scoliosis, symptomatic diseases, and spinal cord tumors. Moiré topography, which describes the 3-dimensional surface of the trunk with band patterns, has been used to screen students for scoliosis, but the interpretation of the band patterns can be ambiguous. Thus, we created a scoliosis screening system that estimates spinal alignment, the Cobb angle, and vertebral rotation from moiré images. In our system, a convolutional neural network (CNN) estimates the positions of 12 thoracic and 5 lumbar vertebrae, 17 spinous processes, and the vertebral rotation angle of each vertebra. We used this information to estimate the Cobb angle. The mean absolute error (MAE) of the estimated vertebral positions was 3.6 pixels (~5.4 mm) per person. T1 and L5 had smaller MAEs than the other levels. The MAE per person between the Cobb angle measured by doctors and the estimated Cobb angle was 3.42°. The MAE was 4.38° in normal spines, 3.13° in spines with a slight deformity, and 2.74° in spines with a mild to severe deformity. The MAE of the angle of vertebral rotation was 2.9°± 1.4°, and was smaller when the deformity was milder. The proposed method of estimating the Cobb angle and AVR from moiré images using a CNN is expected to enhance the accuracy of scoliosis screening.

show abstract

Deep Learning Model to Classify and Monitor Idiopathic Scoliosis in Adolescents Using a Single Smartphone Photograph

Zhang,

Zhu,

Zhao

et al. 2023

JAMA Netw Open

View full text Add to dashboard Cite

ImportanceAdolescent idiopathic scoliosis (AIS) is the most common pediatric spinal disorder. Routine physical examinations by trained personnel are critical to diagnose severity and monitor curve progression in AIS. In the presence of concerning malformation, radiographs are necessary for diagnosis or follow-up, guiding further management, such as bracing correction for moderate malformation and spine surgery for severe malformation. If left unattended, progressive deterioration occurs in two-thirds of patients, leading to significant health concerns for growing children.ObjectiveTo assess the ability of an open platform application (app) using a validated deep learning model to classify AIS severity and curve type, as well as identify progression.Design, Setting, and ParticipantsThis diagnostic study was performed with data from radiographs and smartphone photographs of the backs of adolescent patients at spine clinics. The ScolioNets deep learning model was developed and validated in a prospective training cohort, then incorporated and tested in the AlignProCARE open platform app in 2022. Ground truths (GTs) included severity, curve type, and progression as manually annotated by 2 experienced spine specialists based on the radiographic examinations of the participants’ spines. The GTs and app results were blindly compared with another 2 spine surgeons’ assessments of unclothed back appearance. Data were analyzed from October 2022 to February 2023.ExposureAcquisitions of unclothed back photographs using a mobile app.Main Outcomes and MeasuresOutcomes of interest were classification of AIS severity and progression. Quantitative statistical analyses were performed to assess the performance of the deep learning model in classifying the deformity as well as in distinguishing progression during 6-month follow-up.ResultsThe training data set consisted of 1780 patients (1295 [72.8%] female; mean [SD] age, 14.3 [3.3] years), and the prospective testing data sets consisted of 378 patients (279 [73.8%] female; mean [SD] age, 14.3 [3.8] years) and 376 follow-ups (294 [78.2%] female; mean [SD] age, 15.6 [2.9] years). The model recommended follow-up with an area under receiver operating characteristic curve (AUC) of 0.839 (95% CI, 0.789-0.882) and considering surgery with an AUC of 0.902 (95% CI, 0.859-0.936), while showing good ability to distinguish among thoracic (AUC, 0.777 [95% CI, 0.745-0.808]), thoracolumbar or lumbar (AUC, 0.760 [95% CI, 0.727-0.791]), or mixed (AUC, 0.860 [95% CI, 0.834-0.887]) curve types. For follow-ups, the model distinguished participants with or without curve progression with an AUC of 0.757 (95% CI, 0.630-0.858). Compared with both surgeons, the model could recognize severities and curve types with a higher sensitivity (eg, sensitivity for recommending follow-up: model, 84.88% [95% CI, 75.54%-91.70%]; senior surgeon, 44.19%; junior surgeon, 62.79%) and negative predictive values (NPVs; eg, NPV for recommending follow-up: model, 89.22% [95% CI, 84.25%-93.70%]; senior surgeon, 71.76%; junior surgeon, 79.35%). For distinguishing curve progression, the sensitivity and NPV were comparable with the senior surgeons (sensitivity, 63.33% [95% CI, 43.86%-80.87%] vs 77.42%; NPV, 68.57% [95% CI, 56.78%-78.37%] vs 72.00%). The junior surgeon reported an inability to identify curve types and progression by observing the unclothed back alone.ConclusionsThis diagnostic study of adolescent patients screened for AIS found that the deep learning app had the potential for out-of-hospital accessible and radiation-free management of children with scoliosis, with comparable performance as spine surgeons experienced in AIS management.

show abstract

Development and validation of deep learning algorithms for scoliosis screening using back images

Cited by 101 publications

References 42 publications

An algorithm for using deep learning convolutional neural networks with three dimensional depth sensor imaging in scoliosis detection

An algorithm for using deep learning convolutional neural networks with three dimensional depth sensor imaging in scoliosis detection

An Application of Artificial Intelligence to Diagnostic Imaging of Spine Disease: Estimating Spinal Alignment From Moiré Images

Deep Learning Model to Classify and Monitor Idiopathic Scoliosis in Adolescents Using a Single Smartphone Photograph

Contact Info

Product

Resources

About