Evaluation of Deep Learning-Based Automated Detection of Primary Spine Tumors on MRI Using the Turing Test

Abstract

BackgroundRecently, the Turing test has been used to investigate whether machines have intelligence similar to humans. Our study aimed to assess the ability of an artificial intelligence (AI) system for spine tumor detection using the Turing test.MethodsOur retrospective study data included 12179 images from 321 patients for developing AI detection systems and 6635 images from 187 patients for the Turing test. We utilized a deep learning-based tumor detection system with Faster R-CNN architecture, which generates region proposals by Region Proposal Network in the first stage and corrects the position and the size of the bounding box of the lesion area in the second stage. Each choice question featured four bounding boxes enclosing an identical tumor. Three were detected by the proposed deep learning model, whereas the other was annotated by a doctor; the results were shown to six doctors as respondents. If the respondent did not correctly identify the image annotated by a human, his answer was considered a misclassification. If all misclassification rates were >30%, the respondents were considered unable to distinguish the AI-detected tumor from the human-annotated one, which indicated that the AI system passed the Turing test.ResultsThe average misclassification rates in the Turing test were 51.2% (95% CI: 45.7%–57.5%) in the axial view (maximum of 62%, minimum of 44%) and 44.5% (95% CI: 38.2%–51.8%) in the sagittal view (maximum of 59%, minimum of 36%). The misclassification rates of all six respondents were >30%; therefore, our AI system passed the Turing test.ConclusionOur proposed intelligent spine tumor detection system has a similar detection ability to annotation doctors and may be an efficient tool to assist radiologists or orthopedists in primary spine tumor detection.

Bimodal artificial intelligence using TabNet for differentiating spinal cord tumors—Integration of patient background information and images

“…Some researchers have reported that artificial intelligence (AI) models aid in the diagnosis of spinal tumors. 5 , 6 , 7 , 8 , 9 Automated object detection and segmentation of spinal tumors yielded a high accuracy that was comparable to that of the physicians. 6 , 7 , 9 Besides these, classification models of spinal tumors have been established 8 and one of them was a model to discriminate between schwannomas and meningiomas.…”

“… 5 , 6 , 7 , 8 , 9 Automated object detection and segmentation of spinal tumors yielded a high accuracy that was comparable to that of the physicians. 6 , 7 , 9 Besides these, classification models of spinal tumors have been established 8 and one of them was a model to discriminate between schwannomas and meningiomas. 5 In this study, magnetic resonance imaging (MRI) of 84 patients was analyzed, and the area under the receiver-operating characteristic curve (AUROC) was 0.88.…”

Automated Detection of Spinal Lesions From CT Scans via Deep Transfer Learning

“…Several works consider deep learning models for detection of abnormalities in the spinal column. In [26], authors propose a study the performances of a Faster R-CNN network on MRI images. In [27] authors exploit Siamese networks to detect spine metastases from MRI images.…”

The clinical application of artificial intelligence technology in spinal surgery

“…[19] Several researchers have proposed an automated DL framework based on an ensemble of U-Nets to perform vertebral morphometry and measure the Cobb angle directly on three-dimensional (3D) computed tomography (CT) images of the spine. [20] Deep learning in spinal disease diagnosis DL algorithms have been employed to diagnose a variety of spinal diseases, such as tumor, [21] infection, [21] osteoporosis, [22] scoliosis, [22] fracture [23] and degenerative disease. [24] For instance, CNNs demonstrated high accuracy in differentiating between normal and stenotic lumbar spine on MRIs.…”