ObjectivesTo systematically review, assess the reporting quality of, and discuss improvement opportunities for studies describing machine learning (ML) models for glioma grade prediction.MethodsThis study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnostic Test Accuracy (PRISMA-DTA) statement. A systematic search was performed in September 2020, and repeated in January 2021, on four databases: Embase, Medline, CENTRAL, and Web of Science Core Collection. Publications were screened in Covidence, and reporting quality was measured against the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) Statement. Descriptive statistics were calculated using GraphPad Prism 9.ResultsThe search identified 11,727 candidate articles with 1,135 articles undergoing full text review and 85 included in analysis. 67 (79%) articles were published between 2018-2021. The mean prediction accuracy of the best performing model in each study was 0.89 ± 0.09. The most common algorithm for conventional machine learning studies was Support Vector Machine (mean accuracy: 0.90 ± 0.07) and for deep learning studies was Convolutional Neural Network (mean accuracy: 0.91 ± 0.10). Only one study used both a large training dataset (n>200) and external validation (accuracy: 0.72) for their model. The mean adherence rate to TRIPOD was 44.5% ± 11.1%, with poor reporting adherence for model performance (0%), abstracts (0%), and titles (0%).ConclusionsThe application of ML to glioma grade prediction has grown substantially, with ML model studies reporting high predictive accuracies but lacking essential metrics and characteristics for assessing model performance. Several domains, including generalizability and reproducibility, warrant further attention to enable translation into clinical practice.Systematic Review RegistrationPROSPERO, identifier CRD42020209938.

PurposePersonalized interpretation of medical images is critical for optimum patient care, but current tools available to physicians to perform quantitative analysis of patient’s medical images in real time are significantly limited. In this work, we describe a novel platform within PACS for volumetric analysis of images and thus development of large expert annotated datasets in parallel with radiologist performing the reading that are critically needed for development of clinically meaningful AI algorithms. Specifically, we implemented a deep learning-based algorithm for automated brain tumor segmentation and radiomics extraction, and embedded it into PACS to accelerate a supervised, end-to- end workflow for image annotation and radiomic feature extraction.Materials and methodsAn algorithm was trained to segment whole primary brain tumors on FLAIR images from multi-institutional glioma BraTS 2021 dataset. Algorithm was validated using internal dataset from Yale New Haven Health (YHHH) and compared (by Dice similarity coefficient [DSC]) to radiologist manual segmentation. A UNETR deep-learning was embedded into Visage 7 (Visage Imaging, Inc., San Diego, CA, United States) diagnostic workstation. The automatically segmented brain tumor was pliable for manual modification. PyRadiomics (Harvard Medical School, Boston, MA) was natively embedded into Visage 7 for feature extraction from the brain tumor segmentations.ResultsUNETR brain tumor segmentation took on average 4 s and the median DSC was 86%, which is similar to published literature but lower than the RSNA ASNR MICCAI BRATS challenge 2021. Finally, extraction of 106 radiomic features within PACS took on average 5.8 ± 0.01 s. The extracted radiomic features did not vary over time of extraction or whether they were extracted within PACS or outside of PACS. The ability to perform segmentation and feature extraction before radiologist opens the study was made available in the workflow. Opening the study in PACS, allows the radiologists to verify the segmentation and thus annotate the study.ConclusionIntegration of image processing algorithms for tumor auto-segmentation and feature extraction into PACS allows curation of large datasets of annotated medical images and can accelerate translation of research into development of personalized medicine applications in the clinic. The ability to use familiar clinical tools to revise the AI segmentations and natively embedding the segmentation and radiomic feature extraction tools on the diagnostic workstation accelerates the process to generate ground-truth data.

We evaluate the topographic distribution of diffuse midline gliomas and hemispheric high-grade gliomas in children with respect to their normal gene expression patterns and pathologic driver mutation patterns. We identified 19 pediatric patients with diffuse midline or high-grade glioma with preoperative MRI from tumor board review. 7 of these had 500 gene panel mutation testing, 11 patients had 50 gene panel mutation testing and one 343 gene panel testing from a separate institution were included as validation set. Tumor imaging features and gene expression patterns were analyzed using Allen Brain Atlas. Twelve patients had diffuse midline gliomas and seven had hemispheric high-grade gliomas. Three diffuse midline gliomas had the K27M mutation in the tail of histone H3 protein. All patients undergoing 500 gene panel testing had additional mutations, the most common being in ACVR1, PPM1D, and p53. Hemispheric high-grade gliomas had either TP53 or IDH1 mutation and diffuse midline gliomas had H3 K27M-mutation. Gene expression analysis in normal brains demonstrated that genes mutated in diffuse midline gliomas had higher expression along midline structures as compared to the cerebral hemispheres. Our study suggests that topographic location of pediatric diffuse midline gliomas and hemispheric high-grade gliomas correlates with driver mutations of tumor to the endogenous gene expression in that location. This correlation suggests that cellular state that is required for increased gene expression predisposes that location to mutations and defines the driver mutations within tumors that arise from that region.

PURPOSE Reporting guidelines are crucial in model development studies to ensure the quality, transparency and objectivity of reporting. While machine learning (ML) models have proven themselves effective in predicting glioma grade, their potential use can only be determined if they are clearly and comprehensively reported. Reporting quality has not yet been evaluated for ML glioma grade prediction studies, to our knowledge. We measured published literature against the TRIPOD Statement, a checklist of items considered essential for the reporting of diagnostic studies. MATERIALS AND METHODS A literature review, in agreement with PRISMA, was conducted by a university librarian in October 2020 and verified by a second librarian in February 2021 using four databases: Cochrane trials (CENTRAL), Ovid Embase, Ovid MEDLINE, and Web of Science core-collection. Keywords and controlled vocabulary included artificial intelligence, machine learning, deep learning, radiomics, magnetic resonance imaging, glioma, and glioblastoma. Publications were screened in Covidence and scored against the 27 items in the TRIPOD Statement that were relevant and applicable. RESULTS The search identified 11,727 candidate articles with 1,135 articles undergoing full text review. 86 articles met the criteria for our study. The mean adherence rate to TRIPOD was 44.4% (range: 22.2% - 66.7%), with poor reporting adherence in categories including abstract (0%), model performance (0%), title (1.2%), justification of sample size (2.3%), full model specification (2.3%), participant demographics and missing data (7%). Studies had high reporting adherence in categories including results interpretation (100%), background (98.8%), study design/source of data (96.5%), and objectives (95.3%). CONCLUSION Existing publications on the use of ML in glioma grade prediction have a low overall quality of reporting. Improvements can be made in the reporting of titles and abstracts, justification of sample size, and model specification and performance.

PURPOSE Machine learning (ML) technologies have demonstrated highly accurate prediction of glioma grade, though it is unclear which methods and algorithms are superior. We have conducted a systematic review of the literature in order to identify the ML applications most promising for future research and clinical implementation. MATERIALS AND METHODS A literature review, in agreement with PRISMA, was conducted by a university librarian in October 2020 and verified by a second librarian in February 2021 using four databases: Cochrane trials (CENTRAL), Ovid Embase, Ovid MEDLINE, and Web of Science core-collection. Keywords and controlled vocabulary included artificial intelligence, machine learning, deep learning, radiomics, magnetic resonance imaging, glioma, and glioblastoma. Screening of publications was done in Covidence, and TRIPOD was used for bias assessment. RESULTS The search identified 11,727 candidate articles with 1,135 articles undergoing full text review. 86 articles published since 1995 met the criteria for our study. 79% of the articles were published between 2018 and 2020. The average glioma prediction accuracy of the highest performing model in each study was 90% (range: 53% to 100%). The most common algorithm used for cML studies was Support Vector Machine (SVM) and for DL studies was Convolutional Neural Network (CNN). BRATS and TCIA datasets were used in 47% of the studies, with the average patient number of study datasets being 186 (range: 23 to 662). The average number of features used in machine learning prediction was 55 (range: 2 to 580). Classical machine learning (cML) was the primary machine learning model in 68% of studies, with deep learning (DL) used in 32%. CONCLUSIONS Using multimodal sequences in ML methods delivers significantly higher grading accuracies than single sequences. Potential areas of improvement for ML glioma grade prediction studies include increasing sample size, incorporating molecular subtypes, and validating on external datasets.