Machine learning can aid in prediction of IDH mutation from H&E-stained histology slides in infiltrating gliomas

Abstract

While Machine Learning (ML) models have been increasingly applied to a range of histopathology tasks, there has been little emphasis on characterizing these models and contrasting them with human experts. We present a detailed empirical analysis comparing expert neuropathologists and ML models at predicting IDH mutation status in H&E-stained histology slides of infiltrating gliomas, both independently and synergistically. We find that errors made by neuropathologists and ML models trained using the TCGA dataset are distinct, representing modest agreement between predictions (human-vs.-human κ = 0.656; human-vs.-ML model κ = 0.598). While no ML model surpassed human performance on an independent institutional test dataset (human AUC = 0.901, max ML AUC = 0.881), a hybrid model aggregating human and ML predictions demonstrates predictive performance comparable to the consensus of two expert neuropathologists (hybrid classifier AUC = 0.921 vs. two-neuropathologist consensus AUC = 0.920). We also show that models trained at different levels of magnification exhibit different types of errors, supporting the value of aggregation across spatial scales in the ML approach. Finally, we present a detailed interpretation of our multi-scale ML ensemble model which reveals that predictions are driven by human-identifiable features at the patch-level.

Practical Application of Deep Learning in Diagnostic Neuropathology—Reimagining a Histological Asset in the Era of Precision Medicine

“…Due to the diagnostic relevance of IDH mutations in adult-type diffuse gliomas, Liechty et al [ 33 ] trained a DenseNet-121 convolutional neural network (CNN) model to predict IDH status using the TCGA datasets. While there was a significant drop in accuracy from the TCGA datasets to a separate internal dataset (AUC 0.988 vs. 0.829), their model achieved similar diagnostic accuracy to expert pathologists; however, the mistakes the pathologists and the model made were distinct.…”

Artificial intelligence in histopathological image analysis of central nervous system tumours: A systematic review

“…Only seven studies conducted external validation. 18,28,32,57,67,74,81 Only three studies within this subset reported model performance on the corresponding unseen datasets. 18,67,74 See Figure 3 for detailed definitions of external validation methods employed.…”

“…37,45 GOAL 6: MOLECULAR CHARACTERISATION Four studies aimed to predict the molecular status of tumours based on H&E-stained tissue sections. [66][67][68][69] One study used nuclear morphology to predict the transcriptional profile of glioblastoma: classical, proneural, neural and mesenchymal. 66 However, this classification has been superseded by other systems because of emerging evidence, including the IDH status.…”

“…Two studies assessed interpretability. 66,67 For example, humanrecognisable features deterministic of IDH mutational status were revealed using methods to make predictions understandable through dimensionality reduction of complex datasets. These characteristics included oligodendroglial cytomorphology and the extent of pleomorphism.…”

Automated glioma multiclass tumor classification

“…16 In addition, researchers expect that neuropathologists and machine learning models would make different sorts of categorization errors, and the aggregate assessment of a hybrid pathologist/machine learning model will be superior to either human or machine assessment alone. 17 Data used for the model training were digital images of tissue samples stained to reveal human leukocyte antigen (HLA-DR) staining on human glioma tissue microarrays (TMAs). 18 Sample preparation is described in the next session.…”