Validation of machine learning models to detect amyloid pathologies across institutions

Abstract

Semi-quantitative scoring schemes like the Consortium to Establish a Registry for Alzheimer's Disease (CERAD) are the most commonly used method in Alzheimer's disease (AD) neuropathology practice. Computational approaches based on machine learning have recently generated quantitative scores for whole slide images (WSIs) that are highly correlated with human derived semi-quantitative scores, such as those of CERAD, for Alzheimer's disease pathology. However, the robustness of such models have yet to be tested in different cohorts. To validate previously published machine learning algorithms using convolutional neural networks (CNNs) and determine if pathological heterogeneity may alter algorithm derived measures, 40 cases from the Goizueta Emory Alzheimer's Disease Center brain bank displaying an array of pathological diagnoses (including AD with and without Lewy body disease (LBD), and / or TDP-43-positive inclusions) and levels of Aβ pathologies were evaluated. Furthermore, to provide deeper phenotyping, amyloid burden in gray matter vs whole tissue were compared, and quantitative CNN scores for both correlated significantly to CERAD-like scores. Quantitative scores also show clear stratification based on AD pathologies with or without additional diagnoses (including LBD and TDP-43 inclusions) vs cases with no significant neurodegeneration (control cases) as well as NIA Reagan scoring criteria. Specifically, the concomitant diagnosis group of AD + TDP-43 showed significantly greater CNN-score for cored plaques than the AD group. Finally, we report that whole tissue computational scores correlate better with CERAD-like categories than focusing on computational scores from a field of view with densest pathology, which is the standard of practice in neuropathological assessment per CERAD guidelines. Together these findings validate and expand CNN models to be robust to cohort variations and provide additional proof-of-concept for future studies to incorporate machine learning algorithms into neuropathological practice.

Deep learning from multiple experts improves identification of amyloid neuropathologies

“…Finally, prospective effectiveness outside this cohort remains open to exploration. Whereas this study was consistent with our earlier work leveraging one expert annotator 14 and its independent application at another institution 15 , we expect larger cohorts and institutionally-diverse datasets will facilitate more comprehensive standards in neuropathology. For instance, performance for CAA with 6E10 was lower than for other stains (Supplementary Figure 4).…”

“…In previous work, we automated a single expert’s annotations using DL 14 . This approach was validated by independent study with a different cohort 15 . However, individual bias of the expert remained, and it was not yet clear if these methods could scale across multiple experts, institutions, and data modalities—all of which are critical for assessing generalizability.…”

Decidual Vasculopathy Identification in Whole Slide Images Using Multiresolution Hierarchical Convolutional Neural Networks

“…Because variations in scanning techniques have the possibility of causing biases between local institutions, future development would benefit from crossinstitutional data sources to help validate the generalizability of the algorithm. However, some recent research 42,43 has found that regularized convolutional neural networks trained on single-institution data were robust to cohort variations during validation on data from other institutions.…”

Application of deep learning to understand resilience to Alzheimer's disease pathology

“…These approaches are becoming more common in AD research with the increase in available multimodal neuroimaging data to classify AD and predict progression from mild cognitive impairment (MCI) to AD ( 9 ). In neuropathology, deep learning has been recently used to classify and quantify tau pathology ( 10 ) as well as Aβ plaques and cerebral Aβ angiopathy ( 11 , 12 ).…”