Data analysis with Shapley values for automatic subject selection in Alzheimer’s disease data sets using interpretable machine learning

Abstract

Background For the recruitment and monitoring of subjects for therapy studies, it is important to predict whether mild cognitive impaired (MCI) subjects will prospectively develop Alzheimer’s disease (AD). Machine learning (ML) is suitable to improve early AD prediction. The etiology of AD is heterogeneous, which leads to high variability in disease patterns. Further variability originates from multicentric study designs, varying acquisition protocols, and errors in the preprocessing of magnetic resonance imaging (MRI) scans. The high variability makes the differentiation between signal and noise difficult and may lead to overfitting. This article examines whether an automatic and fair data valuation method based on Shapley values can identify the most informative subjects to improve ML classification. Methods An ML workflow was developed and trained for a subset of the Alzheimer’s Disease Neuroimaging Initiative (ADNI) cohort. The validation was executed for an independent ADNI test set and for the Australian Imaging, Biomarker and Lifestyle Flagship Study of Ageing (AIBL) cohort. The workflow included volumetric MRI feature extraction, feature selection, sample selection using Data Shapley, random forest (RF), and eXtreme Gradient Boosting (XGBoost) for model training as well as Kernel SHapley Additive exPlanations (SHAP) values for model interpretation. Results The RF models, which excluded 134 of the 467 training subjects based on their RF Data Shapley values, outperformed the base models that reached a mean accuracy of 62.64% by 5.76% (3.61 percentage points) for the independent ADNI test set. The XGBoost base models reached a mean accuracy of 60.00% for the AIBL data set. The exclusion of those 133 subjects with the smallest RF Data Shapley values could improve the classification accuracy by 2.98% (1.79 percentage points). The cutoff values were calculated using an independent validation set. Conclusion The Data Shapley method was able to improve the mean accuracies for the test sets. The most informative subjects were associated with the number of ApolipoproteinE ε4 (ApoE ε4) alleles, cognitive test results, and volumetric MRI measurements.

Machine learning application for classification of Alzheimer's disease stages using 18F-flortaucipir positron emission tomography

“…We explained the classification prediction of our model with SHAP [ 47 , 48 ], which is one of the methods to explain the model with feature importance and is inspired by the concept of coalition game theory by replacing “player” in coalition game theory with “data feature” in tabular data. In the ML context, the agents correspond to the features of the data, and the goal is to explain the model's prediction.…”

Predictive models for small-for-gestational-age births in women exposed to pesticides before pregnancy based on multiple machine learning algorithms

“…In addition, we used the Shapley Additive Explanation (SHAP) method to apply the post hoc explainability on the prediction models based on CatBoost, GBDT and XGBoost classifiers, in order to interpret the impact of variables on the prediction outcome. SHAP uses game theory for evaluating the impact of specific input variables to the outcome of a certain model ( 36 ). Moreover, decision curve analysis (DCA) was applied to evaluate the net benefit of the prediction models based on CatBoost, GBDT and XGBoost algorithms.…”

Artificial Intelligence for Risk Prediction of Rehospitalization with Acute Kidney Injury in Sepsis Survivors

“…The prediction abilities of various machine learning models were examined based on the area under the curve of receiver operating characteristics (AUCs) and precision-recall curves of each model. As the methods of prediction in machine learning models are often unclear, we used SHapley Additive exPlanation (SHAP) values to provide accurate attribution values for each clinical feature in our prediction model [33][34][35]. The data were analyzed by using Python (Python Software Foundation version 3.7.6, available at http://www.python.org, accessed on 1 November 2021).…”