Identifying the Presence and Severity of Dementia by Applying Interpretable Machine Learning Techniques on Structured Clinical Records

Abstract

Background: Dementia occurs through declining of cognitive abilities, and its early detection stands essential for effective preventive measures. However, mainstream diagnostic tests and screening tools, such as Cambridge Cognition Examination (CAMCOG) and Mini-Mental State Examination (MMSE), often fail to detect dementia accurately. Various graph-based or feature-dependent prediction and progression models have been proposed. Whenever these models exploit information in the patients’ electronic medical records, they represent promising options to identify the presence and severity of dementia more precisely. Methods: The methods presented in this paper aim to address two problems related to dementia: (a) Basic diagnosis: identifying the presence of dementia in individuals, and (b) Severity diagnosis: predicting the presence of dementia, as well as the severity of the disease. We formulate these two tasks as classification problems and address them using machine learning models based on random forests, analysing structured clinical data from an elderly population cohort. We perform a hybrid data curation method where a dementia expert is in the loop to ensure meaningful curation decisions. We then employ the machine learning algorithms that classify individual episodes into a specific dementia class. Decision trees are also used for enhancing the explainability of decisions made by prediction models, allowing medical experts to identify the most crucial patient features and their threshold values for the detection of dementia. Results: Our experiment results prove that clinical record features (such as age or comorbidities), along with baseline arithmetic or memory tests, can predict dementia and its severity with high accuracy. In specific, our prediction models have reached an average f1-score of 0.93 and 0.85 for problems (a) and (b), respectively. Moreover, the decision trees produced for the two issues empower the interpretability of the prediction models. Conclusions: This study proves that there can be an accurate estimation of the existence and severity of dementia disease by analysing various clinical features and cognitive tests from the past episodes of the elderly population. Moreover, a set of rules represent building blocks in the efficiently patient classification. Relevant clinical and screening test features (e.g., simple arithmetic or animal fluency task) represent precise predictors without calculating the scores of mainstream cognitive tests such as MMSE and CAMCOG. They identify not only meaningful features but also justifications of classifications. As a result, the predictive power of machine learning models over curated clinical data is demonstrated, paving the way for accurate diagnosis of dementia.

An app for predicting patient dementia classes using convolutional neural networks (CNN) and artificial neural networks (ANN): Comparison of prediction accuracy in Microsoft Excel

“…There are numerous underlying causes (e.g., Alzheimer disease, which is the most prevalent at least in later life) in addition to the biology and pathophysiology of dementia, all of which are influenced by different factors (e.g., comorbidity, lifestyle, and genetics). [3] To differentiate dementia from patients’ conditions, we require individualized and precise treatment for patients in the future. According to the approach described in this paper, numerous factors result in unpredictability that impacts the accuracy and efficiency of prediction.…”

“…Although dementia cannot be diagnosed by a single diagnostic test, clinicians employ a variety of tools and tests to detect dementia, whether it is caused by Alzheimer disease or by another disease. [3] One of the most common tests for assessing cognitive impairment is the mini mental state examination (MMSE). [4] The diagnostic accuracy of the MMSE has been investigated at various cut points for dementia in people >65 years of age.…”

“…[13][14][15] Models of this type have also been used to assess the risk of a person converting to dementia from a stage of mild cognitive impairment. [16,17] Another study [3] conducted and analyzed by demographic and clinical data from EMRs to determine individual patients' risk scores for dementia based on both structured and unstructured data from EMRs. A similar retrospective study was performed [18] using structured data, including medical diagnoses, primary care tests, investigations, lifestyle information, and prescription data.…”

“…A similar retrospective study was performed [18] using structured data, including medical diagnoses, primary care tests, investigations, lifestyle information, and prescription data. The study [3] analyzed structured clinical data from an elderly population cohort and found that their prediction models reached average F1-scores of 0.93 and 0.81 for both basic and severity diagnoses, respectively.…”

“…ML algorithms are typically used as black boxes without the ability to interpret individual predictions. [3] To provide clinicians with a clear understanding of the rationale for using ML, one emerging challenge is to make it easy and clear to understand the process of ML. Microsoft Excel is a familiar program to many researchers.…”