Deep learning vs conventional learning algorithms for clinical prediction in Crohn's disease: A proof-of-concept study

Abstract

BACKGROUND Traditional methods of developing predictive models in inflammatory bowel diseases (IBD) rely on using statistical regression approaches to deriving clinical scores such as the Crohn's disease (CD) activity index. However, traditional approaches are unable to take advantage of more complex data structures such as repeated measurements. Deep learning methods have the potential ability to automatically find and learn complex, hidden relationships between predictive markers and outcomes, but their application to clinical prediction in CD and IBD has not been explored previously. AIM To determine and compare the utility of deep learning with conventional algorithms in predicting response to anti-tumor necrosis factor (anti-TNF) therapy in CD. METHODS This was a retrospective single-center cohort study of all CD patients who commenced anti-TNF therapy (either adalimumab or infliximab) from January 1, 2010 to December 31, 2015. Remission was defined as a C-reactive protein (CRP) < 5 mg/L at 12 mo after anti-TNF commencement. Three supervised learning algorithms were compared: (1) A conventional statistical learning algorithm using multivariable logistic regression on baseline data only; (2) A deep learning algorithm using a feed-forward artificial neural network on baseline data only; and (3) A deep learning algorithm using a recurrent neural network on repeated data. Predictive performance was assessed using area under the receiver operator characteristic curve (AUC) after 10× repeated 5-fold cross-validation. RESULTS A total of 146 patients were included (median age 36 years, 48% male). Concomitant therapy at anti-TNF commencement included thiopurines (68%), methotrexate (18%), corticosteroids (44%) and aminosalicylates (33%). After 12 mo, 64% had CRP < 5 mg/L. The conventional learning algorithm selected the following baseline variables for the predictive model: Complex disease behavior, albumin, monocytes, lymphocytes, mean corpuscular hemoglobin concentration and gamma-glutamyl transferase, and had a cross-validated AUC of 0.659, 95% confidence interval (CI): 0.562-0.756. A feed-forward artificial neural network using only baseline data demonstrated an AUC of 0.710 (95%CI: 0.622-0.799; P = 0.25 vs conventional). A recurrent neural network using repeated biomarker measurements demonstrated significantly higher AUC compared to the conventional algorithm (0.754, 95%CI: 0.674-0.834; P = 0.036). CONCLUSION Deep learning methods are feasible and have the potential for stronger predictive performance compared to conventional model building methods when applied to predicting remission after anti-TNF therapy in CD.

Machine learning models and over-fitting considerations

“…Con et al [ 1 ] explore artificial intelligence (AI) in a classification problem of predicting biochemical remission of Crohn’s disease at 12 mo post-induction with infliximab or adalimumab. They illustrate that, after applying appropriate machine learning (ML) methodologies, ML methods outperform conventional multivariable logistic regression (a statistical learning algorithm).…”

“…, yes/no binary outcomes). In this study[ 1 ], the outcome prevalence was 64% ( n ≈ 93). With a chosen k = 5, the training folds would comprise 80% of that data, leading to approximately 74 positive cases of biochemical remission.…”

“…The number of positive outcomes in each training fold must be considered, especially in logistic regression, where the rule of thumb recommends at least ten positive events per independent predictor, to minimize over-fitting[ 3 ]. In this study[ 1 ], six predictors were eventually used in the multivariable model, making over-fitting less likely from a model-specification standpoint. Finally, k-folds are recommended to be stratified by the outcome, so the outcome prevalence is equal among the training and testing folds.…”

“…This becomes crucial when the prevalence of outcome of interest is < 10%-20% (imbalanced classification problem). While imbalanced classification is not an issue in this study[ 1 ], the authors did not mention whether they used outcome-stratified k-folds.…”

Cross-scale Attention Guided Multi-instance Learning for Crohn's Disease Diagnosis with Pathological Images

“…Pathologists carefully examine biopsies at multiple scales through microscopes to examine morphological patterns [6], which is a laborious task. With the rapid development of whole slide imaging (WSI) and deep learning methods, computerassisted CD clinical predicion and exploration [9,18,19,27] are increasingly promising endeavors. However, annotating images pixel-or patch-wise is computationally expensive for a standard supervised learning system [11,16,23,24].…”