A deep-learning classifier identifies patients with clinical heart failure using whole-slide images of H&E tissue

Abstract

Over 26 million people worldwide suffer from heart failure annually. When the cause of heart failure cannot be identified, endomyocardial biopsy (EMB) represents the gold-standard for the evaluation of disease. However, manual EMB interpretation has high inter-rater variability. Deep convolutional neural networks (CNNs) have been successfully applied to detect cancer, diabetic retinopathy, and dermatologic lesions from images. In this study, we develop a CNN classifier to detect clinical heart failure from H&E stained whole-slide images from a total of 209 patients, 104 patients were used for training and the remaining 105 patients for independent testing. The CNN was able to identify patients with heart failure or severe pathology with a 99% sensitivity and 94% specificity on the test set, outperforming conventional feature-engineering approaches. Importantly, the CNN outperformed two expert pathologists by nearly 20%. Our results suggest that deep learning analytics of EMB can be used to predict cardiac outcome.

Deep Learning in Cardiology

“…Method Application/Notes a CT Lessman 2016 [195] CNN detect coronary calcium using three independently trained CNNs Shadmi 2018 [196] DenseNet compared DenseNet and u-net for detecting coronary calcium Cano 2018 [197] CNN 3D regression CNN for calculation of the Agatston score Wolterink 2016 [198] CNN detect coronary calcium using three CNNs for localization and two CNNs for detection Santini 2017 [199] CNN coronary calcium detection using a seven layer CNN on image patches Lopez 2017 [200] CNN thrombus volume characterization using a 2D CNN and postprocessing Hong 2016 [201] DBN detection, segmentation, classification of abdominal aortic aneurysm using DBN and image patches Liu 2017 [202] CNN left atrium segmentation using a twelve layer CNN and active shape model (STA13) de Vos 2016 [203] CNN 3D localization of anatomical structures using three CNNs, one for each orthogonal plane Moradi 2016 [204] CNN detection of position for a given CT slice using a pretrained VGGnet, handcrafted features and SVM Zheng 2015 [205] Multiple carotid artery bifurcation detection using multi-layer perceptrons and probabilistic boosting-tree Montoya 2018 [206] ResNet 3D reconstruction of cerebral angiogram using a 30 layer ResNet Zreik 2018 [207] CNN, AE identify coronary artery stenosis using CNN for LV segmentation and an AE, SVM for classification Commandeur 2018 [208] CNN quantification of epicardial and thoracic adipose tissue from non-contrast CT Gulsun 2016 [209] CNN extract coronary centerline using optimal path from computed flow field and a CNN for refinement CNN carotid intima media thickness video interpretation using two CNNs with two layers on Ultrasound Tom 2017 [226] GAN IVUS image generation using two GANs (IV11) Wang 2017 [227] CNN breast arterial calcification using a ten layer CNN on mammograms Liu 2017 [228] CNN CAC detection using CNNs on 1768 X-Rays Pavoni 2017 [229] CNN denoising of percutaneous transluminal coronary angioplasty images using four layer CNN Nirschl 2018 [230] CNN trained a patch-based six layer CNN for identifying heart failure in endomyocardial biopsy images Betancur 2018 [231] CNN trained a three layer CNN for obstructive CAD prediction from myocardial perfusion imaging a Results from these imaging modalities are not reported in this review because they were highly variable in terms of the research question they were trying to solve and highly inconsistent in respect with the use of metrics. Additionally all papers use private databases besides…”

Same same but different: A Web‐based deep learning application revealed classifying features for the histopathologic distinction of cortical malformations

“…Based on these tasks, more abstract functions like disease grading, prognosis prediction, and imaging biomarkers for genetic subtype identification have been established. 4,5 Successful examples include utilization in different types of cancer detection/classification/grading, 6,7 classification of liver cirrhosis, 8 heart failure detection, 9 and classification of Alzheimer plaques. 10 The most commonly used deep learning architectures are convolutional neural networks (CNNs; Figure 1C).…”

Same same but different: a web-based deep learning application for the histopathologic distinction of cortical malformations

“…Based on these tasks more abstract functions like disease grading, prognosis prediction and imaging biomarkers for genetic subtype identification have been established [4,5]. Successful examples range from utilization in different types of cancer detection/classification/grading [6,7], classification of liver cirrhosis [8], heart failure detection [9] and classification of Alzheimer plaques [10].…”