A Wide and Deep Transformer Neural Network for 12-Lead ECG Classification

Abstract

Cardiac abnormalities are a leading cause of death and their early diagnosis are of importance for providing timely interventions. The goal of 2020 PhysioNet/CinC challenge was to develop algorithms to diagnose multiple cardiac abnormalities using 12-lead ECG data. In this work, we develop a wide and deep transformer neural network to classify each 12-lead ECG sequence into 27 cardiac abnormality classes. Our approach combines handcrafted ECG features, which were determined to be important by a random forest model, and discriminative feature representations that are automatically learned from a transformer neural network. Our entry to the 2020 Phys-ioNet/CinC challenge placed 1 st out of 41 official ranking teams (team name = prna). Using the official generalized weighted accuracy metric for evaluation, we achieved a validation score of 0.587 and top score of 0.533 on the full held-out test set.

Potential of Rule-Based Methods and Deep Learning Architectures for ECG Diagnostics

“…Various teams that participated in the Physionet/Challenge considered the deep learning approach [ 27 , 28 , 29 , 30 ], showing a particular interest in this methodology. For example, the team with the highest score [ 27 ] considered both raw ECG data and ECG features extracted from ECG signals, including age and gender.…”

“…Various teams that participated in the Physionet/Challenge considered the deep learning approach [ 27 , 28 , 29 , 30 ], showing a particular interest in this methodology. For example, the team with the highest score [ 27 ] considered both raw ECG data and ECG features extracted from ECG signals, including age and gender. A deep neural network with a modified residual neural network architecture was considered in [ 28 ], in which the scatter blocks processed the 12 leads separately.…”

Practical Lessons on 12-Lead ECG Classification: Meta-Analysis of Methods From PhysioNet/Computing in Cardiology Challenge 2020

“…Data distributions can be unified and the influence of noise and outliers can be alleviated through normalization. Other signal processing techniques such as zeropadding (Natarajan et al, 2020), median filters (Hsu et al, 2020), and wavelet transformation denoising (Zhu et al, 2020) can also be used.…”

“…Another advantage is that RNNs and Transformer can handle inputs of various lengths, which is also sometimes necessary for time series data. Some teams combine two kinds of architectures in Challenge 2020 (Fayyazifar et al, 2020;Hasani et al, 2020;Natarajan et al, 2020;Oppelt et al, 2020) The results are notable: 4 highest-ranking teams all add the attention mechanism to their models, showing the prevalence of attention. The result of the Mann-Whitney U-test also proves that attention can improve the performance of models (p-value is 0.0059, less than 0.01).…”

“…We notice that the three highest-ranking teams used the model ensemble (Natarajan et al, 2020;Zhao et al, 2020;Zhu et al, 2020), but only 14 out of 41 teams employed this strategy. The results are not expected, and we believe that this is because the model ensemble can help improve the single model, whereas it is less meaningful to compare among different teams.…”