Transformer-Based Spatio-Temporal Analysis for Classification of Aortic Stenosis Severity From Echocardiography Cine Series

Abstract

Aortic stenosis (AS) is characterized by restricted motion and calcification of the aortic valve and is the deadliest valvular cardiac disease. Assessment of AS severity is typically done by expert cardiologists using Doppler measurements of valvular flow from echocardiography. However, this limits the assessment of AS to hospitals staffed with experts to provide comprehensive echocardiography service. As accurate Doppler acquisition requires significant clinical training, in this paper, we present a deep learning framework to determine the feasibility of AS detection and severity classification based only on two-dimensional echocardiographic data. We demonstrate that our proposed spatio-temporal architecture effectively and efficiently combines both anatomical features and motion of the aortic valve for AS severity classification. Our model can process cardiac echo cine series of varying length and can identify, without explicit supervision, the frames that are most informative towards the AS diagnosis. We present an empirical study on how the model learns phases of the heart cycle without any supervision and frame-level annotations. Our architecture outperforms state-of-the-art results on a private and a public dataset, achieving 95.2% and 91.5% in AS detection, and 78.1% and 83.8% in AS severity classification on the private and public datasets, respectively. Notably, due to the lack of a large public video dataset for AS, we made slight adjustments to our architecture for the public dataset. Furthermore, our method addresses common problems in training deep networks with clinical ultrasound data, such as a low signal-to-noise ratio and frequently uninformative frames. Our source code is available at: https://github.com/ neda77aa/FTC.git

Artificial Intelligence-Enhanced Comprehensive Assessment of the Aortic Valve Stenosis Continuum in Echocardiography

“…Several studies used CNNs to extract AVS-related features from 2D TTE videos through end-to-end learning without requiring Doppler information. 5,7,19,20 The implications of our AI-based system extend beyond precise AVS diagnosis. Our DLi-AVSc exhibits significant prognostic capability, comparable to traditional AVS parameters, even when utilizing only PLAX and PSAX views.…”

“…In that case, a high DLi-AVSc can suggest the likelihood of significant AVS, thereby guiding further necessary evaluations.Another strength of our study is that, unlike previous research, it reflects the continuous nature of AVS progression. For instance, Wessler et al trained CNNs to classify AVS severity into three categories (no, early, and significant AVS) using limited 2D images.7 Similarly, Ahmadi et al proposed a transformer-based spatiotemporal architecture to classify AVS into four categories (normal, mild, moderate, and severe AVS) by capturing anatomical features and AV motion 19. Vaseli et al focused on model explainability in AVS severity classification, incorporating uncertainty estimation and classifying AVS severity into three classes (no, early, and significant AVS).…”

Ultrasound Image Analysis with Vision Transformers—Review

“…Ahmadi et al [ 63 ] examined aortic stenosis severity by focusing on the temporal localization of the opening and closing of the valve, and the shape and mobility of the aortic valve. They applied Temporal Deformable Attention (TDA) in frame-level embedding to enhance the transformers’ understanding of locality and a temporal coherent loss to increase its sensitivity to minor aortic valve movements.…”