Deep Learning Analysis to Automatically Detect the Presence of Penetration or Aspiration in Videofluoroscopic Swallowing Study

Abstract

Background Videofluoroscopic swallowing study (VFSS) is currently considered the gold standard to precisely diagnose and quantitatively investigate dysphagia. However, VFSS interpretation is complex and requires consideration of several factors. Therefore, considering the expected impact on dysphagia management, this study aimed to apply deep learning to detect the presence of penetration or aspiration in VFSS of patients with dysphagia automatically. Methods The VFSS data of 190 participants with dysphagia were collected. A total of 10 frame images from one swallowing process were selected (five high-peak images and five low-peak images) for the application of deep learning in a VFSS video of a patient with dysphagia. We applied a convolutional neural network (CNN) for deep learning using the Python programming language. For the classification of VFSS findings (normal swallowing, penetration, and aspiration), the classification was determined in both high-peak and low-peak images. Thereafter, the two classifications determined through high-peak and low-peak images were integrated into a final classification. Results The area under the curve (AUC) for the validation dataset of the VFSS image for the CNN model was 0.942 for normal findings, 0.878 for penetration, and 1.000 for aspiration. The macro average AUC was 0.940 and micro average AUC was 0.961. Conclusion This study demonstrated that deep learning algorithms, particularly the CNN, could be applied for detecting the presence of penetration and aspiration in VFSS of patients with dysphagia.

AI-Based Detection of Aspiration for Video-Endoscopy with Visual Aids in Meaningful Frames to Interpret the Model Outcome

“…Therefore, the final explanation provided by the system is effective and acceptable. This goes beyond most existing approaches for this task (except the VFSS approach [ 25 ]), because they only provide predictions or classifications without providing proper interpretable information for the diagnostician [ 28 , 35 , 37 , 42 , 43 , 62 , 63 ]. As this lack of transparency conflicts with EU GDPR, which prohibits decisions based solely on automated processing [ 44 , 45 ], a subsequent FEES or VFSS would become necessary, in any event, before critical decisions—such as abstinence from food, insertion of a nasogastric tube, or even re-intubation and tracheotomy—could be made.…”

“…Although no one, to date, has developed an AI tool to detect aspiration for FEES videos, various other attempts using machine-learning approaches to detect aspiration or signs of unsafe swallowing have been performed. The only high potential application is a CNN for aspiration detection of VFSS videos, with an accuracy of AUC of 1.00 [ 25 ], but, as described above, VFSS is limited in its clinical use. Further studies investigated the possibility of identifying dysphagia by means of localization of the hyoid bone or hyoid bone movements by an AI tool: on the one hand, the detection of auscultations, swallowing sounds, and vibrations is used [ 26 , 27 , 28 , 29 ], and on the other hand, video material (VFSS or ultrasound) [ 30 , 31 , 32 ] is used.…”

“…Combinations of various biometric data are also used for AI-based dysphagia diagnosis [ 40 ]. Studies looking at aspiration detection using image data (VFSS) [ 25 ] or swallow-onset detection [ 41 ] also yield promising results. A different approach is an image analysis of the external neck appearance for the detection of sarcopenic dysphagia [ 42 ].…”

A Retrospective Clinical Evaluation of an Artificial Intelligence Screening Method for Early Detection of STEMI in the Emergency Department

“…Artificial intelligence (AI) systems powered by deep learning technology have innovated many industries 5 and are being actively adopted in the medical field. 6 7 8 9 10 If there is an AI system that can predict STEMI as accurate as a human cardiologist using initial ECG alone so that a triage nurse or an ECG technician can activate CCL directly, we can expect a significant reduction in door-to-balloon (D2B) time and cost as well as a significant improvement in patient outcomes.…”

AI-based Detection of Aspiration for Video-endoscopy with Visual Aids in Meaningful Frames to Interpret the Model Outcome

“…Whilst up to date no one has developed an AI to detect aspiration for FEES videos, various other attempts using machine-learning approaches to detect aspiration or signs for unsafe swallowing have been performed. The only high potential application is a CNN for Aspiration detection of VFSS videos with an accuracy of AUC of 1.00 [25], but as described above, VFSS is limited in its clinical use. Further studies investigated the possibility of identifying dysphagia by means of localization of the hyoid bone or hyoid bone movements by an AI: On the one hand, the detection of auscultations, swallowing sounds and vibrations is used [26][27][28][29] and on the other hand, video material (VFSS or ultrasound) [30][31][32].…”