Non-invasive identification of swallows via deep learning in high resolution cervical auscultation recordings

Abstract

High resolution cervical auscultation is a very promising noninvasive method for dysphagia screening and aspiration detection, as it does not involve the use of harmful ionizing radiation approaches. Automatic extraction of swallowing events in cervical auscultation is a key step for swallowing analysis to be clinically effective. Using time-varying spectral estimation of swallowing signals and deep feed forward neural networks, we propose an automatic segmentation algorithm for swallowing accelerometry and sounds that works directly on the raw swallowing signals in an online fashion. The algorithm was validated qualitatively and quantitatively using the swallowing data collected from 248 patients, yielding over 3000 swallows manually labeled by experienced speech language pathologists. With a detection accuracy that exceeded 95%, the algorithm has shown superior performance in comparison to the existing algorithms and demonstrated its generalizability when tested over 76 completely unseen swallows from a different population. The proposed method is not only of great importance to any subsequent swallowing signal analysis steps, but also provides an evidence that such signals can capture the physiological signature of the swallowing process.Electronic human activity monitoring devices and wearable technology have evolved in the past decade from simple macrodetection of gross events such as the number of steps taken during a walk around the block, to the detection of micro-events that exist within each gross event 1 . As a result, the quantity of data generated by these devices has exponentially increased along with the clinical questions arising with this data challenge 2 . Therefore, efforts to automate signal analysis are receiving more attention. Any systematic analysis of signals requires an important first step in which individual signal events are demarcated or segmented from one another before detailed analysis of signal components can be performed. This necessitates the development of robust automatic event detection methods to reduce the number of manual steps in signal analysis, mitigating human error and guaranteeing consistent detection criteria 3 . Event extraction algorithms have been introduced in many applications including speech analysis 4 , heart sounds segmentation 5 , brain signals analysis 6 , and swallowing activity analysis 3,7 . Many of these algorithms relied on multi-channel data to improve detection quality 8,9 .All these applications share a common need of accurately defining the temporal borders (onset and offset) of certain events in order to be used for further processing and analysis. Particularly, we are interested in automated identification of vibratory and acoustic signals demarcating individual swallows using accelerometers and microphones 3 . Such automatic segmentation algorithms are critical for many applications that rely on swallowing sounds and vibrations which have been suggested as alternative bedside tools for dysphagia screening [10][11][12][13][14][15][16][17][18] ...

A Preliminary Investigation of Similarities of High Resolution Cervical Auscultation Signals Between Thin Liquid Barium and Water Swallows

“…Unlike VFSS, the noisy nature of HRCA signals makes their visual interpretation by clinicians extremely difficult due the presence of other signal components originating from different physiological processes such as coughing and head movement. On the other hand, advanced signal processing and machine learning techniques have produced several sets of preliminary evidence confirming the precision of automatic interpretation of HRCA signals in the detection of swallowing kinematic events and airway protection during swallowing [15] , [24] – [28] . For instance, HRCA has been shown to accurately track the hyoid bone throughout the duration of a swallow without assistance or supervision from human experts, with similar accuracy to these experts [27] , [29] .…”

Characterizing Effortful Swallows from Healthy Community Dwelling Adults Across the Lifespan Using High-Resolution Cervical Auscultation Signals and MBSImP Scores: A Preliminary Study

“…Another trained rater coded 10% of swallows for inter-rater reliability with intra-class coefficients (ICCs) of at least 0.9 [ 32 ]. The methods for swallow segmentation have been described in previous publications [ 14 , 33 ]. No other temporal kinematic measurements were performed aside from identifying the onset and offset of each swallow, and the sole purpose of these measurements was to segment the video files into individual swallows.…”

“…The onset and offset of swallows were taken from the segmented videos after applying the proper sampling mapping between videos and signals. The signals were then segmented using the mapped onset and offset times for feature extraction [ 33 ]. A summary of the features extracted from the HRCA signals and the explanations of their meanings can be viewed in Table 2 .…”

A review of Hidden Markov models and Recurrent Neural Networks for event detection and localization in biomedical signals

“…3D acceleration is an emerging technology as well, that has been extensively used in the assessment and detection of many medical conditions in swallowing [188] and human gait analysis [189]. In swallowing, acceleration signals have been used for the detection of pharyngeal swallowing activity via maximum likelihood methods with minimum description length in [16] and using short time Fourier transform and neural networks in [14]. RNNs were also employed for event detection in swallowing acceleration signals including the upper esophageal sphincter opening in [15,190], laryngeal vestibule closure [191], and hyoid bone motion during swallowing [192].…”