Automatic hyoid bone detection in fluoroscopic images using deep learning

Abstract

The displacement of the hyoid bone is one of the key components evaluated in the swallow study, as its motion during swallowing is related to overall swallowing integrity. In daily research settings, experts visually detect the hyoid bone in the video frames and manually plot hyoid bone position frame by frame. This study aims to develop an automatic method to localize the location of the hyoid bone in the video sequence. To automatically detect the location of the hyoid bone in a frame, we proposed a single shot multibox detector, a deep convolutional neural network, which is employed to detect and classify the location of the hyoid bone. We also evaluated the performance of two other state-of-art detection methods for comparison. The experimental results clearly showed that the single shot multibox detector can detect the hyoid bone with an average precision of 89.14% and outperform other auto-detection algorithms. We conclude that this automatic hyoid bone tracking system is accurate enough to be widely applied as a pre-processing step for image processing in dysphagia research, as well as a promising development that may be useful in the diagnosis of dysphagia.

Online Learning for the Hyoid Bone Tracking During Swallowing With Neck Movement Adjustment Using Semantic Segmentation

“…The performance of the proposed method were compared to that of the previous CNN-based the hyoid bone detection algorithm and the Single Shot MultiBox Detector (SSD) 500-VGG [11,27] after one-pass evaluation (OPE) [28]. The results show that the proposed method has an average area under curve score of 0.774 in the success plot of the OPE, as represented in Fig.…”

“…Recently, Zhang et al proposed a CNN-based model to detect coordinates of the hyoid bone in VFSS as an attempt to overcome these low recognition issues. The architecture used in this previous study was a Single Shot MultiBox Detector (SSD) and it was applied to 256 patients with swallowing difficulty [11]. However, this detection method using a SSD still has a limitation in that tracking failures can occur when the hyoid bone is overlapped by the mandible during swallowing in the fluoroscopic images.…”

“…Previously, various methods have been proposed to automatically track the swallowing structures such as the hyoid bone for kinematic analysis through image processing and machine learning algorithms. These algorithms include sobel edge detection [7,8] and active shape matching [9], Haar classifier matching [10] and local binary patterns in the image processing, and recently, convolutional neural networks (CNN) in machine learning [11]. However, in these previously proposed methods, manual corrections of the swallowing kinematic analyses was still required because tracking the hyoid bone is frequently failed when the image contrast is abruptly changed in the duration of passing through the mandible.…”

Automatic Detection of Airway Invasion from Videofluoroscopy via Deep Learning Technology

“…Most existing models use automatic tracking of the salient anatomical structures following human manual demarcation in the first few frames. The usefulness of the existing models is limited because they use obsolete semiautomated tracking and segmentation algorithms that still require tedious and time-consuming demarcation, no better than visual inspection in terms of efficiency [15,16].…”

Machine learning analysis to automatically measure response time of pharyngeal swallowing reflex in videofluoroscopic swallowing study

“…With the recent advances in research on machine learning in the medical field, several methods of machine learning-based VFSS analysis have been reported. Using the single shot multi-box detector, one of state-of-the-art deep learning methods for object detection, Zhang et al 15 developed a tracking system for the detection of the hyoid bone. However, it is challenging to analyze motion or action in the VFSS videos using this method, because this method focuses on detection of the spatial region of a single image rather than on the analysis of a sequence of images from a video.…”