Vision-Based Measurement of Heart Rate from Ballistocardiographic Head Movements Using Unsupervised Clustering

Abstract

Heart rate has been measured comfortably using a camera without the skin-contact by the development of vision-based measurement. Despite the potential of the vision-based measurement, it has still presented limited ability due to the noise of illumination variance and motion artifacts. Remote ballistocardiography (BCG) was used to estimate heart rate from the ballistocardiographic head movements generated by the flow of blood through the carotid arteries. It was robust to illumination variance but still limited in the motion artifacts such as facial expressions and voluntary head motions. Recent studies on remote BCG focus on the improvement of signal extraction by minimizing the motion artifacts. They simply estimated the heart rate from the cardiac signal using peak detection and fast fourier transform (FFT). However, the heart rate estimation based on peak detection and FFT depend on the robust signal estimation. Thus, if the cardiac signal is contaminated with some noise, the heart rate cannot be estimated accurately. This study aimed to develop a novel method to improve heart rate estimation from ballistocardiographic head movements using the unsupervised clustering. First, the ballistocardiographic head movements were measured from facial video by detecting facial points using the good-feature-to-track (GFTT) algorithm and by tracking using the Kanade–Lucas–Tomasi (KLT) tracker. Second, the cardiac signal was extracted from the ballistocardiographic head movements by bandpass filter and principal component analysis (PCA). The relative power density (RPD) was extracted from its power spectrum between 0.75 Hz and 2.5 Hz. Third, the unsupervised clustering was performed to construct a model to estimate the heart rate from the RPD using the dataset consisting of the RPD and the heart rate measured from electrocardiogram (ECG). Finally, the heart rate was estimated from the RPD using the model. The proposed method was verified by comparing it with previous methods using the peak detection and the FFT. As a result, the proposed method estimated a more accurate heart rate than previous methods in three experiments by levels of the motion artifacts consisting of facial expressions and voluntary head motions. The four main contributions are as follows: (1) the unsupervised clustering improved the heart rate estimation by overcoming the motion artifacts (i.e., facial expressions and voluntary head motions); (2) the proposed method was verified by comparing with the previous methods using the peak detection and the FFT; (3) the proposed method can be combined with existing vision-based measurement and can improve their performance; (4) the proposed method was tested by three experiments considering the realistic environment including the motion artifacts, thus, it increases the possibility of the non-contact measurement in daily life.

Combining Photoplethysmography and Ballistocardiography to Address Voluntary Head Movements in Heart Rate Monitoring

“…Lee et al [38] proposed a similar method to [17]. However, for ROI selection, they considered only forehead and nose regions.…”

“…We compared our method with six conventional methods [11], [17], [19], [20], [24], [38]. As our method mainly depends on BCG methods, most of the methods chosen for the comparison were BCG methods.…”

“…For all the PPG and BCG methods in Table 1, we used both the Butterworth BPF [17] and MAF in the signal filtering step because recent methods, such as [19], [20], [38], saw a clear improvement by using several filters together.…”

Robust heart rate estimation from mobile face videos using an auto‐encoder

“…Lee et al [8 ] also proposed a similar method to that in [5 ], selecting the nose and forehead regions as the ROI. Rather than simply selecting the most dominant frequency (MDF) [5 ] as the heart rate, they proposed training a K‐means model.…”

“…The most non‐invasive and effective means to do this is to extract physiological changes on the human face using cameras. Generally, face video‐based methods can be divided into two major categories: photoplethysmography (PPG) methods [2–4 ] and ballistocardiography (BCG) methods [5–11 ].…”