Robust heart rate estimation from mobile face videos using an auto‐encoder

Abstract

Providing an easy‐to‐use and non‐invasive heart rate estimation system for home healthcare is imperative for detecting health deterioration in the early stages and for providing preventive treatments. To this end, we propose a ballistocardiographic method that estimates heart rate by analysing face videos and exploiting cyclic head motions caused by blood pumping through the carotid arteries during heartbeats. The proposed method addresses hand motion artefacts that occur while tracking head motions on a mobile platform. This is achieved by training an auto‐encoder that retrieves a heart rate‐related signal by learning consistent and statistically dominant relationships between facial landmarks. Moreover, unlike conventional methods, where a feature point tracker was used to track the facial landmarks, we propose using region‐based tracking to increase tracking robustness in challenging lighting conditions. Experimental results demonstrate that the proposed method performs best in challenging light conditions and can robustly manage hand motion artefacts on a mobile platform.

Robust Pulse Rate Measurements from Facial Videos in Diverse Environments

“…Previous research has demonstrated various methods for measuring pulse rate by applying independent component analysis (ICA), principal component analysis (PCA), fast Fourier transform (FFT), band pass filter (BPF) to RGB color data calculated in the region of interest (ROI) of facial images [ 1 , 7 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 ], and by analyzing head movement and blinking [ 12 , 14 , 35 ] under controlled laboratory conditions. However, in external environments (outdoor bench, car, drone), the pulse rate cannot be accurately measured due to factors such as the user’s fine body tremor, illumination changes, non-detection of the face, etc.…”