PPG3D: Does 3D head tracking improve camera-based PPG estimation?

Abstract

Over the last few years, camera-based estimation of vital signs referred to as imaging photoplethysmography (iPPG) has garnered significant attention due to the relative simplicity, ease, unobtrusiveness and flexibility offered by such measurements. It is expected that iPPG may be integrated into a host of emerging applications in areas as diverse as autonomous cars, neonatal monitoring, and telemedicine. In spite of this potential, the primary challenge of non-contact camera-based measurements is the relative motion between the camera and the subjects. Current techniques employ 2D feature tracking to reduce the effect of subject and camera motion but they are limited to handling translational and in-plane motion. In this paper, we study, for the first-time, the utility of 3D face tracking to allow iPPG to retain robust performance even in presence of out-of-plane and large relative motions. We use a RGB-D camera to obtain 3D information from the subjects and use the spatial and depth information to fit a 3D face model and track the model over the video frames. This allows us to estimate correspondence over the entire video with pixel-level accuracy, even in the presence of out-of-plane or large motions. We then estimate iPPG from the warped video data that ensures per-pixel correspondence over the entire window-length used for estimation. Our experiments demonstrate improvement in robustness when head motion is large.

Heart Rate Estimation from Facial Image Sequences of a Dual-Modality RGB-NIR Camera

“…Regev et al [12] and Yang et al [14] proposed the capturing of depth video (without RGB information) of a human subject using a Kinect or RealSense camera to estimate the heart rate. On the other hand, Dosso et al [8] estimated the heart rate based on fusion of three streams (RGB, NIR, and depth) via consensus voting, in contrast to others which used depth for ROI (Region of Interest) extraction or head pose estimation [10,15]. Though depth information was adopted by some of them, the instability in measurement accuracy actually prevents it from accurate heart rate estimation.…”