A deep learning approach to estimate pulse rate by remote photoplethysmography

Abstract

Objective. This study proposes an U-net shaped Deep Neural Network (DNN) model to extract remote photoplethysmography (rPPG) signals from skin color signals to estimate Pulse Rate (PR). Approach. Three input window sizes are used into the DNN: 256 samples (5.12 s), 512 samples (10.24 s), and 1024 (20.48 s). A data argumentation algorithm based on interpolation is also used here to artificially increase the number of training samples. Main results. The proposed model outperformed a prior-knowledge rPPG method by using input signals with window of 256 and 512 samples. Also, it was found that the data augmentation procedure only increased the performance for window of 1024 samples. The trained model achieved a Mean Absolute Error (MAE) of 3.97 Beats per Minute (BPM) and Root Mean Squared Error (RMSE) of 6.47 BPM, for the 256 samples window, and MAE of 3.00 BPM and RMSE of 5.45 BPM for the window of 512 samples. On the other hand, the prior-knowledge rPPG method got a MAE of 8.04 BPM and RMSE of 16.63 BPM for the window of 256 samples, and MAE of 3.49 BPM and RMSE of 7.92 BPM for the window of 512. For the longest window (1024 samples), the concordance of the predicted PRs from the DNNs and the true PRs was higher when applying the data augmentation procedure. Significance. These results demonstrate a big potential of this technique for PR estimation, showing that the DNN proposed here may generate reliable rPPG signals even with short window lengths (5.12 s and 10.24 s), suggesting that it needs less data for a faster rPPG measurement and PR estimation.

Remote Photoplethysmography (rPPG): A State-of-the-Art Review

“…In addition to that, this method may not work well when there is high noise present in the signal. Research conducted also used machine learning techniques to measure remote HR [180], [181], [182], [183], [184], [185], [186]. Where the researcher [130] expanded on the work done by Poh et al [73].…”

Privacy Preserving Heart Rate Estimation from ECG and PPG Signals for Application in Remote Healthcare

“…A noteworthy work [43] proposed using various CNN variants combined with short-time Fourier transform, and outperformed existing model-driven methods. Among other neural network structures, U-net shaped deep neural network [44] and residual functional connectivity network (Residual FCN) has shown encouraging performance in heart rate estimation [45].…”

Graph connection Laplacian allows for enhanced outcomes of consumer camera based photoplethysmography imaging

“…At this point, we can mention the R-G algorithm [23] combining the red and green channels in the case of PPGI via an RGB camera, or the POS (Plane-Orthogonal-to-Skin) algorithm [24], which works with all three layers, or others such as, also in this case, the historically sorted G [12], which works only with the green layer or greyscale data, the aforementioned G-R [23], PCA [25], ICA [26], CHROM [27], PBV [28], 2SR [29], nonlinear-type time-frequency analysis like synchrosqueezing transform [30], the aforementioned POS [24] or Face2PPG [31]. Another approach is deep or machine learning methods [32], [33], which bring additional potential for feature extraction, classification and understanding of complex links within PPG signals. Another approach is using a lock in amplification method with the reference PPG signal either from external device or by averaging of skin pixels from whole recorded area [34], [35].…”

“…Others such as the historically sorted G [12], (which works only with the green layer or greyscale data), G-R [23], PCA [25], ICA [26], CHROM [27], PBV [28], 2SR [29], synchrosqueezing transform [30], the aforementioned POS [24] or Face2PPG [31]. A new approach is deep or machine learning methods [32], [33], which bring additional potential for feature extraction, classification and understanding of the complex links within the PPG signal. Also worth mentioning, is using a lock in amplification method with the reference PPG signal either from an external device or by the averaging of skin pixels from whole recorded area [34], [35].…”