Machine Learning Models and Videos of Facial Regions for Estimating Heart Rate: A Review on Patents, Datasets, and Literature

Pagano, Tiago Palma; Santos, Victor Rocha; Bonfim, Yasmin da Silva; Paranhos, José Vinícius Dantas; Ortega, Lucas Lemos; Sá, Paulo Henrique Miranda; Nascimento, Lian Filipe Santana; Winkler, Ingrid; Nascimento, Erick Giovani Sperandio

doi:10.3390/electronics11091473

Cited by 9 publications

(3 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To truly test the performance of the peak detection method we proposed, a comparison was made to other previously developed methods. Table 11 shows the comparison between our proposed method and other methods, such as those involving deep learning [31] and neural networks [32], with the commonly used metrics to evaluate their performance [33]. From Table 11, we can see that our proposed methodology is first in both RMSE and Me and ranked fourth in SD, which leads us to believe that the information and setup we are using can be reliable when it comes to computing HRV features.…”

Section: Discussionmentioning

confidence: 94%

Measuring Heart Rate Variability Using Facial Video

Martinez-Delgado

Correa-Balan

May-Chan

et al. 2022

Sensors

View full text Add to dashboard Cite

Heart Rate Variability (HRV) has become an important risk assessment tool when diagnosing illnesses related to heart health. HRV is typically measured with an electrocardiogram; however, there are multiple studies that use Photoplethysmography (PPG) instead. Measuring HRV with video is beneficial as a non-invasive, hands-free alternative and represents a more accessible approach. We developed a methodology to extract HRV from video based on face detection algorithms and color augmentation. We applied this methodology to 45 samples. Signals obtained from PPG and video recorded an average mean error of less than 1 bpm when measuring the heart rate of all subjects. Furthermore, utilizing PPG and video, we computed 61 variables related to HRV. We compared each of them with three correlation metrics (i.e., Kendall, Pearson, and Spearman), adjusting them for multiple comparisons with the Benjamini–Hochberg method to control the false discovery rate and to retrieve the q-value when considering statistical significance lower than 0.5. Using these methods, we found significant correlations for 38 variables (e.g., Heart Rate, 0.991; Mean NN Interval, 0.990; and NN Interval Count, 0.955) using time-domain, frequency-domain, and non-linear methods.

show abstract

Section: Discussionmentioning

confidence: 94%

Measuring Heart Rate Variability Using Facial Video

Martinez-Delgado

Correa-Balan

May-Chan

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…This systematic review followed the preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines [37] (Supplementary Materials) and was conducted using a method which encompasses five steps: planning, scoping, searching, assessing, and synthesizing [38,39]. This study is registered on open science framework, number https://osf.io/q3h2a accessed on 2 December 2022.…”

Section: Methodsmentioning

confidence: 99%

Bias and Unfairness in Machine Learning Models: A Systematic Review on Datasets, Tools, Fairness Metrics, and Identification and Mitigation Methods

Pagano

Loureiro

Lisboa

et al. 2023

BDCC

Self Cite

View full text Add to dashboard Cite

One of the difficulties of artificial intelligence is to ensure that model decisions are fair and free of bias. In research, datasets, metrics, techniques, and tools are applied to detect and mitigate algorithmic unfairness and bias. This study examines the current knowledge on bias and unfairness in machine learning models. The systematic review followed the PRISMA guidelines and is registered on OSF plataform. The search was carried out between 2021 and early 2022 in the Scopus, IEEE Xplore, Web of Science, and Google Scholar knowledge bases and found 128 articles published between 2017 and 2022, of which 45 were chosen based on search string optimization and inclusion and exclusion criteria. We discovered that the majority of retrieved works focus on bias and unfairness identification and mitigation techniques, offering tools, statistical approaches, important metrics, and datasets typically used for bias experiments. In terms of the primary forms of bias, data, algorithm, and user interaction were addressed in connection to the preprocessing, in-processing, and postprocessing mitigation methods. The use of Equalized Odds, Opportunity Equality, and Demographic Parity as primary fairness metrics emphasizes the crucial role of sensitive attributes in mitigating bias. The 25 datasets chosen span a wide range of areas, including criminal justice image enhancement, finance, education, product pricing, and health, with the majority including sensitive attributes. In terms of tools, Aequitas is the most often referenced, yet many of the tools were not employed in empirical experiments. A limitation of current research is the lack of multiclass and multimetric studies, which are found in just a few works and constrain the investigation to binary-focused method. Furthermore, the results indicate that different fairness metrics do not present uniform results for a given use case, and that more research with varied model architectures is necessary to standardize which ones are more appropriate for a given context. We also observed that all research addressed the transparency of the algorithm, or its capacity to explain how decisions are taken.

show abstract

“…It is possible to determine the heart rate from facial videos using Machine Learning (ML) [ 17 , 18 , 19 ]. ML enables the recognition of patterns in data containing multiple variables with discrete variations that have a significant impact on the outcomes [ 20 ].…”

Section: Introductionmentioning

confidence: 99%

Remote Heart Rate Prediction in Virtual Reality Head-Mounted Displays Using Machine Learning Techniques

Pagano

Santos²,

Santos³

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

Head-mounted displays are virtual reality devices that may be equipped with sensors and cameras to measure a patient’s heart rate through facial regions. Heart rate is an essential body signal that can be used to remotely monitor users in a variety of situations. There is currently no study that predicts heart rate using only highlighted facial regions; thus, an adaptation is required for beats per minute predictions. Likewise, there are no datasets containing only the eye and lower face regions, necessitating the development of a simulation mechanism. This work aims to remotely estimate heart rate from facial regions that can be captured by the cameras of a head-mounted display using state-of-the-art EVM-CNN and Meta-rPPG techniques. We developed a region of interest extractor to simulate a dataset from a head-mounted display device using stabilizer and video magnification techniques. Then, we combined support vector machine and FaceMash to determine the regions of interest and adapted photoplethysmography and beats per minute signal predictions to work with the other techniques. We observed an improvement of 188.88% for the EVM and 55.93% for the Meta-rPPG. In addition, both models were able to predict heart rate using only facial regions as input. Moreover, the adapted technique Meta-rPPG outperformed the original work, whereas the EVM adaptation produced comparable results for the photoplethysmography signal.

show abstract

Machine Learning Models and Videos of Facial Regions for Estimating Heart Rate: A Review on Patents, Datasets, and Literature

Cited by 9 publications

References 41 publications

Measuring Heart Rate Variability Using Facial Video

Measuring Heart Rate Variability Using Facial Video

Bias and Unfairness in Machine Learning Models: A Systematic Review on Datasets, Tools, Fairness Metrics, and Identification and Mitigation Methods

Remote Heart Rate Prediction in Virtual Reality Head-Mounted Displays Using Machine Learning Techniques

Contact Info

Product

Resources

About