Deception Detection and Remote Physiological Monitoring: A Dataset and Baseline Experimental Results

Speth, Jeremy; Vance, Nathan; Czajka, Adam; Bowyer, Kevin W.; Wright, Diane; Flynn, Patrick J.

doi:10.1109/tbiom.2022.3218956

Cited by 8 publications

(4 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We used the generated bounding box coordinates to crop ROIs from each frame for each body part and downsized these ROIs to 64x64 pixels using bicubic interpolation. The RPNet model we used was trained on the DDPM dataset [37,42] where the frame rate is 90 frames per second (fps), which is the same as our MSPM dataset. The trained RPNet model was fed video clips of 135 frames (1.5 seconds) as described in the original paper [36].…”

Section: Learning-based Methodsmentioning

confidence: 99%

“…These three methods perform well for both stationary and moving subjects, which are characteristic of our MSPM dataset. The RPNet model was trained on face videos from the large and challenging deception detection and physiological monitoring dataset (DDPM) [37,42]. The model was not fine-tuned on any data from MSPM, so we tested how well the model could transfer to new subjects, lighting, standoff, and movement.…”

Section: Multi-site Rppg Approachmentioning

confidence: 99%

“…UBFC-rPPG [5] contains occasional movements and subjects under stress from mathematical games. DDPM [37,42] was the first long-form dataset containing unconstrained facial movements. Synthetic face rPPG datasets containing avatars also only contain faces [24,46].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Full-Body Cardiovascular Sensing with Remote Photoplethysmography

Lü¹,

Speth²,

Vance³

et al. 2023

Preprint

View full text Add to dashboard Cite

Remote photoplethysmography (rPPG) allows for noncontact monitoring of blood volume changes from a camera by detecting minor fluctuations in reflected light. Prior applications of rPPG focused on face videos. In this paper we explored the feasibility of rPPG from non-face body regions such as the arms, legs, and hands. We collected a new dataset titled Multi-Site Physiological Monitoring (MSPM), which will be released with this paper. The dataset consists of 90 frames per second video of exposed arms, legs, and face, along with 10 synchronized PPG recordings. We performed baseline heart rate estimation experiments from non-face regions with several state-of-the-art rPPG approaches, including chrominance-based (CHROM), planeorthogonal-to-skin (POS) and RemotePulseNet (RPNet). To our knowledge, this is the first evaluation of the fidelity of rPPG signals simultaneously obtained from multiple regions of a human body. Our experiments showed that skin pixels from arms, legs, and hands are all potential sources of the blood volume pulse. The best-performing approach, POS, achieved a mean absolute error peaking at 7.11 beats per minute from non-facial body parts compared to 1.38 beats per minute from the face. Additionally, we performed experiments on pulse transit time (PTT) from both the contact PPG and rPPG signals. We found that remote PTT is possible with moderately high frame rate video when distal locations on the body are visible. These findings and the supporting dataset should facilitate new research on nonface rPPG and monitoring blood flow dynamics over the whole body with a camera.

show abstract

Section: Learning-based Methodsmentioning

confidence: 99%

Section: Multi-site Rppg Approachmentioning

confidence: 99%

See 1 more Smart Citation

Full-Body Cardiovascular Sensing with Remote Photoplethysmography

Lü¹,

Speth²,

Vance³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…To discover deceptive cues, different methods have been proposed by scientists from different disciplines, ranging from psychologists and physiologists to technologists. For instance, methods based on the psycho-physiological detection of deception that uses mainly psychological tests and physiological monitoring have been applied [6,7]. A positive contribution is also given by technologists that developed methods for deceptive detection based on technological tools.…”

Section: Introductionmentioning

confidence: 99%

Detecting Deceptive Behaviours through Facial Cues from Videos: A Systematic Review

et al. 2023

View full text Add to dashboard Cite

Interest in detecting deceptive behaviours by various application fields, such as security systems, political debates, advanced intelligent user interfaces, etc., makes automatic deception detection an active research topic. This interest has stimulated the development of many deception-detection methods in the literature in recent years. This work systematically reviews the literature focused on facial cues of deception. The most relevant methods applied in the literature of the last decade have been surveyed and classified according to the main steps of the facial-deception-detection process (video pre-processing, facial feature extraction, and decision making). Moreover, datasets used for the evaluation and future research directions have also been analysed.

show abstract