Face Illumination Transfer and Swapping via Dense Landmark and Semantic Parsing

Jin, Xin; Li, Zhonglan; Ning, Ning; Lu, Huimin; Li, Xiaodong; Zhang, Xiaokun; Zhu, Xingfan; Fang, Xi

doi:10.1109/jsen.2020.3025918

Cited by 3 publications

(3 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Numerous other approaches for different applications such as face expression and emotion recognition are based on the ERT model presented in [ 14 ]. While the approach presented in [ 14 ] concerns 2D face shape alignment, X. Jin in [ 17 ] extended this method in [ 17 ] to detect 68 landmarks on face images based on a 3D model. The illumination of the images is also taken into consideration in [ 17 ].…”

Section: Introductionmentioning

confidence: 99%

“…While the approach presented in [ 14 ] concerns 2D face shape alignment, X. Jin in [ 17 ] extended this method in [ 17 ] to detect 68 landmarks on face images based on a 3D model. The illumination of the images is also taken into consideration in [ 17 ]. The authors in [ 18 ] were also based on [ 14 ] to correct the face shapes according to the estimated landmarks positions.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A High Performance and Robust FPGA Implementation of a Driver State Monitoring Application

Christakos

Petrellis

Mousouliotis

et al. 2023

Sensors

View full text Add to dashboard Cite

A high-performance Driver State Monitoring (DSM) application for the detection of driver drowsiness is presented in this paper. The popular Ensemble of Regression Trees (ERTs) machine learning method has been employed for the alignment of 68 facial landmarks. Open-source implementation of ERTs for facial shape alignment has been ported to different platforms and adapted for the acceleration of the frame processing speed using reconfigurable hardware. Reducing the frame processing latency saves time that can be used to apply frame-to-frame facial shape coherency rules. False face detection and false shape estimations can be ignored for higher robustness and accuracy in the operation of the DSM application without sacrificing the frame processing rate that can reach 65 frames per second. The sensitivity and precision in yawning recognition can reach 93% and 97%, respectively. The implementation of the employed DSM algorithm in reconfigurable hardware is challenging since the kernel arguments require large data transfers and the degree of data reuse in the computational kernel is low. Hence, unconventional hardware acceleration techniques have been employed that can also be useful for the acceleration of several other machine learning applications that require large data transfers to their kernels with low reusability.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A High Performance and Robust FPGA Implementation of a Driver State Monitoring Application

Christakos

Petrellis

Mousouliotis

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…The acceleration and robustness techniques presented in this paper can also be exploited in these approaches. For example, X. Jin in [13] extended the approach in [11] to detect 68 landmarks on face images based on a 3D model, while the illumination of the images is also taken into consideration. The authors in [14] used the approach of [11] to define the landmarks from the dataset they use and correct the face shapes according to the estimated landmarks positions.…”

Section: Introductionmentioning

confidence: 99%

A High Performance and Robust FPGA Implementation of a Driver State Monitoring Application

Christakos¹,

Petrellis²,

Mousouliotis³

et al. 2023

Preprint

View full text Add to dashboard Cite

A high performance Driver State Monitoring (DSM) application for the detection of driver drowsiness is presented in this paper. It relies on the usage of an Ensemble of Regression Trees (ERTs) machine learning method that aligns 68 facial landmarks. Special focus is given on the acceleration of the frame processing using reconfigurable hardware. Reducing the frame processing latency saves time that can be used to apply frame-to-frame facial shape coherency rules. False face detection and false shape estimations can be ignored for higher robustness and accuracy in the operation of the DSM application without reducing the frame processing rate that can reach 65 frames per second. The sensitivity and precision in yawning recognition can reach 93% and 97%, respectively. The implementation of the employed DSM algorithm in reconfigurable hardware is challenging since the kernel arguments require large data transfers and the degree of data reuse in the computational kernel is low. Due to this, unconventional hardware acceleration techniques have been employed that can also be useful for the acceleration of several other applications.

show abstract

A survey on deep learning based reenactment methods for deepfake applications

Dhanyalakshmi,

Popirlan,

Hemanth

2024

IET Image Processing

View full text Add to dashboard Cite

Among the sectors that deep learning has transformed, deepfake, a novel method of manipulating multimedia, deserves particular attention. The long‐term objective of many researchers is to seamlessly mimic human facial movement or whole‐body activity, referred to as reenactment. Deepfake progress has made this goal much more feasible in recent years. Yet, achieving more realistic facial and body reenactment remains a challenging task. The primary focus of this study is to explore the current capability of the reenactment techniques and expand them further to attain greater results. The analysis offers a thorough overview of the various techniques involved, the challenges addressed, the datasets utilized, and the metrics employed by the underlying methods of reenactment technologies. The study also addresses the potential risks and their mitigating strategies to ensure responsible reenactment techniques. To the best of the authors' knowledge, this is the first survey paper that delves deeper into the topic of deepfake reenactment.

show abstract

Face Illumination Transfer and Swapping via Dense Landmark and Semantic Parsing

Cited by 3 publications

References 29 publications

A High Performance and Robust FPGA Implementation of a Driver State Monitoring Application

A High Performance and Robust FPGA Implementation of a Driver State Monitoring Application

A High Performance and Robust FPGA Implementation of a Driver State Monitoring Application

A survey on deep learning based reenactment methods for deepfake applications

Contact Info

Product

Resources

About