Generating Talking Face With Controllable Eye Movements by Disentangled Blinking Feature

Emotional Voice Puppetry

“…Hao et al present a two-stage approach for the generation of talking-face videos that have realistic controllable eye blinking capabilities [15]. Liu et al produced talking face that have controllable eye blinking capabilities which are driven by joint features of identity, audio and blinking [22]. Given the extensive level of research on the synthetization of talking face videos, this paper focuses on the animation of 3D stylized characters where the head and eye are animated by rig parameters.…”

Shenzhen Fengshen Industrial Development Co., Ltd v France Eurasian International Technology Development Co., Ltd

“…Then the features pass through a fully connected layer and output a 71-dimension vector which predicts the AU45 value in 6th dimension. In video-blink network, we adopt eye stage classifier [14] in our video-blink network and pretrain it with our dataset. In details, we use dlib to detect the landmarks movement of eyes in frames and get current size of eyes from eye stage classifier.…”

“…Nowadays, eye blinking has become an important part for current deep fake detection, related to speaker's habits and speech content. In the field of talking face, Liu et al [14] generates lip motions and controllable blinking actions with fixed head poses. Most research on eye movement relies on driven-video.…”