Voice2Face: Audio‐driven Facial and Tongue Rig Animations with cVAEs

Abstract

Figure 1: System overview. During training, two cVAEs are used to encode and generate facial and tongue mesh animations conditioned on speech. During inference, fixed latent vectors are used by the decoders to generate mesh animation sequences, that are then transformed into rig space via models approximating the inverse rig function.