Cross-Species 3D Face Morphing via Alignment-Aware Controller

Abstract

We address cross-species 3D face morphing (i.e., 3D face morphing from human to animal), a novel problem with promising applications in social media and movie industry. It remains challenging how to preserve target structural information and source fine-grained facial details simultaneously. To this end, we propose an Alignment-aware 3D Face Morphing (AFM) framework, which builds semantic-adaptive correspondence between source and target faces across species, via an alignment-aware controller mesh (Explicit Controller, EC) with explicit source/target mesh binding. Based on EC, we introduce Controller-Based Mapping (CBM), which builds semantic consistency between source and target faces according to the semantic importance of different face regions. Additionally, an inference-stage coarse-to-fine strategy is exploited to produce fine-grained meshes with rich facial details from rough meshes. Extensive experimental results in multiple people and animals demonstrate that our method produces high-quality deformation results.

A Scalable Vector Graphics Warping System for Anthropomorphizing Game Characters

“…Yan et al [27] proposed an alignment-aware 3D face morphing framework utilizing an encoder-decoder structure, effectively morphing three-dimensional human face mesh data into an animal face with alignment-aware control. Wang et al [28] proposed neural cages, a method for cage-based deformations that predicts deformations in a more natural space with greater accuracy than existing techniques.…”

Image Morphing Techniques: A Review

“…It should be noted 3d image morphing is also used to blend 3D human faces into different structures such as animals, but the problem with this morphing is their faces structures and feature are totally different. Later Yan et al presented a new technique for building semantic-adaptive correspondences between human and animal faces which helps preserve human features better [42]. The proposed Alignment-aware 3D Face Morphing framework applies morphing using an alignment-aware controller mesh using controller-based mapping, which builds multi-density correspondences between the source controller and the target controller according to the importance of semantic information [42].…”

“…Later Yan et al presented a new technique for building semantic-adaptive correspondences between human and animal faces which helps preserve human features better [42]. The proposed Alignment-aware 3D Face Morphing framework applies morphing using an alignment-aware controller mesh using controller-based mapping, which builds multi-density correspondences between the source controller and the target controller according to the importance of semantic information [42]. In fact, 3D image morphing techniques have developed rapidly in the last few years, Egger et al in [43] presented a detailed survey of 3d image morphing techniques and challenges over 20 years starting from the first proposed method.…”