Implicit neural representation for physics-driven actuated soft bodies

Abstract

Active soft bodies can affect their shape through an internal actuation mechanism that induces a deformation. Similar to recent work, this paper utilizes a differentiable, quasi-static, and physics-based simulation layer to optimize for actuation signals parameterized by neural networks. Our key contribution is a general and implicit formulation to control active soft bodies by defining a function that enables a continuous mapping from a spatial point in the material space to the actuation value. This property allows us to capture the signal's dominant frequencies, making the method discretization agnostic and widely applicable. We extend our implicit model to mandible kinematics for the particular case of facial animation and show that we can reliably reproduce facial expressions captured with high-quality capture systems. We apply the method to volumetric soft bodies, human poses, and facial expressions, demonstrating artist-friendly properties, such as simple control over the latent space and resolution invariance at test time.

Local anatomically-constrained facial performance retargeting

“…Recently Choi et al proposed Animatomy [15], a muscle fiber based anatomical basis for animator friendly face modeling applications. Lastly we recognize several physically based face models [39,41,44,48] which inherently have the ability to model anatomy constraints through simulation.…”

“…Since this structure does not exist in reality and is, therefore, not available for supervised learning, we formulate a learning framework where such rigidly deforming surface can be learnt only from the sparse set of anatomic constraints that can be computed between the skin and the underlying bones. As we will see in Section 5, learning this anatomic surface from data leads to several interesting applications in shape manipulation and performance retargeting that were previously challenging to obtain without expensive physical simulation [48] or extensive volumetric data capture [37].…”

“…As they are ubiquitously used both in high-end production workflows and lightweight consumer applications, building expressive face models for various applications continues to remain an active area of research [17]. Face models today can range from simple linear global shape models [4,27,29] to highly complex local models that incorporate the underlying facial anatomy through physical simulation [15,44,48] or through anatomical constraints [47].…”

“…As a result, fitting these anatomical face models to a target scan or an image for instance, is a computationally intensive procedure taking several minutes per frame on a CPU, or requires hand crafted GPU solvers [20]. Furthermore anatomy constraints are enforced only in sparse regions of the face, whereas in reality the facial skin surface is more densely constrained by the underlying anatomy, and simulating this dense interaction between the anatomy and facial skin through physical simulation can be even more computationally intensive [39,48].…”