Rig-space Neural Rendering

Abstract

Figure 1: With rig-space neural rendering, we train a deep neural network (DNN) to generate an image of a character directly from rig parameters, such as skeleton joint angles, or lattice cage positions. We first generate the training set (blue area) by posing the character in many poses and views, while saving the rig parameters associated to the images. We then train a DNN to generate the character image from the rig parameters (red area). Finally, at run-time (green area) we control the rig parameters and feed them to our Rig2Image network that generates an image in real-time, which we then overlay onto scene background. Details on how to extend this to dynamic lights and interactions with other scene objects are shown in figures bellow.