Data-driven autocompletion for keyframe animation

Zhang, Xinyi; Panne, Michiel van de

doi:10.1145/3274247.3274502

Cited by 26 publications

(19 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This can make combinations of actions look scripted and sequential. The scalability and expressiveness of deep neural networks has been applied to keyframe animation by Zhang et al [2018], who use an RNN conditioned on key-frames to produce jumping motions for a simple 2D model. Harvey et al…”

Section: Transition Generationmentioning

confidence: 99%

Robust motion in-betweening

et al. 2020

View full text Add to dashboard Cite

In this work we present a novel, robust transition generation technique that can serve as a new tool for 3D animators, based on adversarial recurrent neural networks. The system synthesises high-quality motions that use temporally-sparse keyframes as animation constraints. This is reminiscent of the job of in-betweening in traditional animation pipelines, in which an animator draws motion frames between provided keyframes. We first show that a state-of-the-art motion prediction model cannot be easily converted into a robust transition generator when only adding conditioning information about future keyframes. To solve this problem, we then propose two novel additive embedding modifiers that are applied at each timestep to latent representations encoded inside the network's architecture. One modifier is a time-to-arrival embedding that allows variations of the transition length with a single model. The other is a scheduled target noise vector that allows the system to be robust to target distortions and to sample different transitions given fixed keyframes. To qualitatively evaluate our method, we present a custom MotionBuilder plugin that uses our trained model to perform in-betweening in production scenarios. To quantitatively evaluate performance on transitions and generalizations to longer time horizons, we present well-defined in-betweening benchmarks on a subset of the widely used Human3.6M dataset and on LaFAN1, a novel high quality motion capture dataset that is more appropriate for transition generation. We are releasing this new dataset along with this work, with accompanying code for reproducing our baseline results.

show abstract

Section: Transition Generationmentioning

confidence: 99%

Robust motion in-betweening

et al. 2020

View full text Add to dashboard Cite

show abstract

“…In the context of code, Pythia [38] offers code completion based on a neural net to rank method and API suggestions. Work on creative domains demonstrated that an RNN, trained on physics-based simulations, can be used to autocomplete keyframe animations [50]. Also, Hsu et al [19] presented autocompletion for aggregate elements that can be used for 2D planes, 3D surfaces, and 3D volumes.…”

Section: Autocompletion Is Not Only For Textmentioning

confidence: 99%

Examining Autocompletion as a Basic Concept for Interaction with Generative AI

Lehmann

Buschek

2020

I-Com

View full text Add to dashboard Cite

Autocompletion is an approach that extends and continues partial user input. We propose to interpret autocompletion as a basic interaction concept in human-AI interaction. We first describe the concept of autocompletion and dissect its user interface and interaction elements, using the well-established textual autocompletion in search engines as an example. We then highlight how these elements reoccur in other application domains, such as code completion, GUI sketching, and layouting. This comparison and transfer highlights an inherent role of such intelligent systems to extend and complete user input, in particular useful for designing interactions with and for generative AI. We reflect on and discuss our conceptual analysis of autocompletion to provide inspiration and a conceptual lens on current challenges in designing for human-AI interaction.

show abstract

“…To accelerate keyframe specification, several works explore methods to generate hand-drawn in-betweens [BW75] automatically. Recently, considering that human motion dynamics can be learned from data, Zhang et al [ZvdP18] learn inbetween patterns with an auto-regressive two-layer recurrent network to automatically autocomplete a hopping lamp motion between two keyframes. Their system offers the flexibility of keyframing and an intelligent autocompletion learned on data, but does not address the case of long completion segments.…”

Section: Keyframing-based Editingmentioning

confidence: 99%

Intuitive Facial Animation Editing Based On A Generative RNN Framework

Berson

Soladié

Stoiber³

2020

Computer Graphics Forum

View full text Add to dashboard Cite

For the last decades, the concern of producing convincing facial animation has garnered great interest, that has only been accelerating with the recent explosion of 3D content in both entertainment and professional activities. The use of motion capture and retargeting has arguably become the dominant solution to address this demand. Yet, despite high level of quality and automation performance‐based animation pipelines still require manual cleaning and editing to refine raw results, which is a time‐ and skill‐demanding process. In this paper, we look to leverage machine learning to make facial animation editing faster and more accessible to non‐experts. Inspired by recent image inpainting methods, we design a generative recurrent neural network that generates realistic motion into designated segments of an existing facial animation, optionally following user‐provided guiding constraints. Our system handles different supervised or unsupervised editing scenarios such as motion filling during occlusions, expression corrections, semantic content modifications, and noise filtering. We demonstrate the usability of our system on several animation editing use cases.

show abstract

Data-driven autocompletion for keyframe animation

Cited by 26 publications

References 32 publications

Robust motion in-betweening

Robust motion in-betweening

Examining Autocompletion as a Basic Concept for Interaction with Generative AI

Intuitive Facial Animation Editing Based On A Generative RNN Framework

Contact Info

Product

Resources

About