TextToon: Real-Time Text Toonify Head Avatar from Single Video

Abstract

User Input: Turn him into American Comic style. User Input: Turn him look like Pixar, as in Toy Story. Real Time Actor Real Time Driven Pre-trained Appearance | 2D Cartoon | Joker in DC | Caricature Comic | Hulk in Marvel Fig. 1. Our method takes a short monocular video as input (top) and animates the toonified appearance with synchronized expressions and movements using human-friendly text descriptions, e.g., "Turn him into an American Comic style" (middle). Moreover, our system achieves real-time animation (bottom), operating at 25 FPS (generation inference is about 48 FPS) on an NVIDIA RTX 4090 machine and 15 FPS on an Apple MacBook (M1 chip). All toonified faces (middle and bottom) are generated from the same pre-trained appearance model. Top block: Natural face©Tee Noir (CC BY). Middle and bottom: Natural face©Trevor Noah (CC BY).We propose TextToon, a method to generate a drivable toonified avatar. Given a short monocular video sequence and a written instruction about Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).