Body Part Segmentation of Anime Characters

Abstract

Semantic segmentation is an important approach to present the perceptual semantic understanding of an image, which is of significant usage in various applications. Especially, body part segmentation is designed for segmenting body parts of human characters to assist different editing tasks, such as style editing, pose transfer, and animation production. Since segmentation requires pixel‐level precision in semantic labeling, classic heuristics‐based methods generally have unstable performance. With the deployment of deep learning, a great step has been taken in segmenting body parts of human characters in natural photographs. However, the existing models are purely trained on natural photographs and generally obtain incorrect segmentation results when applied on anime character images, due to the large visual gap between training data and testing data. In this article, we present a novel approach to achieving body part segmentation of cartoon characters via a pose‐based graph‐cut formulation. We demonstrate the use of the acquired body part segmentation map in various image editing tasks, including conditional generation, style manipulation, pose transfer, and video‐to‐anime.