Graphonomy: Universal Human Parsing via Graph Transfer Learning

Abstract

Prior highly-tuned human parsing models tend to fit towards each dataset in a specific domain or with discrepant label granularity, and can hardly be adapted to other human parsing tasks without extensive re-training. In this paper, we aim to learn a single universal human parsing model that can tackle all kinds of human parsing needs by unifying label annotations from different domains or at various levels of granularity. This poses many fundamental learning challenges, e.g. discovering underlying semantic structures among different label granularity, performing proper transfer learning across different image domains, and identifying and utilizing label redundancies across related tasks.To address these challenges, we propose a new universal human parsing agent, named "Graphonomy", which incorporates hierarchical graph transfer learning upon the conventional parsing network to encode the underlying label semantic structures and propagate relevant semantic information. In particular, Graphonomy first learns and propagates compact high-level graph representation among the labels within one dataset via Intra-Graph Reasoning, and then transfers semantic information across multiple datasets via Inter-Graph Transfer. Various graph transfer dependencies (e.g., similarity, linguistic knowledge) between different datasets are analyzed and encoded to enhance graph transfer capability. By distilling universal semantic graph representation to each specific task, Graphonomy is able to predict all levels of parsing labels in one system without piling up the complexity. Experimental results show Graphonomy effectively achieves the state-of-the-art results on three human parsing benchmarks as well as advantageous universal human parsing performance.

You can Try without Visiting: A Comprehensive Survey on Virtually Try-on Outfits

“…In 2019, Ruan et al [21] presented a Context Embedding with Edge Perceiving (CE2P) framework to deal with both single and multiple human parsing. Most recently, hierarchical graph is considered for human parsing tasks [23,24] to ensure perfect parsing performance. Wang et al [23] considered the human body as a hierarchy of multi-level semantic parts to capture the human parsing information.…”

“…Wang et al [23] considered the human body as a hierarchy of multi-level semantic parts to capture the human parsing information. Basing on transfer learning technique, Gong et al [24] designed a human parsing model untitled Graphonomy by including hierarchical graph into conventional parsing network.…”

Readily Design and Try-On Garments by Manipulating Segmentation Images

“…Fashion parsing uses segmentation to classify apparel such as tops, pants, and dresses to obtain the desired information. Graphonomy [8] showed good fashion parsing performance by introducing a hierarchical graph. For example, the head area can be divided into hat, face, and hair.…”

Grapy-ML: Graph Pyramid Mutual Learning for Cross-Dataset Human Parsing

“…To deal with these issues, prior methods have achieved significant progress based on the success of the deep neural network. Since human body is highly structural, many methods have been proposed to model the context correlations efficiently between body parts using convolutional neural network (CNN) (Liang et al 2015b;Li et al 2017;Zhu et al 2018;Ruan et al 2019), recurrent network with long short-term memory units (LSTM) (Liang et al 2016b;2016b;2017), and graph convolutional neural network (GCN) (Gong et al 2019). For instance, Ruan et al propose a context embedding with edge perceiving (CE2P) network for single human parsing after identifying three key factors affecting the parsing performance, including highresolution maintenance, global context embedding, and edge perceiving (Ruan et al 2019).…”

“…Another direction is to explore multiple datasets or heterogeneous annotations by utilizing multi-task learning, transfer learning, or mutual learning (Xiao et al 2018;Nie, Feng, and Yan 2018;Gong et al 2019). For instance, Xiao et al propose a multi-task learning framework to learn visual concepts from heterogeneous image annotations for unified perceptual scene parsing (Xiao et al 2018).…”