Visual Person Understanding Through Multi-task and Multi-dataset Learning

Abstract

We address the problem of learning a single model for person re-identification, attribute classification, body part segmentation, and pose estimation. With predictions for these tasks we gain a more holistic understanding of persons, which is valuable for many applications. This is a classical multi-task learning problem. However, no dataset exists that these tasks could be jointly learned from. Hence several datasets need to be combined during training, which in other contexts has often led to reduced performance in the past. We extensively evaluate how the different task and datasets influence each other and how different degrees of parameter sharing between the tasks affect performance. Our final model matches or outperforms its single-task counterparts without creating significant computational overhead, rendering it highly interesting for resource-constrained scenarios such as mobile robotics.

Dual-Mode Learning for Multi-Dataset X-Ray Security Image Detection

“…Multi-dataset learning, which aims to learn a universal model from multiple datasets, has received increasing attention in various computer vision tasks, including depth estimation [24]- [26], stereo matching [27], [28], pedestrian detection [29], [30], semantic segmentation [31], [32], and object detection [33]- [37]. In this subsection, we mainly review multidataset object detection.…”

Multi-Dataset, Multitask Learning of Egocentric Vision Tasks

“…Additionally, the gradient updates for the main block may not be representative of all the tasks in each training step, affecting the statistics in the batch normalization layers [89]. To alleviate this issue, [90] proposed training on interleaved minibatches per dataset and the use of group normalization [91] to facilitate network convergence. The main difference in our approach is that we create mixed batches that enable the network to grasp information across datasets on every training iteration.…”

TarViS: A Unified Approach for Target-based Video Segmentation

“…Multi-task Models. Multi-task learning has a long history [11] with several architectures and training strategies [24,36,38,52,60,77]. Earlier approaches mostly consist of a shared backbone with fixed task-specific heads, whereas we design a more general architecture for video segmentation with task-specific targets to specify what to segment.…”