AuxNet: Auxiliary Tasks Enhanced Semantic Segmentation for Automated Driving

Abstract

Decision making in automated driving is highly specific to the environment and thus semantic segmentation plays a key role in recognizing the objects in the environment around the car. Pixel level classification once considered a challenging task which is now becoming mature to be productized in a car. However, semantic annotation is time consuming and quite expensive. Synthetic datasets with domain adaptation techniques have been used to alleviate the lack of large annotated datasets. In this work, we explore an alternate approach of leveraging the annotations of other tasks to improve semantic segmentation. Recently, multi-task learning became a popular paradigm in automated driving which demonstrates joint learning of multiple tasks improves overall performance of each tasks. Motivated by this, we use auxiliary tasks like depth estimation to improve the performance of semantic segmentation task. We propose adaptive task loss weighting techniques to address scale issues in multi-task loss functions which become more crucial in auxiliary tasks. We experimented on automotive datasets including SYNTHIA and KITTI and obtained 3% and 5% improvement in accuracy respectively.1. Construction of auxiliary task learning architecture for semantic segmentation.2. Novel loss function weighting strategy for one main task and one auxiliary task.

Mask-guided Image Classification with Siamese Networks

“…Likewise, Channupati et al improve the results of their semantic segmentation network by adding a branch that estimates the depth of the pixels as an auxiliary task (Chennupati et al, 2019). Since the depth was available in their used dataset, they propose to exploit it at training time and create a multi-task network.…”

MultiDepth: Single-Image Depth Estimation via Multi-Task Regression and Classification

“…In order to achieve this goal, we propose MultiDepth, a multi-task training procedure for jointly learning two representations of a depth map, exemplarily shown in Figure 1. As network architectures for SIDE often use encoder-decoder structures, a second branch for the classification of depth intervals can easily be added as an auxiliary decoder [33,8]. The auxiliary decoder is only active during training and can alternatively be disabled during test-time to make inference more time and memory efficient, or produce an additional redundant depth map that can be used to enhance or verify the regression result.…”

“…A different approach to employing multi-task learning is the utilization of auxiliary tasks [33,8,46] that merely serve as additional supervision to the network during training and are discarded during test-time. This approach can be seen as an extension to comprehensive regularization terms in loss functions as used by Li et al [32].…”

“…This approach can be seen as an extension to comprehensive regularization terms in loss functions as used by Li et al [32]. It could be shown that by adding auxiliary tasks to a network the performance of the main task increases [33,8].…”

“…Kendall et al [23] propose uncertainty-based weighting of individual tasks, which we build upon, but do not make use of auxiliary tasks. Auxiliary tasks have been utilized before [33,8,46], however not with posing the same task in two different ways. In contrast to Gurram et al [19], who use depth interval classification as pre-training, we train for regression and classification in an end-to-end manner.…”