MultiDepth: Single-Image Depth Estimation via Multi-Task Regression and Classification

Abstract

We introduce MultiDepth, a novel training strategy and convolutional neural network (CNN) architecture that allows approaching single-image depth estimation (SIDE) as a multi-task problem. SIDE is an important part of road scene understanding. It, thus, plays a vital role in advanced driver assistance systems and autonomous vehicles. Best results for the SIDE task so far have been achieved using deep CNNs. However, optimization of regression problems, such as estimating depth, is still a challenging task. For the related tasks of image classification and semantic segmentation, numerous CNN-based methods with robust training behavior have been proposed. Hence, in order to overcome the notorious instability and slow convergence of depth value regression during training, MultiDepth makes use of depth interval classification as an auxiliary task. The auxiliary task can be disabled at test-time to predict continuous depth values using the main regression branch more efficiently. We applied MultiDepth to road scenes and present results on the KITTI depth prediction dataset. In experiments, we were able to show that end-to-end multi-task learning with both, regression and classification, is able to considerably improve training and yield more accurate results.

MoNet3D: Towards Accurate Monocular 3D Object Localization in Real Time

“…Considering that most of the research on image depth estimation now was pixel-level depth estimation, we compared the instance-level depth estimation we invented with them. According to the latest research on pixel-level depth estimation (Fu et al, 2018;Ren et al, 2019;Liebel & Körner, 2019), for example, the relative absolute error of the depth estimation of DORN on the KITTI dataset was 8.78% (Fu et al, 2018), and the relative absolute error of the depth estimation of MultiDepth on the same dataset was 13.82% (Liebel & Körner, 2019). Compared with pixel- a FPS means frames per second and the FPS here refers to the FPS running on the computer.…”

A generalized multi-task learning approach to stereo DSM filtering in urban areas

“…Similarly, Kendall et al [15] improved depth estimation results for road scenes by evaluating semantic and instance labels at the same time. A different approach on single-image depth estimation using multi-task learning has been pursued by Liebel et al [6] who posed this natural regression task as the classification of discrete depth ranges as an additional auxiliary task and jointly solved for both targets. Other problems that have recently been tackled by multi-task learning include facial landmark detection [34] and person attribute classification [22].…”

“…As a secondary source of information, buildings are classified based on their roof geometries with classes for flat and non-flat roofs. In the closely related field of depth estimation it can be observed that the optimization of regression tasks is generally harder than the optimization of corresponding classification tasks [12,6]. By restricting the highly non-convex optimization space through this constraint, this auxiliary objective doubles as a regularization measure [16].…”

Novel Hybrid Neural Network for Dense Depth Estimation using On-Board Monocular Images

“…In addition, they exploited a special module to remove the shadows existing in real-world images when applying their model to real data (34). Liebel et al proposed MultiDepth, a sort of training strategy, to solve the problems of notorious instability and slow convergence in depth training, by developing a auxiliary task of depth interval classification (35).…”