Depth Map Decomposition for Monocular Depth Estimation

Versatile Depth Estimator Based on Common Relative Depth Estimation and Camera-Specific Relative-to-Metric Depth Conversion

“…In relative depth estimation, the scale-invariant loss [13] and its variants [33,34,40,48] have been used to cope with the scale ambiguity of depth labels. Recently, listwise ranking [35], instead of pairwise ranking, and depth normalization [23] have been considered.…”

“…For instance, relative depths were fitted to metric depths directly using least-squares in [35,40], and a relative depth estimator was fine-tuned to a metric depth estimator in [39]. Also, relative and metric depths were jointly learned through depth map decomposition in [23].…”

“…However, [35,40] require a large depth-labeled dataset for fitting. Also, [23,39] fine-tune a pre-trained network to estimate metric depth maps for a single camera only. Hence, they are not versatile, i.e., not applicable to images captured by different cameras.…”

“…To solve this problem, we decompose a depth map into camera-invariant and camera-specific components: relative depth information is camera-invariant, whereas depth scale information is camera-specific. We adopt the normalized depth map as the camerainvariant component, as in [23]. Specifically, given a depth map D, the normalized depth map N is obtained by…”

“…Since the depth order is camera-invariant, relative depth estimators can be trained using heterogeneous depth data from various sources [40]. However, they do not yield actual depths required in applications, so they should be fine-tuned for a specific camera to generate metric depths [23,39].…”