Self-adapting Confidence Estimation for Stereo

Abstract

Project page: https://fedstereo.github.io/ ~80 FPS 13.86% D1-all 9.16% D1-all 8.36% D1-all ~10 FPS ~80 FPS (a) (b) (c) (d) Figure 1. Federated adaptation in challenging environments. When facing a domain very different from those observed during training -e.g., nighttime images (a) -stereo models [55] suffer drops in accuracy (b). By enabling online adaptation [41] (c) the network can improve its predictions, at the expense of decimating the framerate. In our federated framework, the model can demand the adaptation process to the cloud, to enjoy its benefits while maintaining the original processing speed (d).

Modeling Stereo-Confidence out of the End-to-End Stereo-Matching Network via Disparity Plane Sweep

“…Thus, we fix the shifting step size K at the minimum pixel unit of 1. Comparison to self-adaptation method Self-adapting confidence (Poggi et al 2020) is close to Ours. While the selfadapting confidence is rooted in a conventional approach, it is built upon the learning-based framework without using internal information from stereo-matching methods.…”

“…Table 6 presents a comparison between Ours and self-adapt confidence, which denoted OTB and OTB-online. The experiments utilize GANet on 6905 samples of the DrivingStereo dataset (Yang et al 2019) provided by authors, as following the procedures laid out in the self-adapt confidence method (Poggi et al 2020). Both OTB and OTB-online are trained in a self-supervised manner and OTB-online is OTB with online adaptation.…”

“…Furthermore, especially in safety-critical systems closely tied to the stereo-matching networks, the architecture is kept confidential, and access to its internal information is restricted to protect the network from potential external threats (Wang et al 2021;Wong, Mundhra, and Soatto 2021;Cheng, Healey, and Wu 2021). As a result, this prompts the necessity of the exploration of estimating confidence without accessing the cost volumes (or internal features) and from outside the stereo-matching network (Poggi et al 2020).…”

Uncertainty estimation for stereo matching based on evidential deep learning

“…Therefore, we propose a heuristic smoothness regularization on the output hyper-parameters β, α, v for pixels in smooth regions. Following the method proposed in [31], we rely on the disparity agreement between neighboring pixels to discard pixels close to depth boundaries, given by:…”