FADE: Feature Aggregation for Depth Estimation With Multi-View Stereo

Yang, Hsiao-Chien; Chen, Po-Heng; Chen, Kuan-Wen; Lee, Chen-Yi; Chen, Yongsheng

doi:10.1109/tip.2020.2991883

Cited by 9 publications

(2 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Monocular Depth Estimation. From approaches based on probabilistic graphical models (e.g., MRFs) with handcrafted features [41,42] to the deep learning-based [43,44,45,46,47,48,49,50,51,52,53,54], the improvement of performance for monocular depth estimation has been pushed forward. Eigen et al [46] were the first to develop deep models for depth estimation.…”

Section: Related Workmentioning

confidence: 99%

Adaptive Context-Aware Multi-Modal Network for Depth Completion

Zhao,

Gong,

et al. 2020

Preprint

View full text Add to dashboard Cite

Depth completion aims to recover a dense depth map from the sparse depth data and the corresponding single RGB image. The observed pixels provide the significant guidance for the recovery of the unobserved pixels' depth. However, due to the sparsity of the depth data, the standard convolution operation, exploited by most of existing methods, is not effective to model the observed contexts with depth values. To address this issue, we propose to adopt the graph propagation to capture the observed spatial contexts. Specifically, we first construct multiple graphs at different scales from observed pixels. Since the graph structure varies from sample to sample, we then apply the attention mechanism on the propagation, which encourages the network to model the contextual information adaptively. Furthermore, considering the mutli-modality of input data, we exploit the graph propagation on the two modalities respectively to extract multi-modal representations. Finally, we introduce the symmetric gated fusion strategy to exploit the extracted multi-modal features effectively. The proposed strategy preserves the original information for one modality and also absorbs complementary information from the other through learning the adaptive gating weights. Our model, named Adaptive Context-Aware Multi-Modal Network (ACMNet), achieves the state-of-theart performance on two benchmarks, i.e., KITTI and NYU-v2, and at the same time has fewer parameters than latest models. Our code is available at: https://github. com/sshan-zhao/ACMNet.

show abstract

Section: Related Workmentioning

confidence: 99%

Adaptive Context-Aware Multi-Modal Network for Depth Completion

Zhao,

Gong,

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…For example, the collected images [6]- [8] can be represented by different visual descriptors (views) [9]- [13], like CTM [14]- [16], GIST [17]- [19], SIFT [20], etc. Integrating the information from different views can help us analyze the data in a comprehensive manner [21]- [23], which motivates the development of multiview learning methods [24]- [27]. In the filed of multi-view learning, multi-view clustering is used in more and more image processing applications due to the ends to eliminate the high cost of time and money on labeling images.…”

Section: Introductionmentioning

confidence: 99%

ANIMC: A Soft Framework for Auto-weighted Noisy and Incomplete Multi-view Clustering

Fang,

Hu,

Zhou

et al. 2020

Preprint

View full text Add to dashboard Cite

Multi-view clustering has wide applications in many image processing scenarios. In these scenarios, original image data often contain missing instances and noises, which is ignored by most multi-view clustering methods. However, missing instances may make these methods difficult to use directly and noises will lead to unreliable clustering results. In this paper, we propose a novel Auto-weighted Noisy and Incomplete Multiview Clustering framework (ANIMC) via a soft auto-weighted strategy and a doubly soft regular regression model. Firstly, by designing adaptive semi-regularized nonnegative matrix factorization (adaptive semi-RNMF), the soft auto-weighted strategy assigns a proper weight to each view and adds a soft boundary to balance the influence of noises and incompleteness. Secondly, by proposing θ-norm, the doubly soft regularized regression model adjusts the sparsity of our model by choosing different θ. Compared with existing methods, ANIMC has three unique advantages: 1) it is a soft algorithm to adjust our framework in different scenarios, thereby improving its generalization ability; 2) it automatically learns a proper weight for each view, thereby reducing the influence of noises; 3) it performs doubly soft regularized regression that aligns the same instances in different views, thereby decreasing the impact of missing instances. Extensive experimental results demonstrate its superior advantages over other state-of-the-art methods.

show abstract