VMRF: View Matching Neural Radiance Fields

Zhang, Jiahui; Zhan, Fangneng; Wu, Rongliang; Yu, Yingchen; Zhang, Wenqing; Bai, Song; Zhang, Xiaoqin; Lu, Shijian

doi:10.48550/arxiv.2207.02621

Cited by 1 publication

(2 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…But all these works typically require precisely known camera poses. There are only a few works (Jeong et al, 2021;Zhang et al, 2022;Lin et al, 2021) trying to deal with uncalibrated cameras. Furthermore, all these methods require a long optimization process, and hence, are unsuitable for real-time applications like visual SLAM.…”

Section: Related Workmentioning

confidence: 99%

“…Multi-resolution Volume Encoding Directly representing the scene map with MLPs, which maps a 3D point to its occupancy and color, confronts a forgetting problem because the MLP is globally updated for any frame (Sucar et al, 2021). To address this, we equip the MLP with multi-resolution volumes {V l } L l=1 , which are updated locally on seen regions of each frame(Sara Fridovich-Keil and Alex Yu et al, 2022;Müller et al, 2022). The input point is encoded by the feature F sampled from the volumes {V l }, which could also explicitly store the geometric information.…”

Section: Implicit Map Representationmentioning

confidence: 99%

See 1 more Smart Citation

Dense RGB SLAM with Neural Implicit Maps

Li¹,

Gu²,

Yuan³

et al. 2023

Preprint

View full text Add to dashboard Cite

There is an emerging trend of using neural implicit functions for map representation in Simultaneous Localization and Mapping (SLAM). Some pioneer works have achieved encouraging results on RGB-D SLAM. In this paper, we present a dense RGB SLAM method with neural implicit map representation. To reach this challenging goal without depth input, we introduce a hierarchical feature volume to facilitate the implicit map decoder. This design effectively fuses shape cues across different scales to facilitate map reconstruction. Our method simultaneously solves the camera motion and the neural implicit map by matching the rendered and input video frames. To facilitate optimization, we further propose a photometric warping loss in the spirit of multi-view stereo to better constrain the camera pose and scene geometry. We evaluate our method on commonly used benchmarks and compare it with modern RGB and RGB-D SLAM systems. Our method achieves favorable results than previous methods and even surpasses some recent RGB-D SLAM methods. Our source code will be publicly available. Another important problem in visual SLAM is map representation. Sparse SLAM methods (Mur-Artal & Tardós, 2017; Engel et al., 2017) typically use point clouds for map representation, while dense methods (Newcombe et al., 2011b;a) usually adopt triangle meshes. As observed in many recent geometry processing works (Mescheder et al., 2019; Park et al., 2019; Chen & Zhang, 2019), neural implicit function offers a promising presentation for 3D data processing. The pioneer work, iMAP (Sucar et al., 2021), introduces an implicit map representation for dense visual SLAM. This map representation is more compact, continuous, and allowing for prediction of unobserved areas, which could potentially benefit applications like path planning (Shrestha et al., 2019) and object manipulation (Sucar et al., 2020). However, as observed in NICE-SLAM (Zhu et al., 2022), iMAP (Sucar et al., 2021 is limited to room-scale scenes due to the restricted representation power of MLPs. NICE-SLAM (Zhu et al., 2022) introduces a hierarchical feature volume to facilitate the map reconstruction and generalize the implicit map to larger scenes. However, both iMAP (Sucar et al., 2021) and NICE-SLAM (Zhu et al., 2022) are limited to RGB-D cameras. This paper presents a novel dense visual SLAM method with regular RGB cameras based on the implicit map representation. We also adopt a hierarchical feature volume like NICE-SLAM to deal * Equal contribution

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Implicit Map Representationmentioning

confidence: 99%