Absolute Camera Pose Regression Using an RGB-D Dual-Stream Network and Handcrafted Base Poses

Abstract

Absolute pose regression (APR) for camera localization is a single-shot approach that encodes the information of a 3D scene in an end-to-end neural network. The camera pose result of APR methods can be observed as the linear combination of the base poses. Previous APR methods’ base poses are learned from training data. However, the training data can limit the performance of the methods, which cannot be generalized to cover the entire scene. To solve this issue, we use handcrafted base poses instead of learning-based base poses, which prevents overfitting the camera poses of the training data. Moreover, we use a dual-stream network architecture to process color and depth images separately to get more accurate localization. On the 7 Scenes dataset, the proposed method is among the best in median rotation error, and in median translation error, it outperforms previous APR methods. On a more difficult dataset—Oxford RobotCar dataset, the proposed method achieves notable improvements in median translation and rotation errors compared to the state-of-the-art APR methods.

An end-to-end learning framework for visual camera relocalization using RGB and RGB-D images

“…Compared to our model, the relative pose regression models are EssNet [31]. PoseNet [25], MapNet [32], DSNet [16], and GL-Net [33] are the absolute pose regression models evaluated with our own. Absolute pose estimation techniques recover the absolute camera posture from the input image in a straightforward and efficient manner using CNN learning, and this class of approaches is mostly a version of PoseNet [25].…”

“…Researchers have discovered in recent years that neural networks can substitute random forests as correspondence learners and achieve excellent results for RGB-based camera relocalization [15]. As the backbone network, a dual-stream network (DSNet) structure is employed to handle picture data for precise pose estimation [16]. Deep feature aggregation module (DFAM) is used to combine contextual depth information into a general neural network to perform end-to-end posture estimation training while balancing the network's efficiency and light weight.…”

Urban Aquatic Scene Expansion for Semantic Segmentation in Cityscapes

“…Gasperini et al [39] presented Panoster, a segmentation method for urban LiDAR data, vital for 3D urban modeling. The researchers also [40,41] proposed a methodology based on a dual-encoder network to process RGB and depth data, enhancing 3D urban scene perception.…”