Learning Monocular Visual Odometry with Dense 3D Mapping from Dense 3D Flow

Abstract

This paper introduces a fully deep learning approach to monocular SLAM, which can perform simultaneous localization using a neural network for learning visual odometry (L-VO) and dense 3D mapping. Dense 2D flow and a depth image are generated from monocular images by sub-networks, which are then used by a 3D flow associated layer in the L-VO network to generate dense 3D flow. Given this 3D flow, the dualstream L-VO network can then predict the 6DOF relative pose and furthermore reconstruct the vehicle trajectory. In order to learn the correlation between motion directions, the Bivariate Gaussian modeling is employed in the loss function. The L-VO network achieves an overall performance of 2.68% for average translational error and 0.0143 • /m for average rotational error on the KITTI odometry benchmark. Moreover, the learned depth is leveraged to generate a dense 3D map. As a result, an entire visual SLAM system, that is, learning monocular odometry combined with dense 3D mapping, is achieved.

Ground-Plane-Based Absolute Scale Estimation for Monocular Visual Odometry

“…The proposed approach can not only automatically learn effective feature representation, but also implicitly model sequential dynamics and relation for VO with the help of deep RNN. In [47] and [48], two approaches have been proposed to robust estimate the VO by considering the optical flow caused by the camera motion. In [48], the camera motion has been estimated by using the constraints with depth and optical flow.…”

“…In [47] and [48], two approaches have been proposed to robust estimate the VO by considering the optical flow caused by the camera motion. In [48], the camera motion has been estimated by using the constraints with depth and optical flow. In [47], a novel network architecture for estimating monocular camera motion which is composed of two branches that jointly learn a latent space representation of the input optical flow field and the camera motion estimate.…”

SteReFo: Efficient Image Refocusing with Stereo Vision

“…With the same result, Zou et al [60] jointly train for optical flow, pose and depth estimation simultaneously while Jiao et al [23] mutually improve semantics and depth and GeoNet [53] jointly estimates depth, optical flow and camera pose from video. Fully unsupervised monocular depth and visual odometry can also be entangled [58] and 3D mapping applications [57] are realized by heavily relying on dense optical flow in 2D and 3D. Despite the superiority of these approaches, they suffer from larger computational burden or come at the cost of additional training data.…”

Dense RGB-D Semantic Mapping with Pixel-Voxel Neural Network

“…Real-time 3D semantic mapping is often desired in a number of robotics applications, such as localization [ 1 , 2 ], semantic navigation [ 3 , 4 ] and human-aware navigation [ 5 ]. The semantic information provided with a 3D dense map is more useful than the geometric information [ 6 ] itself in robot-human or robot-environment interaction.…”