MagicVO: An End-to-End Hybrid CNN and Bi-LSTM Method for Monocular Visual Odometry

Jiao, Jichao; Mo, Yaokai; Liu, Weilun; Deng, Zhongliang

doi:10.1109/access.2019.2926350

Cited by 29 publications

(19 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Monocular visual odometry is the task of tracking the flow of information from a single camera and predicting the path undertaken by it. Traditional methods [ 12 , 22 , 23 ] are solely insufficient and have to rely on external parameters to account for the ambiguities involved, such as scale, whereas deep learning approaches [ 24 , 25 ] are self-sufficient and show improved results compared to older traditional methods.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Leveraging Deep Learning for Visual Odometry Using Optical Flow

Pandey

Pena

Byrne

et al. 2021

Sensors

View full text Add to dashboard Cite

In this paper, we study deep learning approaches for monocular visual odometry (VO). Deep learning solutions have shown to be effective in VO applications, replacing the need for highly engineered steps, such as feature extraction and outlier rejection in a traditional pipeline. We propose a new architecture combining ego-motion estimation and sequence-based learning using deep neural networks. We estimate camera motion from optical flow using Convolutional Neural Networks (CNNs) and model the motion dynamics using Recurrent Neural Networks (RNNs). The network outputs the relative 6-DOF camera poses for a sequence, and implicitly learns the absolute scale without the need for camera intrinsics. The entire trajectory is then integrated without any post-calibration. We evaluate the proposed method on the KITTI dataset and compare it with traditional and other deep learning approaches in the literature.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Jiao et al [ 25 ] proposed an iterative update to [ 24 ] using bi-directional LSTMs, and outperformed [ 24 ] on the KITTI dataset. Bi-directional LSTMs model relationships for both forward and backward sequences.…”

Section: Related Workmentioning

confidence: 99%

Leveraging Deep Learning for Visual Odometry Using Optical Flow

Pandey

Pena

Byrne

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…However, we use spherical cameras that have already been calibrated and design a distortion weight to address the distortion problem. Numerous self-(un)supervised visual odometry approaches have displayed the possibility of learning the network using unlabeled training data [13], [14], [18], [30]- [32]. These approaches attempted to provide supervision signals to train their networks using pseudo-labels generated from unlabeled training data.…”

Section: Related Workmentioning

confidence: 99%

“…This indicates the accuracy in test scenes cannot be guaranteed. In contrast, self-supervised learning approaches for deep learning tasks have recently attracted much attention [13]- [17]. These approaches do not require any explicitly labeled data for training, and thus present the possibility of optimizing the network using unlabeled data.…”

Section: Introductionmentioning

confidence: 99%

“…These approaches do not require any explicitly labeled data for training, and thus present the possibility of optimizing the network using unlabeled data. They have been adopted in various regression tasks, such as depth estimation [18], semantic segmentation [19], and motion estimation [13]. For these tasks, it is difficult to acquire precise ground-truth labels.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

SelfSphNet: Motion Estimation of a Spherical Camera via Self-Supervised Learning

et al. 2020

View full text Add to dashboard Cite

In this paper, we propose SelfSphNet, that is, a self-supervised learning network to estimate the motion of an arbitrarily moving spherical camera without the need for any labeled training data. Recently, numerous learning-based methods for camera motion estimation have been proposed. However, most of these methods require an enormous amount of labeled training data, which is difficult to acquire experimentally. To solve this problem, our SelfSphNet employs two loss functions to estimate the frame-to-frame camera motion, thus giving two supervision signals to the network with the usage of unlabeled training data. First, a 5 DoF epipolar angular loss, which is composed of a dense optical flow of spherical images, estimates the 5 DoF motion between two image frames. This loss function utilizes a unique property of the spherical optical flow, which allows the rotational and translational components to be decoupled by using a derotation operation. This operation is derived from the fact that spherical images can be rotated to any orientation without any loss of information, hence making it possible to ''decouple'' the dense optical flow between pairs of spherical images to a pure translational state. Next, a photometric reprojection loss estimates the full 6 DoF motion using a depth map generated from the decoupled optical flow. This minimization strategy enables our network to be optimized without using any labeled training data. To confirm the effectiveness of our proposed approach (SelfSphNet), several experiments to estimate the camera trajectory, as well as the camera motion, were conducted in comparison to a previous self-supervised learning approach, SfMLearner, and a fully supervised learning approach whose baseline network is the same as SelfSphNet. Moreover, transfer learning in a new scene was also conducted to verify that our proposed method can optimize the network with newly collected unlabeled data. INDEX TERMS Motion estimation, computer vision, image processing, deep learning, convolutional neural networks.

show abstract