Ego-Motion Estimation Using Recurrent Convolutional Neural Networks through Optical Flow Learning

Abstract

Visual odometry (VO) refers to incremental estimation of the motion state of an agent (e.g., vehicle and robot) by using image information, and is a key component of modern localization and navigation systems. Addressing the monocular VO problem, this paper presents a novel end-to-end network for estimation of camera ego-motion. The network learns the latent subspace of optical flow (OF) and models sequential dynamics so that the motion estimation is constrained by the relations between sequential images. We compute the OF field of consecutive images and extract the latent OF representation in a self-encoding manner. A Recurrent Neural Network is then followed to examine the OF changes, i.e., to conduct sequential learning. The extracted sequential OF subspace is used to compute the regression of the 6-dimensional pose vector. We derive three models with different network structures and different training schemes: LS-CNN-VO, LS-AE-VO, and LS-RCNN-VO. Particularly, we separately train the encoder in an unsupervised manner. By this means, we avoid non-convergence during the training of the whole network and allow more generalized and effective feature representation. Substantial experiments have been conducted on KITTI and Malaga datasets, and the results demonstrate that our LS-RCNN-VO outperforms the existing learning-based VO approaches.

ARTFLOW: A Fast, Biologically Inspired Neural Network that Learns Optic Flow Templates for Self-Motion Estimation

“…Simulations reveal that ARTFLOW is capable of learning stable templates with only one pass through optic flow samples corresponding to simulated self-motion through a number of virtual environments (one-shot learning). This contrasts with deep learning networks that require larger amounts of data and many training epochs [46][47][48] and that suffer from the catastrophic forgetting problem [33]. While I used separate training and prediction phases in the simulations reported here, this distinction is not necessary, and ARTFLOW may continue to learn during on-going operation, unlike deep learning approaches.…”

Accurate Realtime Motion Estimation Using Optical Flow on an Embedded System

“…This problem consists of estimating from the temporal sequences of 2D images the apparent movement of the objects constituting a three-dimensional scene. As a result, the optical flow tries to find the vector field that connects two successive images in a video sequence [10,11]. Hence, we can deduce the global motion of the object by analyzing the direction of these vectors.…”

Fault-Tolerance by Resilient State Transition for Collaborative Cyber-Physical Systems

“…Accurate estimation of the ego-motion of a platooning system is critical for selfdiagnostic and decision making. The ego-motion in autonomous vehicles can be determined using a number of sensors [26,27]. Among them, cameras, IMUs and wheel encoders are famous because of their ubiquity and low cost while providing sufficient information for ego-motion estimation.…”