A deep neural network approach to fusing vision and heteroscedastic motion estimates for low-SWaP robotic applications

Shamwell, E. Jared; Nothwang, William D.; Perlis, Donald

doi:10.1109/mfi.2017.8170407

Cited by 3 publications

(5 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Increased mean predictive performance of

of pixel-position RMSE compared to our previous approach [ 4 ] with identical runtime after pruning (158 Hz); and…”

Section: Introductionmentioning

confidence: 92%

“…With the original DEs fast runtime, we saw the possibility of generating many different hypothetical outputs for each input image and then selecting the most accurate at execution time. By learning how to produce n image reconstruction predictions, we have extended DE into the Multi-Hypothesis DeepEfference (MHDE) [ 4 ] architecture to better handle real-world noise sources.…”

Section: Introductionmentioning

confidence: 99%

“…The main contributions of this paper and the models here described are: Increased mean predictive performance of

of pixel-position root mean squared error (RMSE) compared to DeepMatching (DM) [ 5 ] with a runtime decrease of

(

with pruning); Increased mean predictive performance of

of pixel-position RMSE compared to our previous approach [ 4 ] with identical runtime after pruning (158 Hz); and New results and analysis from networks trained with up to 20 hypothesis sub-pathways. …”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An Embodied Multi-Sensor Fusion Approach to Visual Motion Estimation Using Unsupervised Deep Networks

Shamwell

Nothwang

Perlis

2018

Sensors

Self Cite

View full text Add to dashboard Cite

Aimed at improving size, weight, and power (SWaP)-constrained robotic vision-aided state estimation, we describe our unsupervised, deep convolutional-deconvolutional sensor fusion network, Multi-Hypothesis DeepEfference (MHDE). MHDE learns to intelligently combine noisy heterogeneous sensor data to predict several probable hypotheses for the dense, pixel-level correspondence between a source image and an unseen target image. We show how our multi-hypothesis formulation provides increased robustness against dynamic, heteroscedastic sensor and motion noise by computing hypothesis image mappings and predictions at 76–357 Hz depending on the number of hypotheses being generated. MHDE fuses noisy, heterogeneous sensory inputs using two parallel, inter-connected architectural pathways and n (1–20 in this work) multi-hypothesis generating sub-pathways to produce n global correspondence estimates between a source and a target image. We evaluated MHDE on the KITTI Odometry dataset and benchmarked it against the vision-only DeepMatching and Deformable Spatial Pyramids algorithms and were able to demonstrate a significant runtime decrease and a performance increase compared to the next-best performing method.

show abstract

“…Increased mean predictive performance of

of pixel-position RMSE compared to our previous approach [ 4 ] with identical runtime after pruning (158 Hz); and…”

Section: Introductionmentioning

confidence: 92%

Section: Introductionmentioning

confidence: 99%

“…The main contributions of this paper and the models here described are: Increased mean predictive performance of

of pixel-position root mean squared error (RMSE) compared to DeepMatching (DM) [ 5 ] with a runtime decrease of

(

with pruning); Increased mean predictive performance of

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Embodied Multi-Sensor Fusion Approach to Visual Motion Estimation Using Unsupervised Deep Networks

Shamwell

Nothwang

Perlis

2018

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…The final level of VIOLearner employs multi-hypothesis pathways similar to [20], [21] where several possible hypotheses for the reconstructions of a target image (and the associated transformations θm , m ∈ M which generated those reconstructions) are computed in parallel. The lowest error hypothesis reconstruction is chosen during each network run and the corresponding affine matrix θm * which generated the winning reconstruction is output as the final network estimate of camera pose change between images I j and I j+1 .…”

Section: Level N and Multi-hypothesis Pathwaysmentioning

confidence: 99%

“…Error for this last multi-hypothesis level is computed according to a winner-take-all (WTA) Euclidean loss rule (see [20] for more detail and justifications):…”

Section: Level N and Multi-hypothesis Pathwaysmentioning

confidence: 99%

Vision-Aided Absolute Trajectory Estimation Using an Unsupervised Deep Network with Online Error Correction

Shamwell

Leung

Nothwang

2018

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

We present an unsupervised deep neural network approach to the fusion of RGB-D imagery with inertial measurements for absolute trajectory estimation. Our network, dubbed the Visual-Inertial-Odometry Learner (VIOLearner), learns to perform visual-inertial odometry (VIO) without inertial measurement unit (IMU) intrinsic parameters (corresponding to gyroscope and accelerometer bias or white noise) or the extrinsic calibration between an IMU and camera. The network learns to integrate IMU measurements and generate hypothesis trajectories which are then corrected online according to the Jacobians of scaled image projection errors with respect to a spatial grid of pixel coordinates. We evaluate our network against state-of-the-art (SOA) visual-inertial odometry, visual odometry, and visual simultaneous localization and mapping (VSLAM) approaches on the KITTI Odometry dataset [1] and demonstrate competitive odometry performance.

show abstract

Analysis of heteroscedastic measurement data by the self-refining method of interval fusion with preference aggregation – IF&PA

Muravyov

Khudonogova

2021

Measurement

View full text Add to dashboard Cite

A deep neural network approach to fusing vision and heteroscedastic motion estimates for low-SWaP robotic applications

Cited by 3 publications

References 17 publications

An Embodied Multi-Sensor Fusion Approach to Visual Motion Estimation Using Unsupervised Deep Networks

An Embodied Multi-Sensor Fusion Approach to Visual Motion Estimation Using Unsupervised Deep Networks

Vision-Aided Absolute Trajectory Estimation Using an Unsupervised Deep Network with Online Error Correction

Analysis of heteroscedastic measurement data by the self-refining method of interval fusion with preference aggregation – IF&PA

Contact Info

Product

Resources

About