Constrained Filtering-based Fusion of Images, Events, and Inertial Measurements for Pose Estimation

Jung, Jae Hyung; Park, Chan Gook

doi:10.1109/icra40945.2020.9197248

Cited by 4 publications

(3 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our architecture could also benefit from a more optimized implementation on specialized hardware (FPGA for instance, eliminating the need for an energy-intensive GPU), which could also allow for even higher frame rates by lowering the minimum accumulation times. Applying our EBOF to complex automotive-related applications, such as proposed in [52], could also be addressed by future work, for instance for improving the detection of obstacles appearing in the close neighbourhood of the vehicle.…”

Section: Discussionmentioning

confidence: 99%

Real-Time Optical Flow for Vehicular Perception With Low- and High-Resolution Event Cameras

Brebion

Moreau

Davoine

2022

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

Event cameras capture changes of illumination in the observed scene rather than accumulating light to create images. Thus, they allow for applications under high-speed motion and complex lighting conditions, where traditional framebased sensors show their limits with blur and over-or underexposed pixels. Thanks to these unique properties, they represent nowadays an highly attractive sensor for ITS-related applications. Event-based optical flow (EBOF) has been studied following the rise in popularity of these neuromorphic cameras. The recent arrival of high-definition neuromorphic sensors, however, challenges the existing approaches, because of the increased resolution of the events pixel array and a much higher throughput. As an answer to these points, we propose an optimized framework for computing optical flow in real-time with both low-and high-resolution event cameras. We formulate a novel dense representation for the sparse events flow, in the form of the "inverse exponential distance surface". It serves as an interim frame, designed for the use of proven, state-of-the-art frame-based optical flow computation methods. We evaluate our approach on both low-and high-resolution driving sequences, and show that it often achieves better results than the current state of the art, while also reaching higher frame rates, 250Hz at 346×260 pixels and 77Hz at 1280×720 pixels.Index Terms-Machine vision, neuromorphic cameras, optical flow, real-time applications.• a specific pipeline-based architecture, for computing realtime optical flow using the events from low-or highresolution neuromorphic sensors; • the formulation of a novel dense "inverse exponential distance surface", that acts as the frame-based representation computed from the events, able to feed any image-based optical flow method; • a coherent choice of algorithms and methods together for all the steps up to the fast frame-based state-of-theart optical flow (with temporal smoothing to fit well with potentially noisy input events); • we finally build and share a complementary highdefinition event-based dataset of indoor sequences with high-speed movements, used as part of our evaluation. Videos accompanying this article, showing results for both low-and high-resolution data, are available at https://youtube.com/playlist?list= PLLL0eWAd6OXBRXli-tB1NREdhBElAxisD.

show abstract

Section: Discussionmentioning

confidence: 99%

Real-Time Optical Flow for Vehicular Perception With Low- and High-Resolution Event Cameras

Brebion

Moreau

Davoine

2022

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

show abstract

“…Much work has been done on observability analysis with continuous models, [16], [8], [3], [6], [20], [4], and [9]. Mirzaei et al .…”

Section: Related Workmentioning

confidence: 99%

“…Jung et al . [6] analyzed observability of an EKF-based estimator whose state vector includes the IMU intrinsic parameters. Villaverde [20] extended the continuous model analysis approach to systems where the control directly feeds to the output (i.e., direct feed-through).…”

Section: Related Workmentioning

confidence: 99%

NOCT: Nonlinear Observability with Constraints and Time Offset

Huai¹,

Lin²,

Zhang³

2022

Preprint

View full text Add to dashboard Cite

Nonlinear systems of affine control inputs overarch many sensor fusion instances. Analyzing whether a state variable in such a nonlinear system can be estimated (i.e., observability) informs better estimator design. Among the research on local observability of nonlinear systems, approaches based on differential geometry have attracted much attention for the solid theoretic foundation and suitability to automated deduction. Such approaches usually work with a system model of unconstrained control inputs and assume that the control inputs and observation outputs are timestamped by the same clock. To our knowledge, it has not been shown how to conduct the observability analysis with additional constraints enforced on the system's observations or control inputs. To this end, we propose procedures to convert a system model of affine control inputs with linear constraints into a constraint-free standard model which is apt to be analyzed by the classic observability analysis procedure. Then, the whole analysis procedure is illustrated by applying to the well-studied visual inertial odometry (VIO) system which estimates the camera-IMU relative pose and time offset. The findings about unobservable variables under degenerate motion concur with those obtained with linearized VIO systems in other studies, whereas the findings about observability of time offset extend those in previous studies. These findings are further validated by simulation.

show abstract

FE-Fusion-VPR: Attention-Based Multi-Scale Network Architecture for Visual Place Recognition by Fusing Frames and Events

Hou

Kong

Jiang

et al. 2023

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Constrained Filtering-based Fusion of Images, Events, and Inertial Measurements for Pose Estimation

Cited by 4 publications

References 23 publications

Real-Time Optical Flow for Vehicular Perception With Low- and High-Resolution Event Cameras

Real-Time Optical Flow for Vehicular Perception With Low- and High-Resolution Event Cameras

NOCT: Nonlinear Observability with Constraints and Time Offset

FE-Fusion-VPR: Attention-Based Multi-Scale Network Architecture for Visual Place Recognition by Fusing Frames and Events

Contact Info

Product

Resources

About