A Neuromorphic Vision-Based Measurement for Robust Relative Localization in Future Space Exploration Missions

Abstract

Space exploration has witnessed revolutionary changes upon landing of the Perseverance Rover on the Martian surface and demonstrating the first flight beyond Earth by the Mars helicopter, Ingenuity. During their mission on Mars, Perseverance Rover and Ingenuity collaboratively explore the Martian surface, where Ingenuity scouts terrain information for rover's safe traversability. Hence, determining the relative poses between both the platforms is of paramount importance for the success of this mission. Driven by this necessity, this work proposes a robust relative localization system based on a fusion of neuromorphic vision-based measurements (NVBMs) and inertial measurements. The emergence of neuromorphic vision triggered a paradigm shift in the computer vision community, due to its unique working principle delineated with asynchronous events triggered by variations of light intensities occurring in the scene. This implies that observations cannot be acquired in static scenes due to illumination invariance. To circumvent this limitation, high frequency active landmarks are inserted in the scene to guarantee consistent event firing. These landmarks are adopted as salient features to facilitate relative localization. A novel event-based landmark identification algorithm using Gaussian Mixture Models (GMM) is developed for matching the landmarks correspondences formulating our NVBMs. The NVBMs are fused with inertial measurements in proposed state estimators, landmark tracking Kalman filter (LTKF) and translation decoupled Kalman filter (TDKF) for landmark tracking and relative localization, respectively. The proposed system was tested in a variety of experiments and has outperformed state-of-the-art approaches in accuracy and range.

M-Calib: A Monocular 3D Object Localization using 2D Estimates for Industrial Robot Vision System

“…To evaluate the proposed method for object location estimation, we conducted experiments using a vision system with a camera positioned above, parallel to the floor, and at a distance of 40cm. Real-world coordinates were defined using a checkerboard pattern, where each square measured 3 × 3cm 2 . Further details about the experimental setup, including hardware specifications, are provided in Table 1.…”

“…The rapid advancement of imaging sensors over recent decades has paved the way for a plethora of intelligent perception algorithms [1]. Leveraging these capabilities, vision technology has made significant strides in diverse fields including space robotics [2], robot manufacturing, rapid object detection, and tracking.…”

“…The rapid advancement of imaging sensors over recent decades has paved the way for a plethora of intelligent perception algorithms [1]. Leveraging these capabilities, vision technology has made significant strides in diverse fields including space robotics [2], robot manufacturing, rapid object detection, and tracking. Industrial Robot Vision (IRV), which integrates computer vision into industrial manufacturing processes, presents a nuanced approach compared to traditional computer vision methodologies.…”

TactiGraph: An Asynchronous Graph Neural Network for Contact Angle Prediction Using Neuromorphic Vision-Based Tactile Sensing

“…The pixel-wise intensity changes, called events or spikes, are recorded at a temporal resolution on the order of microseconds. Event-based cameras have been applied in autonomous drone racing [38], space imaging [39], space exploration [40], automated drilling [41], and visual servoing [42,43]. Neuromorphic cameras' fast update rate, along with their high dynamic range (140 dB compared to conventional cameras with 60 dB [44]) and low power consumption, make them apt for robotics tasks [45].…”

TactiGraph: An Asynchronous Graph Neural Network for Contact Angle Prediction Using Neuromorphic Vision-Based Tactile Sensing

“…The pixel-wise intensity changes, called events or spikes, are recorded at a temporal resolution on the order of microseconds. Event-based cameras have been applied in autonomous drone racing [39], space imaging [40], space exploration [41], automated drilling [42], and visual servoing [43,44]. Neuromorphic cameras' fast update rate, along with their high dynamic range (140 dB compared to conventional cameras with 60 dB [45]) and low power consumption, make them apt for robotics tasks [46].…”