Event-Based Neuromorphic Vision for Autonomous Driving: A Paradigm Shift for Bio-Inspired Visual Sensing and Perception

Chen, Guang; Cao, Hu; Conradt, Jörg; Tang, Huajin; Röhrbein, Florian; Knoll, Alois

doi:10.1109/msp.2020.2985815

Cited by 210 publications

(112 citation statements)

References 47 publications

Supporting

Mentioning

111

Contrasting

Unclassified

Order By: Relevance

“…However, in the current research, it is found that deep learning is only not good at dealing with the mathematical problems in SLAM, but also has the problem of insufficient calibration data sets, resulting in the inability of the detection accuracy of moving objects for excellent performance. Therefore, it is impossible to completely replace the traditional SLAM target detection module with deep learning and neuromorphic vision sensor [39] at this stage.…”

Section: It Can Be Learned From Human's Common Sense Andmentioning

confidence: 99%

PSPNet-SLAM: A Semantic SLAM Detect Dynamic Object by Pyramid Scene Parsing Network

2020

View full text Add to dashboard Cite

Simultaneous Localization and Mapping (SLAM) plays an important role in the computer vision and robotic field. The traditional SLAM framework adopts a strong static world assumption for convenience of analysis. It is very essential to know how to deal with the dynamic environment in the entire industry with widespread attention. Faced with these challenges, researchers consider introducing semantic information to collaboratively solve dynamic objects in the scene. So, in this paper, we proposed a PSPNet-SLAM: Pyramid Scene Parsing Network SLAM, which integrated the Semantic thread of pyramid structure and geometric threads of reverse ant colony search strategy into ORB-SLAM2. In the proposed system, a pyramid-structured PSPNet was used for semantic thread to segment dynamic objects in combination with context information. In the geometric thread, we proposed a OCMulti-View Geometry thread. On the one hand, the optimal error compensation homography matrix was designed to improve the accuracy of dynamic point detection. On the other hand, we came up with a reverse ant colony collection strategy to enhance the real-time performance of the system and reduce its time consumption during the detection of dynamic objects. We have evaluated our SLAM in public data sheets and real-time world and compared it with ORB-SLAM2, DynaSLAM. Many improvements have been achieved in this system including location accuracy in high-dynamic scenarios, which also outperformed the other four state-of-the-art SLAM systems coping with the dynamic environments. The real-time performance has been delivered, compared with the geometric thread of the excellent DynaSALM system.

show abstract

Section: It Can Be Learned From Human's Common Sense Andmentioning

confidence: 99%

PSPNet-SLAM: A Semantic SLAM Detect Dynamic Object by Pyramid Scene Parsing Network

2020

View full text Add to dashboard Cite

show abstract

“…Dynamic control is one of the most crucial tasks for autonomous driving vehicle (Chen et al, 2020). H-infinity output feedback controller (Hu et al, 2016), sliding mode controller (Jiang and Wu, 2018), model predictive control (MPC) (Sun et al, 2019), etc.…”

Section: Introductionmentioning

confidence: 99%

Matrix Inequalities Based Robust Model Predictive Control for Vehicle Considering Model Uncertainties, External Disturbances, and Time-Varying Delay

2021

Self Cite

View full text Add to dashboard Cite

In this paper, we design a robust model predictive control (MPC) controller for vehicle subjected to bounded model uncertainties, norm-bounded external disturbances and bounded time-varying delay. A Lyapunov-Razumikhin function (LRF) is adopted to ensure that the vehicle system state enters in a robust positively invariant (RPI) set under the control law. A quadratic cost function is selected as the stage cost function, which yields the upper bound of the infinite horizon cost function. A Lyapunov-Krasovskii function (LKF) candidate related to time-varying delay is designed to obtain the upper bound of the infinite horizon cost function and minimize it at each step by using matrix inequalities technology. Then the robust MPC state feedback control law is obtained at each step. Simulation results show that the proposed vehicle dynamic controller can steer vehicle states into a very small region near the reference tracking signal even in the presence of external disturbances, model uncertainties and time-varying delay. The source code can be downloaded on https://github.com/wenjunliu999.

show abstract

“…The researchers of Chen et al ( 2018 ) applied neuromorphic vision sensor in intelligent transportation system. Some works have been reviewed in Chen et al ( 2020 ) and Gallego et al ( 2019 ). Considering the high requirement of calculation time and storage consumption, the traditional RGB-D camera cannot satisfy the real-time feature.…”

Section: Introductionmentioning

confidence: 99%

Event-Based Robotic Grasping Detection With Neuromorphic Vision Sensor and Event-Grasping Dataset

Cao

et al. 2020

Front. Neurorobot.

Self Cite

View full text Add to dashboard Cite

Robotic grasping plays an important role in the field of robotics. The current state-of-the-art robotic grasping detection systems are usually built on the conventional vision, such as the RGB-D camera. Compared to traditional frame-based computer vision, neuromorphic vision is a small and young community of research. Currently, there are limited event-based datasets due to the troublesome annotation of the asynchronous event stream. Annotating large scale vision datasets often takes lots of computation resources, especially when it comes to troublesome data for video-level annotation. In this work, we consider the problem of detecting robotic grasps in a moving camera view of a scene containing objects. To obtain more agile robotic perception, a neuromorphic vision sensor ( Dynamic and Active-pixel Vision Sensor, DAVIS ) attaching to the robot gripper is introduced to explore the potential usage in grasping detection. We construct a robotic grasping dataset named Event-Grasping dataset with 91 objects. A spatial-temporal mixed particle filter (SMP Filter) is proposed to track the LED-based grasp rectangles, which enables video-level annotation of a single grasp rectangle per object. As LEDs blink at high frequency, the Event-Grasping dataset is annotated at a high frequency of 1 kHz. Based on the Event-Grasping dataset, we develop a deep neural network for grasping detection that considers the angle learning problem as classification instead of regression. The method performs high detection accuracy on our Event-Grasping dataset with 93% precision at an object-wise level split. This work provides a large-scale and well-annotated dataset and promotes the neuromorphic vision applications in agile robot.

show abstract

Event-Based Neuromorphic Vision for Autonomous Driving: A Paradigm Shift for Bio-Inspired Visual Sensing and Perception

Cited by 210 publications

References 47 publications

PSPNet-SLAM: A Semantic SLAM Detect Dynamic Object by Pyramid Scene Parsing Network

PSPNet-SLAM: A Semantic SLAM Detect Dynamic Object by Pyramid Scene Parsing Network

Matrix Inequalities Based Robust Model Predictive Control for Vehicle Considering Model Uncertainties, External Disturbances, and Time-Varying Delay

Event-Based Robotic Grasping Detection With Neuromorphic Vision Sensor and Event-Grasping Dataset

Contact Info

Product

Resources

About