Lvio-Fusion: A Self-adaptive Multi-sensor Fusion SLAM Framework Using Actor-critic Method

Abstract

State estimation with sensors is essential for mobile robots. Due to sensors have different performance in different environments, how to fuse measurements of various sensors is a problem. In this paper, we propose a tightly-coupled multi-sensor fusion framework, Lvio-Fusion, which fuses stereo camera, Lidar, IMU, and GPS based on the graph optimization. Especially for urban traffic scenes, we introduce a segmented global pose graph optimization with GPS and loop-closure, which can eliminate accumulated drifts. Additionally, we creatively use a actor-critic method in reinforcement learning to adaptively adjust sensors' weight. After training, actor-critic agent can provide the system with better and dynamic sensors' weight. We evaluate the performance of our system on public datasets and compare it with other state-of-the-art methods, showing that the proposed method achieves high estimation accuracy and robustness to various environments. And our implementations are open source and highly scalable.

Continuous-Time Fixed-Lag Smoothing for LiDAR-Inertial-Camera SLAM

“…[16] 2018-RAL I3 C1, C4 L5 F4 LIC-Fusion [4] 2019-IROS I1 C1 L1 MSCKF LIC-Fusion2.0 [10] 2020-IROS I1 C1 L1, L2 MSCKF LVI-SAM [6] 2021-ICRA I2 C1, C3 L1 F3 VILENS [29] 2021-RAL I2 C1, C3 L2 F4 VILENS [30] 2021-Arxiv I2 C1, C3 L2, L3 F4 R2live [31] 2021-RAL I1, I2 C1 L1 ESIKF, F4 R3live [7] 2021-Arxiv I1 C2, C3 L1 ESIKF Super Odom. [32] 2021-IROS I2 C1, C3 L4 F3 Lvio-Fusion [33] 2021-IROS I2 C1 L1 F4…”

“…A. Multi-sensor Fusion for SLAM 1) LiDAR-Inertial-Camera Fusion for SLAM: State estimation in LIC systems has been achieved either by graph-based optimization [6,16,25,26,29,30,32,33] or filter-based methods [4,7,10], while work [31] also combines both the graph optimization and filter for localization and mapping. We summarize typical LIC systems in Tab.…”

“…Some other works [6,32] get pose estimation via IMU preintegration, visual-inertial odometry and LiDARinertial odometry separately, then fuse the three types of pose estimation in a pose graph, which can be solved by factorgraph optimization. In contrast to loosely-coupled methods, which somehow fuse intermediate pose estimation from sensor measurements, tightly coupled approaches [16,29,30,33] directly integrate sensor data, estimating system state with IMU data (preintegration/raw measurements), image features and LiDAR features (illustrated in F4 of Fig. 1).…”

An automatic robot for mine inspection and rescue based on multi-sensor fusion

“…In summary, multi-sensor fusion systems have stronger environmental adaptability and higher reliability. Compared to a single sensor system, each component can complement its own advantages and disadvantages, with higher data reliability, stronger redundancy, and stronger ability to resist external interference [10][11][12].…”

An Enhanced Multi-Sensor Simultaneous Localization and Mapping (SLAM) Framework with Coarse-to-Fine Loop Closure Detection Based on a Tightly Coupled Error State Iterative Kalman Filter

“…GR-Fusion [23], built upon a foundational factor graph optimization system, tightly couples local constraints with GNSS constraints, allowing real-time detection of sensor degradation and suitability for various scenarios. LVIO-Fusion [24] employs a similar architecture, introducing reinforcement learning to adaptively adjust sensor weights in different scenes. Incorporating deep learning into tightly coupled VIO systems for dynamic object recognition and dynamic impact elimination also enhances the SLAM system's capabilities in dynamic scenes [25,26].…”