A Lightweight and Drift-Free Fusion Strategy for Drone Autonomous and Safe Navigation

Abstract

Self-localization and state estimation are crucial capabilities for agile drone autonomous navigation. This article presents a lightweight and drift-free vision-IMU-GNSS tightly coupled multisensor fusion (LDMF) strategy for drones’ autonomous and safe navigation. The drone is carried out with a front-facing camera to create visual geometric constraints and generate a 3D environmental map. Ulteriorly, a GNSS receiver with multiple constellations support is used to continuously provide pseudo-range, Doppler frequency shift and UTC time pulse signals to the drone navigation system. The proposed multisensor fusion strategy leverages the Kanade–Lucas algorithm to track multiple visual features in each input image. The local graph solution is bounded in a restricted sliding window, which can immensely predigest the computational complexity in factor graph optimization procedures. The drone navigation system can achieve camera-rate performance on a small companion computer. We thoroughly experimented with the LDMF system in both simulated and real-world environments, and the results demonstrate dramatic advantages over the state-of-the-art sensor fusion strategies.

Perceiving like a Bat: Hierarchical 3D Geometric–Semantic Scene Understanding Inspired by a Biomimetic Mechanism

“…Finally, we utilized our self-developed drone to perform a 3D geometric–semantic scene-understanding test in the real world (in Section 6.3 ). The evaluation for robot state estimation has already been performed in our previous work [ 32 , 33 , 34 , 35 ], so is not a contribution of this article.…”

“…We use the stereo image matching toolbox contained in the robot operating system (ROS) to convert binocular vision into spatial pointclouds. We employ the self-developed LDMF state estimation system [ 35 ] as the odometer for UAV pose estimation. All operations in this section were executed using an NVIDIA Jetson Xavier NX Embedded computer.…”

“…We directly leveraged the stereo image matching kit to convert RGB and depth images collected by the ZED camera into spatial pointclouds. We employed the self-developed LDMF state estimation system [ 35 ] as the visual–inertial odometry for drone pose estimation. We utilized BatNet-tiny to achieve pixel-level image semantic segmentation, and semantically propagated a label to each spatial mesh.…”

“…Finally, the UAV pose can be obtained by optimizing the UAV state vector by employing probability factor graph optimization. For the detailed process of factor graph optimization, please refer to our previous literature [ 32 , 33 , 34 , 35 ].…”

An All-Day Attitude Sensor Integrating Stars and Sun Measurement Based on Extended Pixel Response of CMOS APS Imager

“…In addition, since the positions of celestial bodies are absolute at a given moment, celestial navigation has no cumulative error so that it is of high value in long-timeduration missions [1], [9]. Unlike visual navigation, celestial navigation does not depend on the ground features of the working environment, and it does not need to map the complex environment [10,11]. Nowadays, automated celestial navigation is combined with other navigation methods to improve navigation accuracy, increase security, and provide an Hongjing Cao, Haiyang Zhan, Jingdong Li, Fei Xing, and Zheng You are with the Beijing Advanced Innovation Center for Integrated Circuits, Department of Precision Instrument, and the State Key Laboratory of Precision effective means of checking for major errors in special scenarios, especially in important military strategic occasions [12], [13].…”