Navion: A 2-mW Fully Integrated Real-Time Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones

Abstract

This paper presents Navion, an energy-efficient accelerator for visual-inertial odometry (VIO) that enables autonomous navigation of miniaturized robots (e.g., nano drones), and virtual/augmented reality on portable devices. The chip uses inertial measurements and mono/stereo images to estimate the drone's trajectory and a 3D map of the environment. This estimate is obtained by running a state-of-the-art VIO algorithm based on non-linear factor graph optimization, which requires large irregularly structured memories and heterogeneous computation flow. To reduce the energy consumption and footprint, the entire VIO system is fully integrated on chip to eliminate costly off-chip processing and storage. This work uses compression and exploits both structured and unstructured sparsity to reduce on-chip memory size by 4.1×. Parallelism is used under tight area constraints to increase throughput by 43%. The chip is fabricated in 65nm CMOS, and can process 752×480 stereo images from EuRoC dataset in real-time at 20 frames per second (fps) consuming only an average power of 2mW. At its peak performance, Navion can process stereo images at up to 171 fps and inertial measurements at up to 52 kHz, while consuming an average of 24mW. The chip is configurable to maximize accuracy, throughput and energy-efficiency trade-offs and to adapt to different environments. To the best of our knowledge, this is the first fully integrated VIO system in an ASIC.

Quantifying the design-space tradeoffs in autonomous drones

“…We find a wide variety of high-level algorithms in autonomous drones are dependent on a core family of algorithms, namely, simultaneous localization and mapping (SLAM) and visual odometry (VO) [39][40][41][42][43]. These algorithms are the fundamental building blocks for many autonomous technologies [19,40,44] and are used in various tasks such as navigation, obstacle avoidance, and path planning. Designing drone systems that provide accurate localization in realtime on platforms with limited computational and energy resources is an active area of research [18,19,24].…”

“…These algorithms are the fundamental building blocks for many autonomous technologies [19,40,44] and are used in various tasks such as navigation, obstacle avoidance, and path planning. Designing drone systems that provide accurate localization in realtime on platforms with limited computational and energy resources is an active area of research [18,19,24]. Therefore, to date, various implementations of SLAM with the focus on algorithmic-level optimizations [39,[41][42][43]45] or hardware acceleration [44,[46][47][48][49][50][51] have been proposed.…”

“…The quadcopter design possesses many advantages over other aerial vehicle designs in terms of simplicity and efficiency [15][16][17]. Thus, quadcopters are becoming prevalent and many control, planning, and perception methods have been assimilated for them [15,[18][19][20][21][22][23][24][25][26][27][28].…”

EDSSA: An Encoder-Decoder Semantic Segmentation Networks Accelerator on OpenCL-Based FPGA Platform

“…For intelligent robot technology, visual semantic SLAM (Simultaneous Localization and Mapping) that merges semantic information is a potential use of visual semantic segmentation for intelligent robot technology. It has been proven that the classic VSLAM (Visual SLAM) technology is an appropriate solution to the positioning and navigation of mobile robots [ 7 , 8 ], and that it can be implemented on low-power embedded platforms [ 9 , 10 , 11 ]. However, classic VSLAM is mostly based on low-level computer vision features (points, lines, etc.)…”

“…The hardware platform for processing of the SLAM algorithm on mobile robots mainly includes a CPU (Central Processing Unit) [ 11 ], FPGA [ 10 , 19 ], and ASIC (Application Specific Integrated Circuit) [ 9 , 20 ]. The platform for the semantic segmentation network is usually based on a GPU [ 15 ].…”