Navion: A Fully Integrated Energy-Efficient Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones

Suleiman, Amr; Zhang, Zhengdong; Carlone, Luca; Karaman, Sertaç; Sze, Vivienne

doi:10.1109/vlsic.2018.8502279

Cited by 36 publications

(22 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [23], an application-specific integrated circuit (ASIC), called NAVION, for onboard visual-inertial odometry was presented. Although this chip exposes enough computational power to perform state estimation up to 171 fps within 24 mW, this represents only one among other basic functionalities required by any UAV to be fully autonomous.…”

Section: Related Workmentioning

confidence: 99%

A 64-mW DNN-Based Visual Navigation Engine for Autonomous Nano-Drones

Palossi

Loquercio

Conti

et al. 2019

IEEE Internet Things J.

154

117

View full text Add to dashboard Cite

Fully-autonomous miniaturized robots (e.g., drones), with artificial intelligence (AI) based visual navigation capabilities, are extremely challenging drivers of Internet-of-Things edge intelligence capabilities. Visual navigation based on AI approaches, such as deep neural networks (DNNs) are becoming pervasive for standard-size drones, but are considered out of reach for nano-drones with a size of a few cm 2 . In this work, we present the first (to the best of our knowledge) demonstration of a navigation engine for autonomous nano-drones capable of closed-loop end-to-end DNN-based visual navigation. To achieve this goal we developed a complete methodology for parallel execution of complex DNNs directly on board resourceconstrained milliwatt-scale nodes. Our system is based on GAP8, a novel parallel ultra-low-power computing platform, and a 27 g commercial, open-source CrazyFlie 2.0 nano-quadrotor. As part of our general methodology, we discuss the software mapping techniques that enable the state-of-the-art deep convolutional neural network presented in [1] to be fully executed aboard within a strict 6 fps real-time constraint with no compromise in terms of flight results, while all processing is done with only 64 mW on average. Our navigation engine is flexible and can be used to span a wide performance range: at its peak performance corner, it achieves 18 fps while still consuming on average just 3.5% of the power envelope of the deployed nano-aircraft. To share our key findings with the embedded and robotics communities and foster further developments in autonomous nano-UAVs, we publicly release all our code, datasets, and trained networks.

show abstract

Section: Related Workmentioning

confidence: 99%

A 64-mW DNN-Based Visual Navigation Engine for Autonomous Nano-Drones

Palossi

Loquercio

Conti

et al. 2019

IEEE Internet Things J.

154

117

View full text Add to dashboard Cite

show abstract

“…One common drawback of VO and SLAM methods is that they are computationally intensive and thus reserved for larger MAVs (Ghadiok et al, 2012 ; Schauwecker and Zell, 2014 ). However, recent developments have also seen the introduction of more light-weight solutions, such as Navion (Suleiman et al, 2018 ).…”

Section: Local Ego-state Estimation and Controlmentioning

confidence: 99%

A Survey on Swarming With Micro Air Vehicles: Fundamental Challenges and Constraints

et al. 2020

View full text Add to dashboard Cite

This work presents a review and discussion of the challenges that must be solved in order to successfully develop swarms of Micro Air Vehicles (MAVs) for real world operations. From the discussion, we extract constraints and links that relate the local level MAV capabilities to the global operations of the swarm. These should be taken into account when designing swarm behaviors in order to maximize the utility of the group. At the lowest level, each MAV should operate safely. Robustness is often hailed as a pillar of swarm robotics, and a minimum level of local reliability is needed for it to propagate to the global level. An MAV must be capable of autonomous navigation within an environment with sufficient trustworthiness before the system can be scaled up. Once the operations of the single MAV are sufficiently secured for a task, the subsequent challenge is to allow the MAVs to sense one another within a neighborhood of interest. Relative localization of neighbors is a fundamental part of self-organizing robotic systems, enabling behaviors ranging from basic relative collision avoidance to higher level coordination. This ability, at times taken for granted, also must be sufficiently reliable. Moreover, herein lies a constraint: the design choice of the relative localization sensor has a direct link to the behaviors that the swarm can (and should) perform. Vision-based systems, for instance, force MAVs to fly within the field of view of their camera. Range or communication-based solutions, alternatively, provide omni-directional relative localization, yet can be victim to unobservable conditions under certain flight behaviors, such as parallel flight, and require constant relative excitation. At the swarm level, the final outcome is thus intrinsically influenced by the on-board abilities and sensors of the individual. The real-world behavior and operations of an MAV swarm intrinsically follow in a bottom-up fashion as a result of the local level limitations in cognition, relative knowledge, communication, power, and safety. Taking these local limitations into account when designing a global swarm behavior is key in order to take full advantage of the system, enabling local limitations to become true strengths of the swarm.

show abstract

“…An emerging trend in the evolution of autonomous navigation systems is the design and development of applicationspecific integrated circuit (ASIC) addressing specific navigation tasks [16], [17]. ASICs deliver levels of performance and energy efficiency for the specific tasks addressed that cannot be achieved by typical nano-UAV computing platforms for workloads such as visual odometry [16] or simultaneous localization and mapping (SLAM) [17]. However, ASICs only accelerate a part of the overall functionality, requiring pairing with additional circuits for complementary onboard computation as well as for interacting with the drone's sensors.…”

Section: Related Workmentioning

confidence: 99%

An Open Source and Open Hardware Deep Learning-Powered Visual Navigation Engine for Autonomous Nano-UAVs

Palossi

Conti

Benini

2019

2019 15th International Conference on Distributed Computing in Sensor Systems (DCOSS)

View full text Add to dashboard Cite

Nano-size unmanned aerial vehicles (UAVs), with few centimeters of diameter and sub-10 Watts of total power budget, have so far been considered incapable of running sophisticated visual-based autonomous navigation software without external aid from base-stations, ad-hoc local positioning infrastructure, and powerful external computation servers. In this work, we present what is, to the best of our knowledge, the first 27 g nano-UAV system able to run aboard an end-to-end, closedloop visual pipeline for autonomous navigation based on a stateof-the-art deep-learning algorithm, built upon the open-source CrazyFlie 2.0 nano-quadrotor. Our visual navigation engine is enabled by the combination of an ultra-low power computing device (the GAP8 system-on-chip) with a novel methodology for the deployment of deep convolutional neural networks (CNNs). We enable onboard real-time execution of a state-of-the-art deep CNN at up to 18 Hz. Field experiments demonstrate that the system's high responsiveness prevents collisions with unexpected dynamic obstacles up to a flight speed of 1.5 m/s. In addition, we also demonstrate the capability of our visual navigation engine of fully autonomous indoor navigation on a 113 m previously unseen path. To share our key findings with the embedded and robotics communities and foster further developments in autonomous nano-UAVs, we publicly release all our code, datasets, and trained networks.

show abstract

Navion: A Fully Integrated Energy-Efficient Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones

Cited by 36 publications

References 3 publications

A 64-mW DNN-Based Visual Navigation Engine for Autonomous Nano-Drones

A 64-mW DNN-Based Visual Navigation Engine for Autonomous Nano-Drones

A Survey on Swarming With Micro Air Vehicles: Fundamental Challenges and Constraints

An Open Source and Open Hardware Deep Learning-Powered Visual Navigation Engine for Autonomous Nano-UAVs

Contact Info

Product

Resources

About