A 64-mW DNN-Based Visual Navigation Engine for Autonomous Nano-Drones

Abstract

Fully-autonomous miniaturized robots (e.g., drones), with artificial intelligence (AI) based visual navigation capabilities, are extremely challenging drivers of Internet-of-Things edge intelligence capabilities. Visual navigation based on AI approaches, such as deep neural networks (DNNs) are becoming pervasive for standard-size drones, but are considered out of reach for nano-drones with a size of a few cm 2 . In this work, we present the first (to the best of our knowledge) demonstration of a navigation engine for autonomous nano-drones capable of closed-loop end-to-end DNN-based visual navigation. To achieve this goal we developed a complete methodology for parallel execution of complex DNNs directly on board resourceconstrained milliwatt-scale nodes. Our system is based on GAP8, a novel parallel ultra-low-power computing platform, and a 27 g commercial, open-source CrazyFlie 2.0 nano-quadrotor. As part of our general methodology, we discuss the software mapping techniques that enable the state-of-the-art deep convolutional neural network presented in [1] to be fully executed aboard within a strict 6 fps real-time constraint with no compromise in terms of flight results, while all processing is done with only 64 mW on average. Our navigation engine is flexible and can be used to span a wide performance range: at its peak performance corner, it achieves 18 fps while still consuming on average just 3.5% of the power envelope of the deployed nano-aircraft. To share our key findings with the embedded and robotics communities and foster further developments in autonomous nano-UAVs, we publicly release all our code, datasets, and trained networks.

FANN-on-MCU: An Open-Source Toolkit for Energy-Efficient Neural Network Inference at the Edge of the Internet of Things

“…Despite being an academic research platform, PULP offers the maturity of a commercial device with OpenMP, OpenCL, and OpenVX support to enable agile application porting, development, performance tuning, and debugging. GreenWaves Technologies produces commercial devices based on the opensource PULP platform to design ultra-low-power embedded solutions for image, sound, and vibration AI processing in sensing devices [39].…”

“…It provides Bluetooth Low Energy (BLE) 5 communication capabilities, performs power management in various modes of operation (sleep, raw data streaming, data acquisition, and processing), and keeps track of the battery charging status. The dual-processor architecture of InfiniWolf allows local end-to-end processing (i.e., on-board classification using ML) with lower power and higher energy efficiency than streaming the data out for remote analysis [41].…”

Wi-Fi-Inertial Indoor Pose Estimation for Microaerial Vehicles

“…The required sensors in our system, which include an IMU and a wireless card, already have miniature designs and they are available in miniature platforms like Crazyflie [29]. The computation power of Crazyflie is enough as it can run a deep neural network for visual processing as demonstrated by [30]. Our WINS is a linear estimator that can be computed quite efficient.…”

Autonomous Navigation via Deep Reinforcement Learning for Resource Constraint Edge Nodes Using Transfer Learning

“…Authors of [11] design and implement an energy-efficient accelerator for visual-inertial odometry (VIO) that enables autonomous navigation of miniaturized robots. [12] demonstrates a navigation engine for autonomous nano-drones which is capable of closed-loop end-to-end DNN-based visual navigation. The other approach is to devise better and improved algorithms that take lesser amount of computations (hence energy) for similar performance such as model compression [13], [14].…”