Robust Person Identification and Following in a Mobile Robot Based on Deep Learning and Optical Tracking

Abstract

There is an exciting synergy between deep learning and robotics, combining the perception skills a deep learning system can achieve with the wide variety of physical responses a robot can perform. This article describes an embedded system integrated into a mobile robot capable of identifying and following a specific person reliably based on a convolutional neural network pipeline. In addition, the design incorporates an optical tracking system for supporting the inferences of the neural networks, allowing the determination of the position of a person using an RGB depth camera. The system runs on an NVIDIA Jetson TX2 board, an embedded System-on-Module capable of performing computationally demanding tasks onboard and handling the complexity needed to run a solid tracking and following algorithm. A robotic mobile base with the Jetson TX2 board attached receives velocity orders to move the system toward the target. The proposed approach has been validated on a mobile robotic platform that successfully follows a determined person, relying on the robustness of the combination of deep learning with optical tracking for working efficiently in a real environment.

Bit-Weight Adjustment for Bridging Uniform and Non-Uniform Quantization to Build Efficient Image Classifiers

“…After quantizing both the weights and activations, a standard 32-bit convolution can be substituted with the sum of the binary convolutions, as shown in Equation (3). Bitwise operations like and, xnor, and popcnt can be leveraged to compute the binary convolutions, significantly reducing computational complexity.…”

“…The success of deep learning models, especially convolutional neural networks (CNNs) [1,2], has dramatically facilitated the advancement of computer vision and graphics, making CNNs the primary tool within the community of computational visual media. To date, CNNs have made remarkable achievements across a wide range of vision tasks and applications such as person identification [3], object detection [4], action recognition [5], and image classification [6].…”