Hand Gesture Controlled Drones: An Open Source Library

Abstract

Drones are conventionally controlled using joysticks, remote controllers, mobile applications, and embedded computers. A few significant issues with these approaches are that drone control is limited by the range of electromagnetic radiation and susceptible to interference noise. In this study we propose the use of hand gestures as a method to control drones. We investigate the use of computer vision methods to develop an intuitive way of agent-less communication between a drone and its operator. Computer vision-based methods rely on the ability of a drones camera to capture surrounding images and use pattern recognition to translate images to meaningful and/or actionable information. The proposed framework involves a few key parts toward an ultimate action to be taken. They are: image segregation from the video streams of front camera, creating a robust and reliable image recognition based on segregated images, and finally conversion of classified gestures into actionable drone movement, such as takeoff, landing, hovering and so forth. A set of five gestures are studied in this work. Haar feature-based AdaBoost classifier [1] is employed for gesture recognition. We also envisage safety of the operator and drone's action calculating the distance based on computer vision for this task. A series of experiments are conducted to measure gesture recognition accuracies considering the major scene variabilities, illumination, background, and distance. Classification accuracies show that well-lit, clear background, and within 3 ft gestures are recognized correctly over 90%. Limitations of current framework and feasible solutions for better gesture recognition are discussed, too. The software library we developed,and hand gesture datasets are open-sourced at project website.

Control of a Drone With Body Gestures

“…Researchers have been successful in creating single-camera systems, allowing the use of the drone's embedded camera. Hasanuzzaman, et al (2004) and Natarajan, et al, (2018) conducted such projects with hand gesture piloting. The solution, however, did not allow very efficient and accurate analysis of hand gestures.…”

Training of a deep learning algorithm for quadcopter gesture recognition

“…Based on the work done by [7] and [8], it is preferable to use vision-based methods based on their simplicity. Consequently, however, a need arises to efficiently process and interpret the large amount of data gathered using cameras in real time.…”

Smart Glove and Hand Gesture-based Control Interface For Multi-rotor Aerial Vehicles

“…The proposed approach has the following novelties and advantages in comparison with existing methods in [3,4,5,6,7,10,11]:…”

“…In [11], a Haar feature-based AdaBoost classifier has been used to categorize five different gestures for controlling a drone. While the accuracy of their framework is highest when the drone's operator posed within a distance of 3 ft, which is the sensing range of the camera used in the study, the proposed method works at its highest accuracy when the operator performs a gesture within the camera's sensing range, which is 8.5 ft.…”