An Intuitive End-to-End Human-UAV Interaction System for Field Exploration

Abstract

This paper presents an intuitive end-to-end interaction system between a human and a hexacopter Unmanned Aerial Vehicle (UAV) for field exploration in which the UAV can be commanded by natural human poses. Moreover, LEDs installed on the UAV are used to communicate the state and intents of the UAV to the human as feedback throughout the interaction. A real time multi-human pose estimation system is built that can perform with low latency while maintaining competitive performance. The UAV is equipped with a robotic arm, kinematic and dynamic attitude models for which are provided by introducing the center of gravity (COG) of the vehicle. In addition, a super-twisting extended state observer (STESO)-based back-stepping controller (BSC) is constructed to estimate and attenuate complex disturbances in the attitude control system of the UAV, such as wind gusts, model uncertainties, etc. A stability analysis for the entire control system is also presented based on the Lyapunov stability theory. The pose estimation system is integrated with the proposed intelligent control architecture to command the UAV to execute an exploration task stably. Additionally, all the components of this interaction system are described. Several simulations and experiments have been conducted to demonstrate the effectiveness of the whole system and its individual components.

İnsan-İHA Etkileşimi İçin Giyilebilir Eldiven Hareketlerinin Sınıflandırılması ve Analizi

“…Son yıllarda İHA kullanımının artması bunun sonucunda gelişen ihtiyaç yönetimi İnsan-İHA etkileşimini önemli kılmıştır. Literatürde yapılan çalışmalara bakılacak olursa gelecekte İHA gibi robot sistemleriyle insanların iletişim sıklığı artacaktır [13][14][15][16][17][18][19][20]. Bu iletişime kolaylık sağlamak amacıyla İnsan-İHA etkileşimi ve giyilebilir teknolojileri beraber kullanma açısından yeni bir yöntem önerilmiştir.…”

An Intelligent Human–Unmanned Aerial Vehicle Interaction Approach in Real Time Based on Machine Learning Using Wearable Gloves

“…Work performed in areas that are not safe for humans and difficult to access can be performed thanks to human–UAV interactions [ 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 ]. One of the areas where human UAV–interactions are used is computer-vision-based hand gesture recognition applications, which enable users to determine hand gestures using pattern recognition algorithms [ 23 , 24 , 25 , 26 , 27 , 28 , 29 ]. Although this method can detect a large number of movements, its efficacy can be affected by the variability of ambient lighting and possible obstacles in the line of sight since this method relies on computer vision.…”

“…Human–UAV interactions are only used for information exchange in some applications, while some applications employ physical contact or force exchange. As can be seen in Figure 1 , this interaction can be performed using a software interface alone or by using physical [ 11 ], tactile [ 12 ], auditory [ 44 ], and human kinetics techniques or creating a specific rule table [ 20 , 21 , 22 , 23 ].…”

“…One of the areas where human–UAV interactions are used is computer-vision-based motion recognition applications that enable users to make predictions using pattern recognition algorithms such as hand gestures and body movements [ 23 , 24 , 25 , 26 , 27 ]. Motion recognition applications are generally costly—e.g., UAV cameras, CCD and micro cameras, Leap motion controllers, and Kinect.…”

Optimal UAV's Deployment and Transmit Power Design for Two Users Uplink NOMA Systems

“…The researches related to unmanned aerial vehicles (UAVs) has become a hot topic due to their wide application prospects, such as goods delivery, search and rescue, aerial photography, and telecommunications (Zeng et al, 2016 ). For example, Jiao et al ( 2020 ) presented an intuitive end-to-end interaction system between a human and an UAV in which the UAV can be commanded by natural human poses. Moreover, a brain-inspired decision-making spiking neural network (BDM-SNN) was proposed in Zhao et al ( 2018 ), which can help UAV making decisions in some tasks.…”