The use of gaze to control drones

Hansen, John Paulin; Alapetite, Alexandre; MacKenzie, I. Scott; Møllenbach, Emilie

doi:10.1145/2578153.2578156

Cited by 80 publications

(25 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Birdly is a simulator that creates a virtual reality environment to allows users to fly like a bird. A modified version of Birdly described in [51] allows users to control an actual drone through body movements and the camera gimbal through head movement, creating an immersive flight experience. The author's goal was to develop an effective and natural embodied human-drone interaction that is easier to train and more immersive than traditional interfaces such as joysticks, keyboards, and touch-screens.…”

Section: E Other Control Interfacesmentioning

confidence: 99%

The State-of-the-Art of Human–Drone Interaction: A Survey

2019

View full text Add to dashboard Cite

Drones have expanded from military operations to performing a broad range of civilian applications. As drone usage increases, humans will interact with such systems more often, therefore, it is important to achieve a natural human-drone interaction. Although some knowledge can be derived from the field of human-robot interaction, drones can fly in a 3D space, which essentially changes how humans can interact with them, making human-drone interaction a field of its own. This paper is the first survey on the emerging field of human-drone interaction focusing on multi-rotor systems, providing an overview of existing literature and the current state of the art in the field. This work begins with an analysis and comparison of the drone models that are commonly used by end-users and researchers in the field of humandrone interaction. Following, the current state of the field is discussed, including the roles of humans in HDI, innovative control methods, remaining aspects of interaction, and novelty drone prototypes and applications. This paper concludes by presenting a discussion of current challenges and future work in the field of humandrone interaction. INDEX TERMS Drone, human-computer interaction, human-drone interaction, human-in-the-loop, humanrobot interaction, unmanned aerial vehicle.

show abstract

Section: E Other Control Interfacesmentioning

confidence: 99%

The State-of-the-Art of Human–Drone Interaction: A Survey

2019

View full text Add to dashboard Cite

show abstract

“…This approach is particularly useful for impaired users. For example, Hansen et al [11] present an experimental study on gaze-based control for UAVs. Gaze is used to control a combination of the drone's translation, altitude, rotation, and speed.…”

Section: Related Workmentioning

confidence: 99%

Connectivity-Maintenance Teleoperation of a UAV Fleet With Wearable Haptic Feedback

Aggravi

Pacchierotti²,

Giordano³

2021

IEEE Trans. Automat. Sci. Eng.

View full text Add to dashboard Cite

This paper presents the design of a decentralized connectivity-maintenance algorithm for the teleoperation of a team of multiple UAVs, together with an extensive human subject evaluation in virtual and real environments. The proposed connectivity-maintenance algorithm enhances earlier works by improving their applicability, safety, effectiveness, and ease of use, by including: (i) an airflow-avoidance behavior that avoids stack downwash phenomena in rotor-based aerial robots; (ii) a consensus-based action for enabling fast displacements with minimal topology changes by having all follower robots moving at the leader's velocity; (iii) an automatic decrease of the minimum degree of connectivity, enabling an intuitive and dynamic expansion/compression of the formation; and (iv) an automatic detection and resolution of deadlock configurations, i.e., when the robot leader cannot move due to counterbalancing connectivityand external-related inputs. We also devised and evaluated different interfaces for teleoperating the team as well as different ways of receiving information about the connectivity force acting on the leader. Results of two human subject experiments show that the proposed algorithm is effective in various situations. Moreover, using haptic feedback to provide information about the team connectivity outperforms providing both no feedback at all and sensory substitution via visual feedback. Note to Practitioners-The control of one drone is usually performed with a remote controller (similar to a joypad) that uses radio-wave signals. When controlling more than one drone, even a simple task such as moving the whole team around become very challenging. Developing an easy, yet efficient way to impart commands to a formation of drones is necessary to achieve any complex task. This work proposes a framework to control a fleet of drones (quadrotors) in an intuitive way, while receiving meaningful and effective information on the state of the formation. The proposed technique does not rely on any absolute positioning system (e.g., GPS) or centralized command center. Instead, it only uses the relative position of the drones with respect to each other, and all computations are designed in a decentralized fashion. These features make the proposed framework ready for deployment in different highimpact applications, such as in surveillance, search-and-rescue, and disaster response scenarios. Index Terms-Multi-robot systems, Human-centered robotics.

show abstract

“…The demo explored the use of gaze as an input for locomotion. In [47], the researchers presented an experimental investigation of gaze-based control modes for UAV, and in [48], the scholars investigated a gaze-controlled remote vehicle while driving it on a racing track with five different input devices.…”

Section: Gaze-based Interaction and Controlmentioning

confidence: 99%

GazeGuide: An Eye-Gaze-Guided Active Immersive UAV Camera

et al. 2020

View full text Add to dashboard Cite

Over the years, gaze input modality has been an easy and demanding human–computer interaction (HCI) method for various applications. The research of gaze-based interactive applications has advanced considerably, as HCIs are no longer constrained to traditional input devices. In this paper, we propose a novel immersive eye-gaze-guided camera (called GazeGuide) that can seamlessly control the movements of a camera mounted on an unmanned aerial vehicle (UAV) from the eye-gaze of a remote user. The video stream captured by the camera is fed into a head-mounted display (HMD) with a binocular eye tracker. The user’s eye-gaze is the sole input modality to maneuver the camera. A user study was conducted considering the static and moving targets of interest in a three-dimensional (3D) space to evaluate the proposed framework. GazeGuide was compared with a state-of-the-art input modality remote controller. The qualitative and quantitative results showed that the proposed GazeGuide performed significantly better than the remote controller.

show abstract

The use of gaze to control drones

Cited by 80 publications

References 23 publications

The State-of-the-Art of Human–Drone Interaction: A Survey

The State-of-the-Art of Human–Drone Interaction: A Survey

Connectivity-Maintenance Teleoperation of a UAV Fleet With Wearable Haptic Feedback

GazeGuide: An Eye-Gaze-Guided Active Immersive UAV Camera

Contact Info

Product

Resources

About