Exocentric Control Scheme for Robot Applications: An Immersive Virtual Reality Approach

Abstract

Unmanned Aerial Vehicles (UAVs) exhibit great agility but usually require an experienced pilot to operate them in certain applications such as inspection for disaster scenarios or buildings. The reduction of cognitive overload when driving this kind of aerial robot becomes a challenge and several solutions can be found in the literature. A new virtual control scheme for reducing this cognitive overload when controlling an aerial robot is proposed in this paper. The architecture is based on a novel interaction Drone Exocentric Advanced Metaphor (DrEAM) located in a Cave Automated Virtual Environment (CAVE) and a real robot containing an embedded controller based on quaternion formulation. The testing room, where real robots are evolving, is located away from the CAVE and they are connected via UDP in a ground station. The user controls manually a virtual drone through the DrEAM interaction metaphor, and the real robot imitates autonomously in real time the trajectory imposed by the user in the virtual environment. Experimental results illustrate the easy implementation and feasibility of the proposed scheme in two different scenarios. Results from these tests show that the mental effort when controlling a drone using the proposed virtual control scheme is lower than when controlling it in direct view. Moreover, the easy maneuverability and controllability of the real drone is also demonstrated in real time experiments.

Complementarity, Interoperability, and Level of Integration of Humanitarian Drones with Emerging Digital Technologies: A State-of-the-Art Systematic Literature Review of Mathematical Models

“…XR is the least employed DT in the proposed models. Three studies used VR, one used AR, and none of them used MR. A VR control scheme for decreasing the cognitive overload when controlling a UAV and addressing the lack of situational awareness of human operators was proposed [222], where a virtual UAV pilots the real UAV. In [223], VR technology was used to compare the advantages and shortcomings of several algorithms to establish a path optimization and multitarget detection model.…”

“…In area coverage operations, UAVs are equipped with optical [199,201,202,204,209,215,216,222] or thermal cameras [141,224] for them to be able to perform the remote sensing of complex disaster environments. The models in [202,222,224] considered the azimuth or elevation [141,199,202] angles between the UAV and the camera.…”

“…In area coverage operations, UAVs are equipped with optical [199,201,202,204,209,215,216,222] or thermal cameras [141,224] for them to be able to perform the remote sensing of complex disaster environments. The models in [202,222,224] considered the azimuth or elevation [141,199,202] angles between the UAV and the camera. Several studies focused on the processing of the UAV images acquired from such operations, for map generation [215], image classification to eliminate redundant photographs [124,209], damage assessment [199], image segmentation for the detection of flooded areas [204], intelligent recognition of damaged poles [202], as well as image de-noising and feature extraction for accurate disaster identification [201].…”

“…Several studies focused on the processing of the UAV images acquired from such operations, for map generation [215], image classification to eliminate redundant photographs [124,209], damage assessment [199], image segmentation for the detection of flooded areas [204], intelligent recognition of damaged poles [202], as well as image de-noising and feature extraction for accurate disaster identification [201]. Some area coverage models targeted the human operators and, in particular, the reduction in their cognitive workload while piloting [222] and their safety while teleoperating a construction machine from a safe distance [216]. Authors [165] proposed the future implementation of their method in communication coverage for monitoring operations as well.…”

DUEL: Depth visUal Ego-motion Learning for autonomous robot obstacle avoidance

“…At the same time, the complex characteristics of the data distribution of expert obstacle avoidance policies makes it difficult to implement cognition intuitively. How to reasonably model and transform complex optimization objectives is also a challenge (Betancourt et al, 2022). Lastly, different experts have diverse policies due to latent factors such as changes in the external environment, and skills and habits of different experts.…”