TeMoto: Intuitive Multi-Range Telerobotic System with Natural Gestural and Verbal Instruction Interface

Abstract

Teleoperated mobile robots, equipped with object manipulation capabilities, provide safe means for executing dangerous tasks in hazardous environments without putting humans at risk. However, mainly due to a communication delay, complex operator interfaces and insufficient Situational Awareness (SA), the task productivity of telerobots remains inferior to human workers. This paper addresses the shortcomings of telerobots by proposing a combined approach of (i) a scalable and intuitive operator interface with gestural and verbal input, (ii) improved Situational Awareness (SA) through sensor fusion according to documented best practices, (iii) integrated virtual fixtures for task simplification and minimizing the operator's cognitive burden and (iv) integrated semiautonomous behaviors that further reduce cognitive burden and negate the impact of communication delays, execution latency and/or failures. The proposed teleoperation system, TeMoto, is implemented using ROS (Robot Operating System) to ensure hardware agnosticism, extensibility and community access. The operator's command interface consists of a Leap Motion Controller for hand tracking, Griffin PowerMate USB as turn knob for scaling and a microphone for speech input. TeMoto is evaluated on multiple robots including two mobile manipulator platforms. In addition to standard, task-specific evaluation techniques (completion time, user studies, number of steps, etc.)-which are platform and task dependent and thus difficult to scale-this paper presents additional metrics for evaluating the user interface including task-independent criteria for measuring generalized (i) task completion efficiency and (ii) operator context switching.

High-Level Teleoperation System for Aerial Exploration of Indoor Environments

“…Very recent work on fully working teleoperation systems for indoor environments is discussed by Valner et al ( 2018 ). In their work, they introduce high-level interaction methods based on gesture and language, but for ground-based robots.…”

Precision Improvement and Delay Reduction in Surgical Telerobotics

“…This can circumvent unnecessary activities of the robot. Robert et al [6] proposed a teleoperational mobile robot called TeMoto that uses robot operating system. This system enables community access, extensibility and hardware agnosticism.…”

Communication of Hazards in Mixed-Reality Telerobotic Systems: The Usage of Naturalistic Avoidance Cues in Driving Tasks

“…In telerobotic tasks errors are likely to occur when the perception of a remote site is limited and/or the available information/interface overwhelms the human operator (e.g., increased cognitive load) (Yanco et al, 2015). In order to lessen the occurrences of operating errors in hazardous situations, many studies have explored supervisory controls and predictive displays in telerobotic systems (Deng & Jagersand, 2003; Kheddar et al, 2002; Ricks et al, 2004; Valner et al, 2018). One technique used to increase the operator’s sense of presence and aid in situational awareness is the incorporation of real-time video and real-time feedback (Azuma et al, 2001; Bell et al, 2002; Green et al, 2008; Livingston et al, 2002).…”

“…One technique used to increase the operator’s sense of presence and aid in situational awareness is the incorporation of real-time video and real-time feedback (Azuma et al, 2001; Bell et al, 2002; Green et al, 2008; Livingston et al, 2002). Approaches to the incorporation of video includes direct video from the remote site (Sheridan, 1992); stitching together video from different image sources; overlaying video feed with virtual objects and instructions (Scurati et al, 2018); mixing direct video feed with model-based representation (Labonte et al, 2010; Marion et al, 2017; Valner et al, 2018); augmenting/overlaying video feed with coordinate cues (Chintamani et al, 2010) to minimize navigation errors, and so on.…”