Review of Human–Machine Interfaces for Small Unmanned Systems With Robotic Manipulators

Abstract

This article reviews the human-machine interaction (HMI) technologies used for telemanipulation by small unmanned systems (SUS) with remote manipulators. SUS, including land, air, and sea vehicles, can perform a wide range of reconnaissance and manipulation tasks with varying levels of autonomy. SUS operations involving physical interactions with the environment require some level of operator involvement, ranging from direct control to goal-oriented supervision. Telemanipulation remains a challenging task for all levels of human interaction because the operator and the vehicle are not colocated, and operators require HMI technologies that facilitate manipulation from a remote location. This article surveys the human operator interfacing for over 70 teleoperated systems, summarizes the effects of physical and visual interface factors on user performance, and discusses these findings in the context of telemanipulating SUS. This article is of importance to SUS researchers and practitioners who will directly benefit from HMI implementations that improve telemanipulation performance.

Still Not Solved: A Call for Renewed Focus on User-Centered Teleoperation Interfaces

“…Using recent systematic surveys as a base [e.g., general telerobotics ( Niemeyer et al, 2016 ), interfaces for teleoperated manipulators ( Young and Peschel, 2020 ), or field robotics ( Liu and Nejat, 2013 ; Delmerico et al, 2019 ; Murphy and Tadokoro, 2019 )], we informally surveyed teleoperation interface research in recent years, as well as more influential work from the past 2 decades. We targeted the keywords of “teleoperation,” “interaction,” “interface,” and “user-centered,” in our journal and conference searches, and excluded work that was primarily engineering, algorithmic, or expert use-cases such as teleoperated space or surgery robots.…”

A Review of Human–Machine Cooperation in the Robotics Domain

“…In the execution layer, human execution includes the following: (15) the motion commands are sent from the HA; (17) the HA acts on interactive media; (19) the results of the robot execution are sent to the HA; (21) the human reflects motion to the outside motivation; and (27) the human copes with the object at hand. Robot execution includes the following: (16) the control signals are sent from the RA; (18) the RA acts on interactive media; (20) the results of the human execution are sent to the RA; (22) the signals from the sensors are directly sent to the actuations; and (28) the robot works on the object. In (31), the human and robot jointly work on the object; in (32), the object and the environment interact.…”

Application of Data-Driven Iterative Learning Algorithm in Transmission Line Defect Detection

“…In order to learn the model incrementally and adapt to unlabeled data, we use the improved Faster RCNN to initialize and train the deep learning neural network and observe the hidden features of defects from different dimensions. In order to make preliminary judgments on the defects, a small amount of labeled data is used to initialize the training network in the early stage of training, and then we continue to perform sample mining in a large amount of unlabeled data in a humanmachine collaborative manner [20][21][22].…”