Using visuomotor tendencies to increase control performance in teleoperation

Webb, Jeremy D.; Li, Songpo; Zhang, Xiaoli

doi:10.1109/acc.2016.7526794

Cited by 14 publications

(9 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Current assistance with (semi-)autonomous agents has focused on approaching/reaching tasks in teleoperation (Khoramshahi and Billard (2018); Michelman and Allen (2002); Kaupp et al (2010); Mulling et al (2015)), however, it is not sufficient to satisfy the tele-grasping and telemanipulation of objects. The methods to provide assistance in approaching-yet may not work as well in grasping scenarios-include envelope motion constraints (Abbott et al (2007); Webb et al (2016)), manually selective assistance levels (Feygin et al (2002); Li and Okamura (2003)), and shared control policies such as linear blending (Aarno et al (2005); Dragan and Srinivasa (2013)). Linear blending strategies may not entirely work as the motion constraints from the manual operator's perspective and the fully autonomous perspective may differ.…”

Section: Related Workmentioning

confidence: 99%

An Intent-based Task-aware Shared Control Framework for Intuitive Hands Free Telemanipulation

Bowman,

Zhang,

Zhang

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Shared control in teleoperation for providing robot assistance to accomplish object manipulation is a new challenging problem. This has unique challenges, on top of teleoperation challenges in general, due to difficulties of physical discrepancy between human hands and robot hands as well as the fine motion constraints to constitute task success. In this work we are interested in the scientific underpinnings of robot assistance for telemanipulation by defining an arbitration strategy for a physically discrepant robot hand structure which flexibly reproduce actions that accommodate the operator's motion inputs as well as autonomously regulate the actions to compensate task constraints that facilitate subsequent manipulation. The overall objective is to present a control strategy where the focus is on generating poses which are better suited for human perception of successful teleoperated object manipulation and feeling of being in control of the robot, rather than developing objective stable grasp configurations for task success. Identifying the commonalities between human and robot feature sets enable the accommodation for the arbitration framework to indicate and appear to intuitively follow the user. Additionally, it is imperative to understand how users perceive good arbitration in object telemanipulation. We have also conducted a user study to analyze the effect of factors including task predictability, perceived following, and user preference.

show abstract

Section: Related Workmentioning

confidence: 99%

An Intent-based Task-aware Shared Control Framework for Intuitive Hands Free Telemanipulation

Bowman,

Zhang,

Zhang

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Gaze estimation enables the study of visual perception mechanisms in humans, and has been used in many fields, such as action recognition [1], situation awareness estimation [2], and driver attention analysis [3]. It is also a non-verbal communication method, and thus, it can be applied to shared autonomy [4] or teleoperation [5] in the context of Human-Robot Interaction (HRI).…”

Section: Introductionmentioning

confidence: 99%

4D Attention: Comprehensive Framework for Spatio-Temporal Gaze Mapping

Oishi

Koide

Yokozuka

et al. 2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

This study presents a framework for capturing human attention in the spatio-temporal domain using eyetracking glasses. Attention mapping is a key technology for human perceptual activity analysis or Human-Robot Interaction (HRI) to support human visual cognition; however, measuring human attention in dynamic environments is challenging owing to the difficulty in localizing the subject and dealing with moving objects. To address this, we present a comprehensive framework, 4D Attention, for unified gaze mapping onto static and dynamic objects. Specifically, we estimate the glasses pose by leveraging a loose coupling of direct visual localization and Inertial Measurement Unit (IMU) values. Further, by installing reconstruction components into our framework, dynamic objects not captured in the 3D environment map are instantiated based on the input images. Finally, a scene rendering component synthesizes a first-person view with identification (ID) textures and performs direct 2D-3D gaze association. Quantitative evaluations showed the effectiveness of our framework. Additionally, we demonstrated the applications of 4D Attention through experiments in real situations 1 . 4D attention Camera inputRendered scene ID Object mask

show abstract

“…A system has been developed to enable users to specify a 2D end-effector path via a click-and-drag operation, and the collision avoidance is implemented with a sampling-based motion planner (Nicholas et al, 2013). The operator's gaze is employed to indicate the target, and then the robotic arm is guided to reach the target, both by utilizing potential fields for autonomy (Webb et al, 2016). In the abovementioned paradigms, since the low-level robot motions are exclusively realized with the motion planner-based autonomy without the involvement of users, the user can regain the control authority only when the reactive behavior (e.g., collision avoidance) finishes (Kim et al, 2006).…”

Section: Introductionmentioning

confidence: 99%

Semi-Autonomous Robotic Arm Reaching With Hybrid Gaze–Brain Machine Interface

Zeng

Shen

et al. 2020

Front. Neurorobot.

View full text Add to dashboard Cite

Recent developments in the non-muscular human-robot interface (HRI) and shared control strategies have shown potential for controlling the assistive robotic arm by people with no residual movement or muscular activity in upper limbs. However, most non-muscular HRIs only produce discrete-valued commands, resulting in non-intuitive and less effective control of the dexterous assistive robotic arm. Furthermore, the user commands and the robot autonomy commands usually switch in the shared control strategies of such applications. This characteristic has been found to yield a reduced sense of agency as well as frustration for the user according to previous user studies. In this study, we firstly propose an intuitive and easy-to-learn-and-use hybrid HRI by combing the Brain-machine interface (BMI) and the gaze-tracking interface. For the proposed hybrid gaze-BMI, the continuous modulation of the movement speed via the motor intention occurs seamlessly and simultaneously to the unconstrained movement direction control with the gaze signals. We then propose a shared control paradigm that always combines user input and the autonomy with the dynamic combination regulation. The proposed hybrid gaze-BMI and shared control paradigm were validated for a robotic arm reaching task performed with healthy subjects. All the users were able to employ the hybrid gaze-BMI for moving the end-effector sequentially to reach the target across the horizontal plane while also avoiding collisions with obstacles. The shared control paradigm maintained as much volitional control as possible, while providing the assistance for the most difficult parts of the task. The presented semi-autonomous robotic system yielded continuous, smooth, and collision-free motion trajectories for the end effector approaching the target. Compared to a system without assistances from robot autonomy, it significantly reduces the rate of failure as well as the time and effort spent by the user to complete the tasks.

show abstract

Using visuomotor tendencies to increase control performance in teleoperation

Cited by 14 publications

References 14 publications

An Intent-based Task-aware Shared Control Framework for Intuitive Hands Free Telemanipulation

An Intent-based Task-aware Shared Control Framework for Intuitive Hands Free Telemanipulation

4D Attention: Comprehensive Framework for Spatio-Temporal Gaze Mapping

Semi-Autonomous Robotic Arm Reaching With Hybrid Gaze–Brain Machine Interface

Contact Info

Product

Resources

About