Implicit Intention Communication in Human–Robot Interaction Through Visual Behavior Studies

Li, Songpo; Zhang, Xiaoli

doi:10.1109/thms.2017.2647882

Cited by 61 publications

(36 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the early 2000's, the eyetracker was used as a direct substitute for a handheld mouse such that the gaze point on a computer display designates the cursor's position, and blinks function as button clicks (Lin et al, 2006 ; Gajwani and Chhabria, 2010 ). Since 2015, eye gaze has been used to communicate a 3D target position (Li et al, 2015a , 2017 ; Dziemian et al, 2016 ; Li and Zhang, 2017 ; Wang et al, 2018 ; Zeng et al, 2020 ) for directing the movement of the robotic end effector. No action recognition was required, as these methods assumed specific actions in advance, such as reach and grasp (Li et al, 2017 ), write and draw (Dziemian et al, 2016 ), and pick and place (Wang et al, 2018 ).…”

Section: Related Workmentioning

confidence: 99%

Toward Shared Autonomy Control Schemes for Human-Robot Systems: Action Primitive Recognition Using Eye Gaze Features

2020

View full text Add to dashboard Cite

The functional independence of individuals with upper limb impairment could be enhanced by teleoperated robots that can assist with activities of daily living. However, robot control is not always intuitive for the operator. In this work, eye gaze was leveraged as a natural way to infer human intent and advance action recognition for shared autonomy control schemes. We introduced a classifier structure for recognizing low-level action primitives that incorporates novel three-dimensional gaze-related features. We defined an action primitive as a triplet comprised of a verb, target object, and hand object. A recurrent neural network was trained to recognize a verb and target object, and was tested on three different activities. For a representative activity (making a powdered drink), the average recognition accuracy was 77% for the verb and 83% for the target object. Using a non-specific approach to classifying and indexing objects in the workspace, we observed a modest level of generalizability of the action primitive classifier across activities, including those for which the classifier was not trained. The novel input features of gaze object angle and its rate of change were especially useful for accurately recognizing action primitives and reducing the observational latency of the classifier.

show abstract

Section: Related Workmentioning

confidence: 99%

Toward Shared Autonomy Control Schemes for Human-Robot Systems: Action Primitive Recognition Using Eye Gaze Features

2020

View full text Add to dashboard Cite

show abstract

“…1) was applied, which had successfully been applied in some previous research. [15][16][17][18][19][20][21] It is an ultra-portable device (320 x 45 x 40 mm, 145 g) that can move 25 cm horizontally, 11 cm vertically and 15 cm in-depth; it can be fitted on the monitor and uses infra camera observation and image procession to detect and follow eye movement with 60 Hz sample rate. It is easy to handle with 0.5-1 degree of visual angle and 5-or 9 point-calibration options.…”

Section: The Gazepointgp3 Eye-tracker Hardware Unitmentioning

confidence: 99%

Evaluation of Eye-Movement Metrics ina Software DebbugingTask using GP3 Eye Tracker

Kővári¹,

Katona²,

Costescu³

2020

ACTA POLYTECH HUNG

View full text Add to dashboard Cite

Teaching different programming subjects is an increasing challenge nowadays as because of growing user demands, the latest paradigms and technologies must be taught. Students in higher education now are of the age who were born into the digital world; however, the success of fulfilling programming courses is lagging behind. Humancomputer interface-based research has emerged in numerous fields of science recently, which could lead to the revolutionising of education. These interfaces could also help professors as a support system in transferring knowledge in a more efficient way besides supporting students acquiring adequate home learning methods. In this study, the applicability of eye movement tracking systems in respect of a programming task is examined, in which during the exploration and correction of the errors of an incorrectly functioning algorithm, the eye movement parameters are observed, recorded and evaluated. The test subjects participating in the research were divided into two groups according to some of their characteristic parameters, where the first group during debugging rather applied minor modifications and the more common technique of compile and run, which otherwise is also the most characteristic feature of students studying programming, while the members of the other group increased emphasis on analysing. In the statistic evaluation of the research, the parameters characteristic of the eye movement tracking of the two groups, as well as the efficiency of these groups were analysed. Based on the results, regarding the efficiency and the number of fixations, a significant difference could be shown between the two groups (U=22.5, Z=-2.236, p=0.025 (2-tailed), r=0.48) and (t(20)=2.507, p=0.021 (2-tailed), d=1.106), while concerning the duration of fixation and the saccade length, the difference shown was infinitesimal (t(20)=0.544, p=0.592 (2tailed), d=0.26) and (t(19.992)=1.965, p=0.063 (2-tailed), d=0.79).

show abstract

“…Joint action in humans Feinman et al, 1992;Chartrand and Bargh, 1999;Driver et al, 1999;Tomasello, 2000;Moore and D'Entremont, 2001;Sebanz et al, 2003Sebanz et al, , 2006Reed et al, 2006;Warneken and Tomasello, 2006;Ganesh et al, 2014;Sawers et al, 2017;Mojtahedi et al, 2017a,b Joint action in human robot interaction Breazeal et al, 2005;Lawitzky et al, 2010;Wang et al, 2013;Magrini et al, 2015;Rozo et al, 2015;Li and Zhang, 2017 Implicit communication in robot-robot applications Two robots: Aiyama et al, 1999;Pereira et al, 2002 Multiple robots: Martinoli and Easton, 2003;Groß and Dorigo, 2004;Groß et al, 2006;Ducatelle et al, 2011;Winfield and Erbas, 2011;Wang and Schwager, 2016 Work exploring the use of explicit communication in robotic systems has been omitted, as explicit communication protocols are so widespread that they are fundamental to most robotics research.…”

Section: Category Publicationsmentioning

confidence: 99%

The Need for Combining Implicit and Explicit Communication in Cooperative Robotic Systems

et al. 2018

View full text Add to dashboard Cite

As the number of robots used in warehouses and manufacturing increases, so too does the need for robots to be able to manipulate objects, not only independently, but also in collaboration with humans and other robots. Our ability to effectively coordinate our actions with fellow humans encompasses several behaviours that are collectively referred to as joint action, and has inspired advances in human-robot interaction by leveraging our natural ability to interpret implicit cues. However, our capacity to efficiently coordinate on object manipulation tasks remains an advantageous process that is yet to be fully exploited in robotic applications. Humans achieve this form of coordination by combining implicit communication (where information is inferred) and explicit communication (direct communication through an established channel) in varying degrees according to the task at hand. Although these two forms of communication have previously been implemented in robotic systems, no system exists that integrates the two in a task-dependent adaptive manner. In this paper, we review existing work on joint action in human-robot interaction, and analyse the state-of-the-art in robot-robot interaction that could act as a foundation for future cooperative object manipulation approaches. We identify key mechanisms that must be developed in order for robots to collaborate more effectively, with other robots and humans, on object manipulation tasks in shared autonomy spaces.

show abstract

Implicit Intention Communication in Human–Robot Interaction Through Visual Behavior Studies

Cited by 61 publications

References 47 publications

Toward Shared Autonomy Control Schemes for Human-Robot Systems: Action Primitive Recognition Using Eye Gaze Features

Toward Shared Autonomy Control Schemes for Human-Robot Systems: Action Primitive Recognition Using Eye Gaze Features

Evaluation of Eye-Movement Metrics ina Software DebbugingTask using GP3 Eye Tracker

The Need for Combining Implicit and Explicit Communication in Cooperative Robotic Systems

Contact Info

Product

Resources

About