A Biological Inspired Cognitive Framework for Memory-Based Multi-Sensory Joint Attention in Human-Robot Interactive Tasks

Eldardeer, Omar; Gonzalez-Billandon, Jonas; Grasse, Lukas; Tata, Matthew S.

doi:10.3389/fnbot.2021.648595

Cited by 3 publications

(1 citation statement)

References 50 publications

(58 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The resulting fully integrated system was piloted with 10 volunteers who did not have prior experience with the robot and were not informed about the gaze-based triggering system. We considered a “successful interaction” every iteration in which the participant was able to trigger the joint action at the expected moment (right before/after finishing his/her part) and within a reasonable time (maximum of 5 s after finishing his/her sub-assembly, inspired by the threshold used by Eldardeer et al (2020) ). Once again, the NOVA tool (see Section 3.2.2 ) was used to annotate the frames corresponding to the moment when the participant was done with his/her part of the assembly and the moment when the cobot receives the trigger and starts moving towards its subassembly.…”

Section: Resultsmentioning

confidence: 99%

Gaze detection as a social cue to initiate natural human-robot collaboration in an assembly task

Lavit Nicora,

Prajod,

Mondellini

et al. 2024

Front. Robot. AI

View full text Add to dashboard Cite

Introduction: In this work we explore a potential approach to improve human-robot collaboration experience by adapting cobot behavior based on natural cues from the operator.Methods: Inspired by the literature on human-human interactions, we conducted a wizard-of-oz study to examine whether a gaze towards the cobot can serve as a trigger for initiating joint activities in collaborative sessions. In this study, 37 participants engaged in an assembly task while their gaze behavior was analyzed. We employed a gaze-based attention recognition model to identify when the participants look at the cobot.Results: Our results indicate that in most cases (83.74%), the joint activity is preceded by a gaze towards the cobot. Furthermore, during the entire assembly cycle, the participants tend to look at the cobot mostly around the time of the joint activity. Given the above results, a fully integrated system triggering joint action only when the gaze is directed towards the cobot was piloted with 10 volunteers, of which one characterized by high-functioning Autism Spectrum Disorder. Even though they had never interacted with the robot and did not know about the gaze-based triggering system, most of them successfully collaborated with the cobot and reported a smooth and natural interaction experience.Discussion: To the best of our knowledge, this is the first study to analyze the natural gaze behavior of participants working on a joint activity with a robot during a collaborative assembly task and to attempt the full integration of an automated gaze-based triggering system.

show abstract

Section: Resultsmentioning

confidence: 99%

Gaze detection as a social cue to initiate natural human-robot collaboration in an assembly task

Lavit Nicora,

Prajod,

Mondellini

et al. 2024

Front. Robot. AI

View full text Add to dashboard Cite

show abstract

AEGO: Modeling Attention for HRI in Ego-Sphere Neural Networks

Chame,

Alami

2024

2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

When Deep is not Enough: Towards Understanding Shallow and Continual Learning Models in Realistic Environmental Sound Classification for Robots

Eldardeer

Sandini

Jirak

2023

Int. J. Human. Robot.

View full text Add to dashboard Cite

Although deep learning models are state-of-the-art models in audio classification, they fall short when applied in developmental robotic settings and human–robot interaction (HRI). The major drawback is that deep learning relies on supervised training with a large amount of data and annotations. In contrast, developmental learning strategies in human–robot interaction often deal with small-scale data acquired from HRI experiments and require the incremental addition of novel classes. Alternatively, shallow learning architectures that enable fast and yet robust learning are provided by simple distance metric-based learning and neural architectures implementing the reservoir computing paradigm. Similarly, continual learning algorithms receive more attention in the last years as they can integrate stable perceptual feature extraction using pre-trained deep learning models with open-set classification. As our research centers around reenacting the incremental learning of audio cues, we conducted a study on environmental sound classification using the iCaRL as well as the GDumb continual learning algorithms in comparison with a popular classifier in this domain, the kNN classifier, as well as employing an Echo State Network. We contrast our results with those obtained from a VGGish network that serves here as the performance upper bound that allows us to quantify the performance differences and to discuss current issues with continual learning in the audio domain. As only little is known about using shallow models or continual learning in the audio domain, we pass on additional techniques like data augmentation and create a simple experimental pipeline that is easy to reproduce. Although our selected algorithms are partially inferior in performance compared to the upper bound, our evaluation on three environmental sound datasets shows promising performance using continual learning for a subset of the DCASE2019 challenge dataset and the ESC10 dataset. As we do not address benchmarking in this paper, our study provides a good foundation for further research and computational improvements on shallow and continual learning models for robotic applications in the audio domain.

show abstract

A Biological Inspired Cognitive Framework for Memory-Based Multi-Sensory Joint Attention in Human-Robot Interactive Tasks

Cited by 3 publications

References 50 publications

Gaze detection as a social cue to initiate natural human-robot collaboration in an assembly task

Gaze detection as a social cue to initiate natural human-robot collaboration in an assembly task

AEGO: Modeling Attention for HRI in Ego-Sphere Neural Networks

When Deep is not Enough: Towards Understanding Shallow and Continual Learning Models in Realistic Environmental Sound Classification for Robots

Contact Info

Product

Resources

About