Starting engagement detection towards a companion robot using multimodal features

Vaufreydaz, Dominique; Johal, Wafa; Combe, Claudine

doi:10.1016/j.robot.2015.01.004

Cited by 53 publications

(40 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The choice of the underlying classification algorithms has been driven mainly by our expertise and previous results on the corpora. Support Vector Machine (SVM) has demonstrated performance in previous tasks [16]. Random Forest (RF) proved their adequacy on the HAART corpus [5].…”

Section: Machine Learning Approachmentioning

confidence: 99%

See 1 more Smart Citation

The Grenoble System for the Social Touch Challenge at ICMI 2015

Johal

Portaz

et al. 2015

Proceedings of the 2015 ACM on International Conference on Multimodal Interaction

Self Cite

View full text Add to dashboard Cite

International audienceNew technologies and especially robotics is going towards more natural user interfaces. Works have been done in different modality of interaction such as sight (visual computing), and audio (speech and audio recognition) but some other modalities are still less researched. The touch modality is one of the less studied in HRI but could be valuable for naturalistic interaction. However touch signals can vary in semantics. It is therefore necessary to be able to recognize touch gestures in order to make human-robot interaction even more natural.We propose a method to recognize touch gestures. This method was developed on the CoST corpus and then directly applied on the HAART dataset as a participation of the Social Touch Challenge at ICMI 2015.Our touch gesture recognition process is detailed in this article to make it reproducible by other research teams.Besides features set description, we manually filtered the training corpus to produce 2 datasets.For the challenge, we submitted 6 different systems.A Support Vector Machine and a Random Forest classifiers for the HAART dataset.For the CoST dataset, the same classifiers are tested in two conditions: using all or filtered training datasets.As reported by organizers, our systems have the best correct rate in this year's challenge (70.91% on HAART, 61.34% on CoST).Our performances are slightly better that other participants but stay under previous reported state-of-the-art results

show abstract

Section: Machine Learning Approachmentioning

confidence: 99%

“…In previous researches in multimodal perception [16], we successfully applied a feature reduction technique from bioinformatics research domain. The Minimum Redundancy Maximum Relevance [23] (MRMR) has the advantage of giving the more relevant features instead of building new features from the observed ones.…”

Section: Feature Selectionmentioning

confidence: 99%

The Grenoble System for the Social Touch Challenge at ICMI 2015

Johal

Portaz

et al. 2015

Proceedings of the 2015 ACM on International Conference on Multimodal Interaction

Self Cite

View full text Add to dashboard Cite

show abstract

“…Among all the works discussed so far, there are some that not only deal with intention recognition in human-robot interaction but especially address elderly assistance, which is of particular interest in this work. Vaufreydaz et al [10] worked on the automatic detection of the intention to interact with a robot. Considered data were face size and position, speech, shoulder pose rotation and movement speed.…”

Section: Related Workmentioning

confidence: 99%

Multimodal Uncertainty Reduction for Intention Recognition in Human-Robot Interaction

Trick

Koert

Peters

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Assistive robots can potentially improve the quality of life and personal independence of elderly people by supporting everyday life activities. To guarantee a safe and intuitive interaction between human and robot, human intentions need to be recognized automatically. As humans communicate their intentions multimodally, the use of multiple modalities for intention recognition may not just increase the robustness against failure of individual modalities but especially reduce the uncertainty about the intention to be predicted. This is desirable as particularly in direct interaction between robots and potentially vulnerable humans a minimal uncertainty about the situation as well as knowledge about this actual uncertainty is necessary. Thus, in contrast to existing methods, in this work a new approach for multimodal intention recognition is introduced that focuses on uncertainty reduction through classifier fusion. For the four considered modalities speech, gestures, gaze directions and scene objects individual intention classifiers are trained, of which all output a probability distribution over all possible intentions. By combining these output distributions using the Bayesian method Independent Opinion Pool [1] the uncertainty about the intention to be predicted can be decreased. The approach is evaluated in a collaborative human-robot interaction task with a 7-DoF robot arm. The results show that fused classifiers which combine multiple modalities outperform the respective individual base classifiers with respect to increased accuracy, robustness, and reduced uncertainty.

show abstract

“…Behavioral measures, such as engagement [21][22][23] or proxemics as a proxy for immediacy [24][25][26], can be computed via computer vision techniques. Some studies have focused on measuring attention of the user during the interaction.…”

Section: According Tomentioning

confidence: 99%

Bridging Multilevel Time Scales in HRI

Asselborn

Sharma

Johal

et al. 2019

J. Hum.-Robot Interact.

Self Cite

View full text Add to dashboard Cite

In this article, we present a multi-level time scales framework for the analysis of human-robot interaction (HRI). Such a framework allows HRI scientists to model the interrelation between measures and factors of an experiment. Our final goal with the introduction of this framework is to unify scientific practice in the HRI community for better reproducibility. Our new approach transposes Newell's framework of human actions to model human-robot interaction. Measures from the interaction are sorted into categories (time scales) corresponding to the temporal constraints proposed by Newell. According to this sorting, a bottom-up or top-down analysis can then be performed to correlate variables which allows a better understanding and explanation of the interaction. The utilization of our method within two experimental use cases is then presented. The first one, a child-robot interaction, involves two robots and one child playing a memory game. The second is based on an analysis of the PInSoRo dataset, involving 30 child-robot pairs in a freeplay interaction. Finally, we introduce clear guidelines to re-use the framework. CCS Concepts: • Human-centered computing → Human computer interaction (HCI); Empirical studies in HCI; • Computer systems organization → Robotics;

show abstract

Starting engagement detection towards a companion robot using multimodal features

Cited by 53 publications

References 33 publications

The Grenoble System for the Social Touch Challenge at ICMI 2015

The Grenoble System for the Social Touch Challenge at ICMI 2015

Multimodal Uncertainty Reduction for Intention Recognition in Human-Robot Interaction

Bridging Multilevel Time Scales in HRI

Contact Info

Product

Resources

About