Multimodal Interaction History and its use in Error Detection and Recovery

Schüssel, Felix; Honold, Frank; Schmidt, Manfred; Bubalo, Nikola; Huckauf, Anke; Weber, Michael

doi:10.1145/2663204.2663255

Cited by 20 publications

(2 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A scenario that fits to these requirements should be clearly related to computer interaction and be complex enough to involve the subject into the task. Sometimes, monetary rewards are offered to the subjects to motivate them to participate in the experiments (Schüssel et al, 2014). This practice is often successful as the amount of rewarded money might be dependent on the performance of the subject.…”

Section: Involvement Of Test Subjectsmentioning

confidence: 99%

The Influence of Annotation, Corpus Design, and Evaluation on the Outcome of Automatic Classification of Human Emotions

2016

View full text Add to dashboard Cite

The integration of emotions into human-computer interaction applications promises a more natural dialog between the user and the technical system operators. In order to construct such machinery, continuous measuring of the affective state of the user becomes essential. While basic research that is aimed to capture and classify affective signals has progressed, many issues are still prevailing that hinder easy integration of affective signals into human-computer interaction. In this paper, we identify and investigate pitfalls in three steps of the work-flow of affective classification studies. It starts with the process of collecting affective data for the purpose of training suitable classifiers. Emotional data have to be created in which the target emotions are present. Therefore, human participants have to be stimulated suitably. We discuss the nature of these stimuli, their relevance to human-computer interaction, and the repeatability of the data recording setting. Second, aspects of annotation procedures are investigated, which include the variances of individual raters, annotation delay, the impact of the used annotation tool, and how individual ratings are combined to a unified label. Finally, the evaluation protocol is examined, which includes, among others, the impact of the performance measure on the accuracy of a classification model. We hereby focus especially on the evaluation of classifier outputs against continuously annotated dimensions. Together with the discussed problems and pitfalls and the ways how they affect the outcome, we provide solutions and alternatives to overcome these issues. As the final part of the paper, we sketch a recording scenario and a set of supporting technologies that can contribute to solve many of the issues mentioned above.

show abstract

Section: Involvement Of Test Subjectsmentioning

confidence: 99%

The Influence of Annotation, Corpus Design, and Evaluation on the Outcome of Automatic Classification of Human Emotions

2016

View full text Add to dashboard Cite

show abstract

“…In this work the newly recorded Interaction Game dataset as introduced in [27] was used. This data consists of a set of video sequences with a length of about 30 minutes each, depicting an experiment involving a participant who interacts both through speech and touch input with a multimodal system.…”

Section: Dataset and Annotationmentioning

confidence: 99%

Detection of Emotional Events utilizing Support Vector Methods in an Active Learning HCI Scenario

Thiam

Meudt

Kächele

et al. 2014

Proceedings of the 2014 Workshop on Emotion Representation and Modelling in Human-Computer-Interaction-Systems

View full text Add to dashboard Cite

In recent years the fields of affective computing and emotion recognition have experienced a steady increase in attention and especially the creation and analysis of multi-modal corpora has been the focus of intense research. Plausible annotation of this data, however is an enormous problem. In detail emotion annotation is very time consuming, cumbersome and sensitive with respect to the annotator. Furthermore emotional reactions are often very sparse in HCI scenarios resulting in a large annotation overhead to gather the interesting moments of a recording, which in turn are highly relevant for powerful features, classifiers and fusion architectures. Active learning techniques provide methods to improve the annotation processes since the annotator is asked to only label the relevant instances of a given dataset. In this work an unsupervised one-class Support Vector Machine is used to build a background model of non-emotional sequences on a novel HCI dataset. The human annotator is iteratively asked to label instances that are not well explained by the background model, which in turn renders them candidates for being interesting events such as emotional reactions that diverge from the norm. The outcome of the active learning procedure is a reduced dataset of only 14% the size of the original dataset that contains most of the significant information, in this case more than 75% of the emotional events.

show abstract

On Annotation and Evaluation of Multi-modal Corpora in Affective Human-Computer Interaction

Kächele

Schels

Meudt

et al. 2015

Multimodal Analyses Enabling Artificial Agents in Human-Machine Interaction

View full text Add to dashboard Cite

Multimodal Interaction History and its use in Error Detection and Recovery

Cited by 20 publications

References 11 publications

The Influence of Annotation, Corpus Design, and Evaluation on the Outcome of Automatic Classification of Human Emotions

The Influence of Annotation, Corpus Design, and Evaluation on the Outcome of Automatic Classification of Human Emotions

Detection of Emotional Events utilizing Support Vector Methods in an Active Learning HCI Scenario

On Annotation and Evaluation of Multi-modal Corpora in Affective Human-Computer Interaction

Contact Info

Product

Resources

About