Multimodal Joke Generation and Paralinguistic Personalization for a Socially-Aware Robot

Ritschel, Hannes; Kiderle, Thomas; Weber, Klaus; Lingenfelser, Florian; Baur, Tobias; André, Elisabeth

doi:10.1007/978-3-030-49778-1_22

Cited by 8 publications

(4 citation statements)

References 30 publications

(29 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, it cannot capture human aspects, such as the user's behavior, personality, or mood. Thus, interaction distance, gaze and smile [Fournier et al 2017, Gordon et al 2016, Hemminghaus and Kopp 2017, Leite et al 2011, motion speed, timing [Mitsunaga et al 2008], gesture and posture [Najar et al 2016, and laughter [Hayashi et al 2008, Katevas et al 2015, Knight 2011, Ritschel et al 2020a, Weber et al 2018 are used in various contexts as feedback for social agents. Physiological feedback includes ECG [Liu et al 2008] or EEG [Tsiakas et al 2018] data.…”

Section: Reinforcement Learningmentioning

confidence: 99%

“…These signals are often aggregated and combined in order to build a user model and calculate reward, e.g. based on the human's estimated affect/emotions [Broekens and Chetouani 2019, Gordon et al 2016, Leite et al 2011, engagement [Mancini et al 2019, Ritschel 2018, curiosity [Fournier et al 2017], amusement , Ritschel et al 2020a, Weber et al 2018] and more.…”

Section: Reinforcement Learningmentioning

confidence: 99%

See 1 more Smart Citation

Adaptive Artificial Personalities

Janowski¹,

Ritschel²,

André³

2022

The Handbook on Socially Interactive Agents

View full text Add to dashboard Cite

Section: Reinforcement Learningmentioning

confidence: 99%

Section: Reinforcement Learningmentioning

confidence: 99%

Adaptive Artificial Personalities

Janowski¹,

Ritschel²,

André³

2022

The Handbook on Socially Interactive Agents

View full text Add to dashboard Cite

“…However, non-verbal expression is key to understand sociability [18,19]. Some authors working with virtual agents and computer graphics have obtained impressive realistic animations of human characters.…”

Section: Emotion Expression In Robotsmentioning

confidence: 99%

Expressing Robot Personality through Talking Body Language

et al. 2021

View full text Add to dashboard Cite

Social robots must master the nuances of human communication as a mean to convey an effective message and generate trust. It is well-known that non-verbal cues are very important in human interactions, and therefore a social robot should produce a body language coherent with its discourse. In this work, we report on a system that endows a humanoid robot with the ability to adapt its body language according to the sentiment of its speech. A combination of talking beat gestures with emotional cues such as eye lightings, body posture of voice intonation and volume permits a rich variety of behaviors. The developed approach is not purely reactive, and it easily allows to assign a kind of personality to the robot. We present several videos with the robot in two different scenarios, and showing discrete and histrionic personalities.

show abstract

“…In computer science and, especially, affective computing, the automatic measurement of humour has attracted increasing research interest in recent years, e. g. [11,12,13]. In particular, humour has been identified as important in human-computer interaction [14,15]. Because of the ubiquity of humour 1 Lukas Christ, Shahin Amiriparian, Alexander Kathan, and Björn Schuller are with the Chair of Embedded Intelligence for Health Care and Wellbeing, University of Augsburg, Germany.…”

Section: Introductionmentioning

confidence: 99%

Multimodal Prediction of Spontaneous Humour: A Novel Dataset and First Results

Christ¹,

Amiriparian²,

Kathan³

et al. 2022

Preprint

View full text Add to dashboard Cite

Humour is a substantial element of human affect and cognition. Its automatic understanding can facilitate a more naturalistic human-device interaction and the humanisation of artificial intelligence. Current methods of humour detection are solely based on staged data making them inadequate for 'real-world' applications. We address this deficiency by introducing the novel Passau-Spontaneous Football Coach Humour (Passau-SFCH) dataset, comprising of about 11 hours of recordings. The Passau-SFCH dataset is annotated for the presence of humour and its dimensions (sentiment and direction) as proposed in Martin's Humor Style Questionnaire. We conduct a series of experiments, employing pretrained Transformers, convolutional neural networks, and expert-designed features. The performance of each modality (text, audio, video) for spontaneous humour recognition is analysed and their complementarity is investigated. Our findings suggest that for the automatic analysis of humour and its sentiment, facial expressions are most promising, while humour direction can be best modelled via text-based features. The results reveal considerable differences among various subjects, highlighting the individuality of humour usage and style. Further, we observe that a decision-level fusion yields the best recognition result. Finally, we make our code publicly available at https://www.github.com/EIHW/passau-sfch. The Passau-SFCH dataset is available upon request.

show abstract

Multimodal Joke Generation and Paralinguistic Personalization for a Socially-Aware Robot

Cited by 8 publications

References 30 publications

Adaptive Artificial Personalities

Adaptive Artificial Personalities

Expressing Robot Personality through Talking Body Language

Multimodal Prediction of Spontaneous Humour: A Novel Dataset and First Results

Contact Info

Product

Resources

About