Entrainable Neural Conversation Model Based on Reinforcement Learning

Abstract

The synchronization of words in conversation, called entrainment, is generally observed in human-human conversations. Entrainment has a high correlation with dialogue success, naturalness, and engagement. In this paper, we define entrainment scores based on the word similarities in semantic space to evaluate the entrainment of system generation. We optimized a neural conversation model to the entrainment scores using reinforcement learning so that the system can control the degree of entrainment of the system response. Experimental results showed that the proposed entrainable neural conversation model generated comparable or more natural responses than conventional models and satisfactorily controlled the degree of entrainment of the generated responses.

Analysis of Style-Shifting on Social Media: Using Neural Language Model Conditioned by Social Meanings

“…The second perspective is the relative change in language style compared with their usual conversation. These aspects are often under-emphasized in existing studies that rely on vocabulary comparisons between speakers within the conversation (Nasir et al, 2019;Kawano et al, 2020;Brugnoli et al, 2019). However, our approach takes both of these perspectives into consideration, enabling us to gain a deeper understanding of style-shifting by assessing stylistic alterations in relation to our own usual conversation.…”

“…First, we investigated the correlation between the metrics s sim and s change proposed in Section 3 and the human evaluation in Section 4.1. As the baseline based on the word-based similarity, we introduced a score using Word Mover's Distance (WMD) (Kusner et al, 2015;Nasir et al, 2019;Kawano et al, 2020). We designed two intuitive baseline scores, s wmd s im and s wmd change corresponding to s s im and s change .…”

“…Research on stylistic variation has underscored the intrinsic properties of conversational styles and their interplay with various non-linguistic factors (Kruijff-Korbayová et al, 2008;Kang and Hovy, 2021;Basile et al, 2019;Flekova et al, 2016). Furthermore, many studies have quantified the changes in a speaker's language style in a conversation based on the similarity in word choice between utterances (Nenkova et al, 2008;Danescu-Niculescu-Mizil et al, 2011;Ireland et al, 2011;Nasir et al, 2019;Kawano et al, 2020;Brugnoli et al, 2019). However, despite these insights, such word-based methods still face challenges in capturing nuanced changes in style that reflect various social meanings.…”

A Neural Network-Based Linguistic Similarity Measure for Entrainment in Conversations

“…To evaluate entrainment, early studies often set experimental conditions or control groups (Brennan and Clark, 1996;Branigan et al, 2000;Garrod and Anderson, 1987). In the later corpus-based studies, evaluations are mostly extrinsic such as comparing entrainment between conversational partners and non-partners (Levitan and Hirschberg, 2011;Rahimi et al, 2017), associating entrainment to other interpersonal behav-iors in dialogue such as group relationships (Yu et al, 2019), positive or negative effects (Nasir et al, 2019), being liked by partners (Levitan et al, 2012), and dialogue success (Kawano et al, 2020).…”

“…There has been a large amount of research that studies this phenomenon in a wide range of linguistic dimensions such as acoustic and prosodic (Levitan et al, 2012;Litman et al, 2016), lexical , and syntactical (Branigan et al, 2000). It has long been an interest of dialogue studies because entrainment has been found to associate with various social outcomes such as group relationship (Yu et al, 2019), positive or negative effect (Nasir et al, 2019), being liked by partners (Levitan et al, 2012), and dialogue success (Kawano et al, 2020;Xu and Reitter, 2017). The characteristics make entrainment a valuable tool to build a human-like dialogue system.…”