Grafting Fine-Tuning and Reinforcement Learning for Empathetic Emotion Elicitation in Dialog Generation

Abstract

For human-like dialogue systems, it is significant to inject the empathetic ability or elicit the opposite’s positive emotions, while existing studies mostly only focus on either of the above two research lines. In this work, we propose a novel and grafted task named Empathetic Emotion Elicitation Dialog to make a dialog system able to possess both aspects of ability simultaneously. We do not train an empathetic dialog system and an emotion elicitation dialog system separately and then simply concatenate the responses generated by these two systems, which will cause illogical and repetitive responses. Instead, we propose a unified solution: (1) To generate empathetic responses and emotion elicitation responses within the same semantic space, we design a unified framework. (2) The unified framework has three stages which first retrieve the empathetic and emotion elicitation exemplars as external knowledge, then fine-tune the emotion/action prediction on a pre-trained language model to enhance the empathetic ability, and finally model the user feedback by reinforcement learning to enhance the emotion elicitation ability. Experiments show that our method outperforms the baselines in the response generation quality and simultaneously empathizes with the user and elicits their positive emotions.