Focus-Constrained Attention Mechanism for CVAE-based Response Generation

Abstract

To model diverse responses for a given post, one promising way is to introduce a latent variable into Seq2Seq models. The latent variable is supposed to capture the discourse-level information and encourage the informativeness of target responses. However, such discourselevel information is often too coarse for the decoder to be utilized. To tackle it, our idea is to transform the coarse-grained discourselevel information into fine-grained word-level information. Specifically, we firstly measure the semantic concentration of corresponding target response on the post words by introducing a fine-grained focus signal. Then, we propose a focus-constrained attention mechanism to take full advantage of focus in well aligning the input to the target response. The experimental results demonstrate that by exploiting the fine-grained signal, our model can generate more diverse and informative responses compared with several state-of-the-art models. 1 1 https://github.com/cuizhi555/Focus-Constrained-Attention-Mechanism-for-CVAE-based-Response-Generation.