Multi-turn Dialogue Model Based on the Improved Hierarchical Recurrent Attention Network

Abstract

When considering the multi-turn dialogue systems, the model needs to generate a natural and contextual response. At present, HRAN, one of the most advanced models for multi-turn dialogue problems, uses a hierarchical recurrent encoder-decoder combined with a hierarchical attention mechanism. However, for complex conversations, the traditional attention-based RNN does not fully understand the context, which results in attention to the wrong context that generates irrelevant responses. To solve this problem, we proposed an improved hierarchical recurrent attention network, a self-attention network (HSAN), instead of RNN, to learn word representations and utterances representations. Empirical studies on both Chinese and English datasets show that the proposed model has achieved significant improvement.

Untitled

“…With the rapid development of deep learning, the use of pre-training models and attention mechanisms in natural language processing entered a new era [10]. Existing news text classification models based on deep learning train only content texts, news title [11] or news title and content texts together for training, these three training methods all ignore the correlation between the news title and the semantic level of the news content, which affects the accuracy of news category judgment.…”