Sequential neural networks for noetic end-to-end response selection

Abstract

The noetic end-to-end response selection challenge as one track in the 7th Dialog System Technology Challenges (DSTC7) aims to push the state of the art of utterance classification for real world goal-oriented dialog systems, for which participants need to select the correct next utterances from a set of candidates for the multi-turn context. This paper presents our systems that are ranked top 1 on both datasets under this challenge, one focused and small (Advising) and the other more diverse and large (Ubuntu). Previous state-of-the-art models use hierarchybased (utterance-level and token-level) neural networks to explicitly model the interactions among different turns' utterances for context modeling. In this paper, we investigate a sequential matching model based only on chain sequence for multi-turn response selection. Our results demonstrate that the potentials of sequential matching approaches have not yet been fully exploited in the past for multi-turn response selection. In addition to ranking top 1 in the challenge, the proposed model outperforms all previous models, including state-of-the-art hierarchy-based models, on two large-scale public multi-turn response selection benchmark datasets. Keywords:DSTC7, response selection, ESIM, BERT, end-to-end, sequential matching approaches 1. We develop an Enhanced Sequential Inference Model (ESIM) based system for the DSTC7 noetic end-to-end response selection track. On top of the ESIM model, we explore methods for exploiting multiple word embeddings, heuristic data augmentation, tuning the ratio between positive and negative samples, and emphasizing the importance of the most recent context utterances. 2. We propose a two-step approach for selecting the next utterance from a large amount of candidates (i.e., for subtask 2 on the Ubuntu dataset, we need to select the next utterance from a candidate pool of 120,000 sentences), by first using a sentence-encoding based method to select the top N candidates from the large set of candidates and then reranking them using ESIM, achieving a high performance with an acceptable overall computational cost. 3. We conduct systematic ablation analysis of the above-mentioned methods for enhancing the ESIM model performance. In particular, we develop effective and efficient model ensemble by averaging the output from models

Towards Persona-Based Empathetic Conversational Models

“…ally have three modules: encoding, matching and aggregation (Lowe et al, 2015;Zhou et al, 2016;Wu et al, 2017;Zhou et al, 2018b;Zhang et al, 2018b;Chen and Wang, 2019;Feng et al, 2019;Yuan et al, 2019). The encoding module encodes text into vector representations using encoders such as LSTM, Transformer, or BERT.…”

Label-Efficient Training for Next Response Selection

“…Advising-3 DailyDialog Train Data MRR R@1 R@10 MAP R@1 R@10 MAP R@1 R@10 Oracle ESIM (Chen and Wang, 2019) 0…”

Multi-Turn Response Selection for Chatbots With Hierarchical Aggregation Network of Multi-Representation

“…Besides interaction representation for utterances, we consider the response as a part of the context, and then encode the response with utterances. We first concatenate all utterances as the context [20], i.e. C = [U 1 , .…”

“…After generating the interaction representations U k and R, we further enhance the matching information by calculating the difference, subtraction [21], [22] and multiplication [20], [23], [24] with self-aggregated representations U k and R. The various matching information of utterances and response are fused into the final aggregated representation using the matching aggregation mechanism, which is formulated as:…”