Bilateral Multi-Perspective Matching for Natural Language Sentences

Abstract

Natural language sentence matching is a fundamental technology for a variety of tasks. Previous approaches either match sentences from a single direction or only apply single granular (wordby-word or sentence-by-sentence) matching. In this work, we propose a bilateral multi-perspective matching (BiMPM) model. Given two sentences P and Q, our model first encodes them with a BiL-STM encoder. Next, we match the two encoded sentences in two directions P against Q and Q against P . In each matching direction, each time step of one sentence is matched against all timesteps of the other sentence from multiple perspectives. Then, another BiLSTM layer is utilized to aggregate the matching results into a fixed-length matching vector. Finally, based on the matching vector, a decision is made through a fully connected layer. We evaluate our model on three tasks: paraphrase identification, natural language inference and answer sentence selection. Experimental results on standard benchmark datasets show that our model achieves the state-of-the-art performance on all tasks.

Neural Paraphrase Identification of Questions with Noisy Pretraining

“…Lei et al (2016) consider a related task leveraging the AskUbuntu corpus (dos Santos et al, 2015), but it contains two orders of magnitude less annotations, thus limiting the quality of any model. Most relevant to this work is that of Wang et al (2017), who present the best results on the Quora dataset prior to this work. The bilateral multi-perspective matching model (BIMPM) of Wang et al uses a character-based LSTM (Hochreiter and Schmidhuber, 1997) at its input representation layer, a layer of bi-LSTMs for computing context information, four different types of multi-perspective matching layers, an additional bi-LSTM aggregation layer, followed by a two-layer feedforward network for prediction.…”

Inter-Weighted Alignment Network for Sentence Pair Modeling

“…• (Wang et al, 2016a) 0.734 0.742 BiMPM (Wang et al, 2017) 0 thogonal decomposition (OD) strategy has a superior performance to direct (DI) strategy on all datasets. The comparison results are posted in Table 1.…”

“…Model r ρ MSE Meaning Factory (Jiménez et al, 2014) 0.8268 0.7721 0.3224 ECNU (Zhao et al, 2014) 0.8414 --BiLSTM (Tai et al, 2015) 0.8567 0.7966 0.2736 Tree-LSTM (Tai et al, 2015) 0.8676 0.8083 0.2532 MPCNN (He et al, 2015) 0 Wang and Ittycheriah (2015) 0.746 0.820 QA-LSTM (Tan et al, 2015) 0.728 0.832 Att-pooling (dos Santos et al, 2016) 0.753 0.851 LDC (Wang et al, 2016b) 0.771 0.845 MPCNN (He et al, 2015) 0.777 0.836 PWIM 0.738 0.827 NCE-CNN (Rao et al, 2016) 0.801 0.877 BiMPM (Wang et al, 2017) 0.802 0.875 IWAN-att (Proposed) 0.822 0.889 IWAN-skip (Proposed) 0.801 0.861 Table 3: Test results on Clean version TrecQA.…”

“…combines attention mechanism with tree-structured RecNN encoder. Some prior works (Wang et al, 2016b;Parikh et al, 2016;Wang et al, 2017) compute softalignment representation for each word in sentences attentively with word level similarity and then compose the alignment representations to determine the relation. Our model is also under this framework however we focus on explicitly calculating weights for each word to get more reasonable semantic composition.…”

The Context-Dependent Additive Recurrent Neural Net

“…Baselines. We compare the performance of our models with that of the state-of-the-art models on the clean version of the TREC-QA dataset (Shen et al, 2017;Bian et al, 2017;Wang et al, 2017;Rao et al, 2016;Tay et al, 2017). We do not have access to the original implementation of IWAN, hence we use our implementation of the IWAN model as the basis for our models.…”