End-to-End Neural Ad-hoc Ranking with Kernel Pooling

Abstract

This paper proposes K-NRM, a kernel based neural model for document ranking. Given a query and a set of documents, K-NRM uses a translation matrix that models word-level similarities via word embeddings, a new kernel-pooling technique that uses kernels to extract multi-level soft match features, and a learning-to-rank layer that combines those features into the final ranking score. The whole model is trained end-to-end. The ranking layer learns desired feature patterns from the pairwise ranking loss. The kernels transfer the feature patterns into soft-match targets at each similarity level and enforce them on the translation matrix. The word embeddings are tuned accordingly so that they can produce the desired soft matches. Experiments on a commercial search engine's query log demonstrate the improvements of K-NRM over prior feature-based and neural-based states-of-the-art, and explain the source of K-NRM's advantage: Its kernel-guided embedding encodes a similarity metric tailored for matching query words to document words, and provides effective multi-level soft matches

Learning Contextualized Document Representations for Healthcare Answer Retrieval

“…Additionally, we evaluate the following document matching methods from the literature: ARC-I and ARC-II [21], DSSM [22], C-DSSM [44], DRMM [17], MatchPyramid [34], aNMM [49], Duet [30], MVLSTM [46], KNRM [48], CONV-KNRM [12] and HAR [52]. For implementing these models, we followed Zhu et al [52] and used the open source implementation MatchZoo [18] with pre-trained glove.840B.300d vectors [35].…”

“…Interaction-based matching models focus on the complex interaction between query and passage. These models use CNNs on sentence level (ARC-II [21]), match query terms and words using word count histograms (DRMM [17]), word-level dot product similarity (MatchPyramid [34]), attention-based neural networks (aNMM [49]), kernel pooling (K-NRM [48]) or convolutional n-gram kernel pooling (Conv-KNRM [12]). Eventually, Zhu et al [52] utilize hierarchical attention on word and sentence level (HAR) to capture interaction of the query with local context in long passages.…”

Ranking Significant Discrepancies in Clinical Reports

“…We call this variant BiPACRR. We also experimented with other neural rankers (e.g., KNRM [20]), but BiPACRR was the most effective. -Textual similarity regression (SimReg) [2].…”

Mining Archive.org’s Twitter Stream Grab for Pharmacovigilance Research Gold

“…For classification task, we experimented with six deep learning techniques from the Matchzoo framework 23 : MVLSTM, DUET, KNRM, CONVKNRM, DSSM, and ARC-II [24][25][26][27][28][29] . DUET is applied for the document ranking task and is composed of two separate deep neural networks, one matches the query and the document using a local representation, and another matches the query and the document using learned distributed representations.…”