No Need to Pay Attention: Simple Recurrent Neural Networks Work!

Abstract

First-order factoid question answering assumes that the question can be answered by a single fact in a knowledge base (KB). While this does not seem like a challenging task, many recent attempts that apply either complex linguistic reasoning or deep neural networks achieve 65%-76% accuracy on benchmark sets. Our approach formulates the task as two machine learning problems: detecting the entities in the question, and classifying the question as one of the relation types in the KB. We train a recurrent neural network to solve each problem. On the SimpleQuestions dataset, our approach yields substantial improvements over previously published results -even neural networks based on much more complex architectures. The simplicity of our approach also has practical advantages, such as efficiency and modularity, that are valuable especially in an industry setting. In fact, we present a preliminary analysis of the performance of our model on real queries from Comcast's X1 entertainment platform with millions of users every day.

Strong Baselines for Simple Question Answering over Knowledge Graphs with and without Neural Networks

“…Results of relation prediction are shown in Table 2 on the validation set. Ture and Jojic (2017) conducted the same component-level evaluation, the results of which we report (but none else that Dai et al (2016) 75.7 Yin et al (2016) 76.4 Yu et al (2017) 77.0 Ture and Jojic (2017) 86.8 Table 3: End-to-end answer accuracy on the test set with different model combinations, compared to a selection of previous results reported in the literature.…”

“…In this push toward complexity, we do not believe that researchers have adequately explored baselines, and thus it is unclear how much various NN techniques actually help. To this end, our work builds on Ture and Jojic (2017), who adopted a straightforward problem decomposition with simple NN models to argue that attentionbased mechanisms don't really help. We take this one step further and examine techniques that do not involve neural networks.…”

Simple Question Answering with Subgraph Ranking and Joint-Scoring

“…Despite that, our approach outperforms the best baseline by 1.2%. Further, the relation accuracy increases by 7.1% over the fifth row, because restricting the subject substantially confines 2 As noted, Ture and Jojic (2017) reported better performance than us but neither Petrochuk and Zettlemoyer (2018) nor Mohammed et al (2018) the choice of relations. This shows that a sufficiently high top-1 subgraph recall reduces the need for subject matching.…”

Pretrained Transformers for Simple Question Answering over Knowledge Graphs

“…For these examples, we set the accuracy and F1 to zero. 15 [19] is not included in the comparison because neither [13] or [16] could reproduce the reported results (86.8%).…”