Jack the Reader – A Machine Reading Framework

Weissenborn, Dirk; Minervini, Pasquale; Augenstein, Isabelle; Welbl, Johannes; Rocktäschel, Tim; Bošnjak, Matko; Mitchell, Jeff; Demeester, Thomas; Dettmers, Tim; Stenetorp, Pontus; Riedel, Sebastian

doi:10.18653/v1/p18-4005

Cited by 7 publications

(6 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also experimented with the Decomposable Attention Model (DAM) (Parikh et al, 2016) -as used in the baseline model, however ESIM consistently performed better. The Jack the Reader (Weissenborn et al, 2018) framework was used for both DAM and ESIM. We first pre-trained the ESIM model on the Stanford Natural Language Inference (SNLI) corpus (Bowman et al, 2015), and then fine-tuned on the FEVER dataset.…”

Section: Natural Language Inference (Nli)mentioning

confidence: 99%

UCL Machine Reading Group: Four Factor Framework For Fact Finding (HexaF)

Yoneda¹,

Mitchell²,

Welbl³

et al. 2018

Proceedings of the First Workshop on Fact Extraction and VERification (FEVER)

Self Cite

122

View full text Add to dashboard Cite

In this paper we describe our 2 nd place FEVER shared-task system that achieved a FEVER score of 62.52% on the provisional test set (without additional human evaluation), and 65.41% on the development set. Our system is a four stage model consisting of document retrieval, sentence retrieval, natural language inference and aggregation. Retrieval is performed leveraging task-specific features, and then a natural language inference model takes each of the retrieved sentences paired with the claimed fact. The resulting predictions are aggregated across retrieved sentences with a Multi-Layer Perceptron, and re-ranked corresponding to the final prediction.

show abstract

Section: Natural Language Inference (Nli)mentioning

confidence: 99%

UCL Machine Reading Group: Four Factor Framework For Fact Finding (HexaF)

Yoneda¹,

Mitchell²,

Welbl³

et al. 2018

Proceedings of the First Workshop on Fact Extraction and VERification (FEVER)

Self Cite

122

View full text Add to dashboard Cite

show abstract

“…Strictly speaking, almost any NLP task can be formulated as question answering, and this is already being leveraged for model reuse and multi-task learning [e.g. 168,271] and zero-shot learning [e.g. 1,143].…”

Section: Task Versus Formatmentioning

confidence: 99%

QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension

Rogers¹,

Gardner²,

Augenstein³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Alongside huge volumes of research on deep learning models in NLP in the recent years, there has been also much work on benchmark datasets needed to track modeling progress. Question answering and reading comprehension have been particularly prolific in this regard, with over 80 new datasets appearing in the past two years. This study is the largest survey of the field to date. We provide an overview of the various formats and domains of the current resources, highlighting the current lacunae for future work. We further discuss the current classifications of "reasoning types" in question answering and propose a new taxonomy. We also discuss the implications of over-focusing on English,

show abstract

“…Following Welbl et al (2017), we use two neural QA models, namely BIDAF (Seo et al, 2016a) and FASTQA (Weissenborn et al, 2017b), as baselines for the considered WIKIHOP predicates. We use the implementation provided by the JACK 2 QA framework (Weissenborn et al, 2018) with the same hyperparameters as used by Welbl et al (2017), and train a separate model for each predicate. 3 To ensure that the performance of the baseline is not adversely affected by the relatively small number of training examples, we also evaluate the BIDAF model trained on the whole WIK-IHOP corpus.…”

Section: Baselinesmentioning

confidence: 99%

NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language

Weber¹,

Minervini²,

Münchmeyer³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

Self Cite

View full text Add to dashboard Cite

Rule-based models are attractive for various tasks because they inherently lead to interpretable and explainable decisions and can easily incorporate prior knowledge. However, such systems are difficult to apply to problems involving natural language, due to its linguistic variability. In contrast, neural models can cope very well with ambiguity by learning distributed representations of words and their composition from data, but lead to models that are difficult to interpret. In this paper, we describe a model combining neural networks with logic programming in a novel manner for solving multi-hop reasoning tasks over natural language. Specifically, we propose to use a Prolog prover which we extend to utilize a similarity function over pretrained sentence encoders. We fine-tune the representations for the similarity function via backpropagation. This leads to a system that can apply rulebased reasoning to natural language, and induce domain-specific rules from training data. We evaluate the proposed system on two different question answering tasks, showing that it outperforms two baselines -BIDAF (Seo et al., 2016a) and FASTQA (Weissenborn et al., 2017b) on a subset of the WIKIHOP corpus and achieves competitive results on the MEDHOP data set (Welbl et al., 2017).

show abstract

Jack the Reader – A Machine Reading Framework

Cited by 7 publications

References 18 publications

UCL Machine Reading Group: Four Factor Framework For Fact Finding (HexaF)

UCL Machine Reading Group: Four Factor Framework For Fact Finding (HexaF)

QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension

NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language

Contact Info

Product

Resources

About