Examining the role of statistical and linguistic knowledge sources in a general-knowledge question-answering system

Abstract

We describe and evaluate an implemented system for general-knowledge question answering. The system combines techniques for standard ad-hoc information retrieval (IR), query-dependent text summarization, and shallow syntactic and semantic sentence analysis. In a series of experiments we examine the role of each statistical and linguistic knowledge source in the question-answering system. In contrast to previous results, we find first that statistical knowledge of word co-occurrences as computed by IR vector space methods can be used to quickly and accurately locate the relevant documents for each question. The use of query-dependent text summarization techniques, however, provides only small increases in performance and severely limits recall levels when inaccurate. Nevertheless, it is the text summarization component that allows subsequent linguistic filters to focus on relevant passages. We find that even very weak linguistic knowledge can offer substantial improvements over purely IRbased techniques for question answering, especially when smoothly integrated with statistical preferences computed by the IR subsystems.

Finding support sentences for entities

“…They filter out sentences which do not contain specific entity-patterns in the result set and re-rank sentences accordingly. Sentence retrieval has also been employed to assist Question Answering systems (QAs); the motivation is to select a small set of sentences which may contain the answer to a given question and employ QA strategies to them instead to whole documents [3]. None of these works deal with the issue of explaining a query-entity relationship.…”

“…There is an important body of work on sentence retrieval (see the book from Murdock [13], and references therein). Research in sentence retrieval has been driven by two main topics: relevance retrieval and novelty detection, both mainly geared towards text summarization, question answering [3], topic detection and tracking [24] or a combination of any of them [25]. Li and Croft [9] employ named entity recognition techniques to improve novelty detection in sentence retrieval.…”

“…Note that some support sentences have partial or no matches with the query terms. There are several reasons why this might happen: the usual synonymy or anaphora problems in IR (as in (3), where Picasso is referred to as "the author"), or more complex sentences requiring some domain knowledge and inference (as in (5)). …”

Robustness beyond shallowness: incremental deep parsing

“…In practice, the availability of huge amounts of electronic text documents raised the need for applications capable of processing those documents automatically, hence capable of parsing real-world texts. The research efforts in that direction have already brought robust parsing systems that successfully contributed either to the automatic acquisition of linguistic resources (Grefenstette 1992;Hindle 1994;Briscoe and Carroll 1997;Faure and Nédellec 1999) or to various NLP applications, such as information extraction (Appelt et al 1993;Hobbs et al 1996;Grishman 1995;Proux et al 2000), translation memory (Gaussier et al 2000) and question answering (Cardie et al 2000;Moldovan et al 2000). There are many references to 'robustness' in the literature, but little agreement on the exact definition of that notion (Ballim et al 1999;Menzel 1995).…”

Automated Question Answering: Review of the Main Approaches

“…SMART is still used in modern systems such as the ones described in [4] and [5]. FAQ Finder [4] is among the first systems for retrieving Frequently Asked Questions (FAQ) from text files.…”