Rank learning for factoid question answering with linguistic and semantic constraints

Bilotti, Matthew W.; Elsas, Jonathan L.; Carbonell, Jaime G.; Nyberg, Eric

doi:10.1145/1871437.1871498

Cited by 36 publications

(22 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although learning to rank methods have extensively been applied to web search, the application to QA has been limited. In fact, to the best of our knowledge, Verberne et al ( [35]) is the only work where learning to rank methods have actually been applied to the QA task with the focus on the learning component -unlike others [3,31] where the focus has been on feature generation. Although similar in spirit, Verberne et al only study the behavior of various learning to rank methods on the QA task; while in this work, we not only study this behavior, but also provide a robust multi-stage system that is able to improve over the results obtained by simply applying learning to rank methods.…”

Section: Related Workmentioning

confidence: 99%

Learning to rank for robust question answering

Agarwal

Raghavan

Subbian

et al. 2012

Proceedings of the 21st ACM International Conference on Information and Knowledge Management

View full text Add to dashboard Cite

This paper aims to solve the problem of improving the ranking of answer candidates for factoid based questions in a state-of-the-art Question Answering system. We first provide an extensive comparison of 5 ranking algorithms on two datasets -from the Jeopardy quiz show and a medical domain. We then show the effectiveness of a cascading approach, where the ranking produced by one ranker is used as input to the next stage. The cascading approach shows sizeable gains on both datasets. We finally evaluate several rank aggregation techniques to combine these algorithms, and find that Supervised Kemeny aggregation is a robust technique that always beats the baseline ranking approach used by Watson for the Jeopardy competition. We further corroborate our results on TREC Question Answering datasets.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning to rank for robust question answering

Agarwal

Raghavan

Subbian

et al. 2012

Proceedings of the 21st ACM International Conference on Information and Knowledge Management

View full text Add to dashboard Cite

show abstract

“…Passage retrieval has a long history within Information Retrieval: improving document retrieval [6,26,27,31,36], focusing query expansion [41], extracting explanatory snippets, locating responses for question answering [4,7,18,37], and retrieving appropriate passages [5,36,38,40]. Our study focuses on retrieving passages that are marked as sections of documents and employing user behavior information to improve accuracy.…”

Section: Related Workmentioning

confidence: 99%

Improving passage ranking with user behavior information

Kong

Aktolga

Allan

2013

Proceedings of the 22nd ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

User behavior information has proved valuable for inferring document relevance, but its role in deducing relevance at the passage/section level is not well explored. In this paper, we study how user behavior information implies section relevance, and use this information to improve section ranking. More specifically, we focus on four types of user search behavior that occur while browsing a document -dwell time, highlighting, copying and clicks at the section level. Experimental results based on a commercial query log show that user behavior information can significantly improve section ranking. While section-level click information is a very powerful signal of relevance, it depends on an interface supporting section-level links. We find comparable levels of gain using other behavior information that does not depend upon such an interface.

show abstract

“…The overall scoring function is an average of the hypotheses, weighted by the success counters. For a complete description of the committee perceptron algorithm, refer to Table 4.1 [6].…”

Section: Committee Perceptronmentioning

confidence: 99%

Linguistic and semantic passage retrieval strategies for question answering

Bilotti

2011

SIGIR Forum

Self Cite

View full text Add to dashboard Cite

Question Answering (QA) is the task of searching a large text collection for specific answers to questions posed in natural language. Though they often have access to rich linguistic and semantic analyses of their input questions, QA systems often rely on off-the-shelf bag-of-words Information Retrieval (IR) solutions to retrieve passages matching a set of terms extracted from the question.There is a fundamental disconnect between the capabilities of the bag-of-words retrieval model and the retrieval needs of the QA system. Bag-of-words IR retrieves documents matching a query, but the QA system really needs documents that contain answers. Through question analysis, the QA system has compiled a sophisticated information need representation for what constitutes an answer to the question. This representation is composed of a set of linguistic and semantic constraints satisfied by answer-bearing passages. Unfortunately, off-the-shelf IR libraries commonly used in QA systems can not, in general, check these types of constraints at query-time. Poor quality retrieval can cause a QA system to fail if no answer-bearing text is retrieved, if it is not ranked highly enough, or if it is outranked or overwhelmed by false positives, text that matches the query well, yet supports a wrong answer.This thesis proposes two linguistic and semantic passage retrieval methods for QA, one based on structured retrieval and the other on rank-learning techniques. In addition, a methodology is proposed for mapping annotated text consisting of labeled spans and typed relations between them into an annotation graph representation. The annotation graph supports query-time linguistic and semantic constraint-checking, and serves as a unifying formalism for the QA system's information need and for retrieved passages. The proposed methods rely only on the relatively weak assumption that the QA system's information need can be represented as an annotation graph. The two approaches are shown to retrieve more answer-bearing text, more highly ranked, compared to a bag-of-words baseline for two different QA tasks. Linguistic and semantic passage retrieval methods are also shown to improve end-to-end QA system accuracy and answer MRR.iv Acknowledgments I would like to thank my advisor, Eric Nyberg, for all he has done, and the rest of my committee, Jamie Callan, Jaime Carbonell and Eric Brown, for helping to guide this work.

show abstract

Rank learning for factoid question answering with linguistic and semantic constraints

Cited by 36 publications

References 17 publications

Learning to rank for robust question answering

Learning to rank for robust question answering

Improving passage ranking with user behavior information

Linguistic and semantic passage retrieval strategies for question answering

Contact Info

Product

Resources

About