Improving Retrieval Performance for Verbose Queries via Axiomatic Analysis of Term Discrimination Heuristic

Ariannezhad, Mozhdeh; Montazeralghaem, Ali; Zamani, Hamed; Shakery, Azadeh

doi:10.1145/3077136.3080761

Cited by 10 publications

(4 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Apart from the diagnosis of existing functions, Fang et al derived novel retrieval functions based on their initial set of constraints [15] and later extended their list of axioms from purely term-matching to semantic-matching based constraints [16,12]. Others have contributed query term proximity [42,18], document length normalization [27] and query term discrimination [1] constraints, consistently showing that traditional models improve when slightly altered to satisfy those constraints. While most of the more than twenty existing axioms have been designed for standard retrieval models, a number of axioms have been proposed for more specialized cases, such as statistical translation models [24] and pseudo-relevance feedback [3,4,30].…”

Section: Related Workmentioning

confidence: 99%

“…Traditional retrieval models have been engineered based on search heuristics that later have been formalized into axioms-formal constraints that should be fulfilled by a good model-which enable us to analytically investigate to what extent retrieval models fulfill them [13,15,16,14]. This analytical approach enabled researchers to identify shortcomings in existing retrieval models and "fix" them [18,1,12,27,4,30], in order to achieve higher retrieval effectiveness. Ideally, we employ a similar axiomatic approach to diagnose & fix neural IR models in order to reap the benefits deep learning has offered in other fields.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Axiomatic Approach to Diagnosing Neural IR Models

Rennings

Moraes

Hauff

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Traditional retrieval models such as BM25 or language models have been engineered based on search heuristics that later have been formalized into axioms. The axiomatic approach to information retrieval (IR) has shown that the effectiveness of a retrieval method is connected to its fulfillment of axioms. This approach enabled researchers to identify shortcomings in existing approaches and "fix" them. With the new wave of neural net based approaches to IR, a theoretical analysis of those retrieval models is no longer feasible, as they potentially contain millions of parameters. In this paper, we propose a pipeline to create diagnostic datasets for IR, each engineered to fulfill one axiom. We execute our pipeline on the recently released large-scale question answering dataset WikiPassageQA (which contains over 4000 topics) and create diagnostic datasets for four axioms. We empirically validate to what extent wellknown deep IR models are able to realize the axiomatic pattern underlying the datasets. Our evaluation shows that there is indeed a positive relation between the performance of neural approaches on diagnostic datasets and their retrieval effectiveness. Based on these findings, we argue that diagnostic datasets grounded in axioms are a good approach to diagnosing neural IR models.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An Axiomatic Approach to Diagnosing Neural IR Models

Rennings

Moraes

Hauff

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Retrieval with verbose queries is also similar to associative document search, a difference being that the verbose queries, in contrast to query-documents, are usually shorter in length, comprised usually of a small number of well-formed sentences [19,31]. IR approaches specifically targeted for verbose queries usually employ a query length normalization component [3,28], or transform the verbose query to a weighted term distribution (assigning higher weights to the terms that better describe the information need) estimated from the top-retrieved documents [31].…”

Section: Related Workmentioning

confidence: 99%

Effective Query Formulation in Conversation Contextualization: A Query Specificity-based Approach

Pal

Ganguly

2021

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval

View full text Add to dashboard Cite

Proactively retrieving relevant information to contextualize conversations has potential applications in better understanding the conversational content between communicating parties. Since in contrast to traditional IR, there is no explicitly formulated userquery, an important research challenge is to first identify the candidate segments of text that may require contextualization for a better comprehension of their content, and then make use of these identified segments to formulate a query and eventually retrieve the potentially relevant information to augment a conversation. In this paper, we propose a generic unsupervised framework that involves shifting overlapping windows of terms through a conversation and estimating scores indicating the likelihood of the existence of an information need within these segments. Within our proposed framework, we investigate a query performance prediction (QPP) based approach for scoring these candidate term windows with the hypothesis that a term window that indicates a higher specificity is likely to be indicative of a potential information need requiring contextualization. Our experiments revealed that the QPP approaches of scoring the term windows provide better contextualization than other term extraction approaches. A post-retrieval QPP approach was observed to yield better results than a pre-retrieval one.

show abstract

“…In natural language processing, the queried patterns can also be long [95]. Examples of such patterns are queries in question answering systems [51], description queries in TREC datasets [13,3], and representative phrases in documents [83]. Similarly, a query pattern can be long when it encodes an entire document (e.g., a webpage in the context of deduplication [53]), or machine-generated messages [58].…”

Section: Introductionmentioning

confidence: 99%

Text Indexing for Long Patterns: Anchors are All you Need

Ayad¹,

Loukides

Pissis

2023

Proc. VLDB Endow.

View full text Add to dashboard Cite

In many real-world database systems, a large fraction of the data is represented by strings: sequences of letters over some alphabet. This is because strings can easily encode data arising from different sources. It is often crucial to represent such string datasets in a compact form but also to simultaneously enable fast pattern matching queries. This is the classic text indexing problem. The four absolute measures anyone should pay attention to when designing or implementing a text index are: (i) index space; (ii) query time; (iii) construction space; and (iv) construction time. Unfortunately, however, most (if not all) widely-used indexes (e.g., suffix tree, suffix array, or their compressed counterparts) are not optimized for all four measures simultaneously, as it is difficult to have the best of all four worlds. Here, we take an important step in this direction by showing that text indexing with locally consistent anchors (lc-anchors) offers remarkably good performance in all four measures, when we have at hand a lower bound l on the length of the queried patterns --- which is arguably a quite reasonable assumption in practical applications. Specifically, we improve on the construction of the index proposed by Loukides and Pissis, which is based on bidirectional string anchors (bd-anchors), a new type of lc-anchors, by: (i) designing an average-case linear-time algorithm to compute bd-anchors; and (ii) developing a semi-external-memory implementation to construct the index in small space using near-optimal work. We then present an extensive experimental evaluation, based on the four measures, using real benchmark datasets. The results show that, for long patterns, the index constructed using our improved algorithms compares favorably to all classic indexes: (compressed) suffix tree; (compressed) suffix array; and the FM-index.

show abstract

Improving Retrieval Performance for Verbose Queries via Axiomatic Analysis of Term Discrimination Heuristic

Cited by 10 publications

References 10 publications

An Axiomatic Approach to Diagnosing Neural IR Models

An Axiomatic Approach to Diagnosing Neural IR Models

Effective Query Formulation in Conversation Contextualization: A Query Specificity-based Approach

Text Indexing for Long Patterns: Anchors are All you Need

Contact Info

Product

Resources

About