A Markov random field model for term dependencies

Abstract

This paper develops a general, formal framework for modeling term dependencies via Markov random fields. The model allows for arbitrary text features to be incorporated as evidence. In particular, we make use of features based on occurrences of single terms, ordered phrases, and unordered phrases. We explore full independence, sequential dependence, and full dependence variants of the model. A novel approach is developed to train the model that directly maximizes the mean average precision rather than maximizing the likelihood of the training data. Ad hoc retrieval experiments are presented on several newswire and web collections, including the GOV2 collection used at the TREC 2004 Terabyte Track. The results show significant improvements are possible by modeling dependencies, especially on the larger web collections.

Probabilistic Models for Text Mining

“…Recently, MRF has been widely used in many text mining tasks, such as text categorization [16] and information retrieval [44]. In [44], MRF is used to model the term dependencies using the joint distribution over queries and documents.…”

“…In [44], MRF is used to model the term dependencies using the joint distribution over queries and documents. The model allows for arbitrary text features to be incorporated as evidence.…”

Generating, Refining and Using Sentiment Lexicons

“…20.6 we need a topical baseline: a set of blog posts potentially relevant to each topic. For this, we use the Indri retrieval engine, and apply the Markov Random Fields to model term dependencies in the query [19] to improve topical retrieval. We retrieve the top 1,000 posts for each query.…”

Modeling Variable Dependencies between Characters in Chinese Information Retrieval

“…The final model combines the unigram model and the biterm model. Metzler and Croft [8] proposed Markov Random Field (MRF) models for IR, in which dependencies between terms in the same clique (a set of fully connected nodes) are considered. In the full dependence model (MRF-FD), all the terms in a sentence (query) are assumed to be dependent.…”

“…However, the relationship between them is simply determined by the scale of the overlap, which does not necessarily reflect the true strength of relationship. In general IR, several models have also been proposed to capture dependencies between terms [4,8,9]. Usually, a dependence model is defined in addition to the traditional bag-of-words or word unigram model.…”