Cooccurrence smoothing for stochastic language modeling

Essen, Ute; Steinbiss, Volker

doi:10.1109/icassp.1992.225947

Cited by 48 publications

(44 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Considerable latitude is allowed in defining the set S(w 1 ), as is evidenced by previous work that can be put in the above form. Essen & Steinbiss (1992) and Karov & Edelman (1996) (implicitly) set S(w 1 ) = V 1 . However, it may be desirable to restrict S(w 1 ) in some fashion for efficiency reasons, especially if V 1 is large.…”

Section: Combining Evidencementioning

confidence: 99%

“…In the disambiguation evaluation, we compared several variants of our initial method and the cooccurrence smoothing method of Essen & Steinbiss (1992) against the estimation method of Katz in a decision task involving unseen pairs of direct objects and verbs. We found that all the similarity-based models performed almost 40% better than back-off, which yielded about 49% accuracy in our experimental setting.…”

Section: Main Contributionsmentioning

confidence: 99%

“…Disambiguation decisions are based on comparing scores for different alternatives, but they do not produce explicit probability estimates and therefore cannot be integrated directly within a larger probabilistic framework. The cooccurrence smoothing model of Essen & Steinbiss (1992), like our model, produces explicit estimates of word cooccurrence probabilities based on the cooccurrence statistics of similar words. The similarity-based estimates are interpolated with direct estimates of n-gram probabilities to form a smoothed n-gram language model.…”

Section: Statistical Similarity and Clustering For Disambiguation Andmentioning

confidence: 99%

“…Lin (1991) notes that L is an upper bound for J: Sugawara, Nishimura, Toshioka, Okochi, & Kaneko (1985), Essen & Steinbiss (1992) used confusion probability to estimate word cooccurrence probabilities. 3 They report 14% improvement in test-set perplexity (defined below) on a small corpus.…”

Section: Jensen-shannon Divergencementioning

confidence: 99%

“…For example, Essen & Steinbiss (1992) report that in a 75%-25% split of the million-word LOB corpus, 12% of the bigrams in the test partition did not occur in the training portion. For trigrams, the sparse data problem is even more severe: for instance, researchers at IBM (Brown, DellaPietra, deSouza, Lai, & Mercer, 1992) examined a training corpus consisting of almost 366 million English words, and discovered that one can expect 14.7% of the word triples in any new English text to be absent from the training sample.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Untitled

1999

View full text Add to dashboard Cite

Abstract. In many applications of natural language processing (NLP) it is necessary to determine the likelihood of a given word combination. For example, a speech recognizer may need to determine which of the two word combinations "eat a peach" and "eat a beach" is more likely. Statistical NLP methods determine the likelihood of a word combination from its frequency in a training corpus. However, the nature of language is such that many word combinations are infrequent and do not occur in any given corpus. In this work we propose a method for estimating the probability of such previously unseen word combinations using available information on "most similar" words.We describe probabilistic word association models based on distributional word similarity, and apply them to two tasks, language modeling and pseudo-word disambiguation. In the language modeling task, a similarity-based model is used to improve probability estimates for unseen bigrams in a back-off language model. The similaritybased method yields a 20% perplexity improvement in the prediction of unseen bigrams and statistically significant reductions in speech-recognition error.We also compare four similarity-based estimation methods against back-off and maximum-likelihood estimation methods on a pseudo-word sense disambiguation task in which we controlled for both unigram and bigram frequency to avoid giving too much weight to easy-to-disambiguate high-frequency configurations. The similaritybased methods perform up to 40% better on this particular task.

show abstract

Section: Combining Evidencementioning

confidence: 99%

Section: Main Contributionsmentioning

confidence: 99%

Section: Statistical Similarity and Clustering For Disambiguation Andmentioning

confidence: 99%

Section: Jensen-shannon Divergencementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Untitled

1999

View full text Add to dashboard Cite

show abstract

The link‐prediction problem for social networks

Liben-Nowell

Kleinberg

2007

J. Am. Soc. Inf. Sci.

3,079

1,701

View full text Add to dashboard Cite

Given a snapshot of a social network, can we infer which new interactions among its members are likely to occur in the near future? We formalize this question as the link-prediction problem, and we develop approaches to link prediction based on measures for analyzing the "proximity" of nodes in a network. Experiments on large co-authorship networks suggest that information about future interactions can be extracted from network topology alone, and that fairly subtle measures for detecting node proximity can outperform more direct measures.

show abstract

Untitled

2007

J. Am. Soc. Inf. Sci.

487

View full text Add to dashboard Cite

In this article we discuss re-retrieving personal information objects and relate the task to recovering from lapse(s) in memory. We propose that fundamentally it is lapses in memory that impede users from successfully re-finding the information they need. Our hypothesis is that by learning more about memory lapses in non-computing contexts and how people cope and recover from these lapses, we can better inform the design of PIM tools and improve the user's ability to re-access and re-use objects. We describe a diary study that investigates the everyday memory problems of 25 people from a wide range of backgrounds. Based on the findings, we present a series of principles that we hypothesize will improve the design of personal information management tools. This hypothesis is validated by an evaluation of a tool for managing personal photographs, which was designed with respect to our findings. The evaluation suggests that users' performance when re-finding objects can be improved by building personal information management tools to support characteristics of human memory.

show abstract

Cooccurrence smoothing for stochastic language modeling

Cited by 48 publications

References 3 publications

Untitled

Untitled

The link‐prediction problem for social networks

Untitled

Contact Info

Product

Resources

About