Kneser-Ney Smoothing on Expected Counts

Zhang, Hui; Chiang, David

doi:10.3115/v1/p14-1072

Cited by 21 publications

(33 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, better estimation methods during the maximization step of EM have been proposed to avoid the over-fitting in WA, such as using Kneser-Ney Smoothing to back-off the expected counts (Zhang and Chiang, 2014) or integrating the smoothed l 0 prior to the estimation of probability (Vaswani et al, 2012). Our work differs from theirs by addressing the over-fitting directly in the EM algorithm by adopting a leave-one-out approach.…”

Section: Related Workmentioning

confidence: 99%

Leave-one-out Word Alignment without Garbage Collector Effects

Wang

Utiyama

Finch

et al. 2015

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Expectation-maximization algorithms, such as those implemented in GIZA++ pervade the field of unsupervised word alignment. However, these algorithms have a problem of over-fitting, leading to "garbage collector effects," where rare words tend to be erroneously aligned to untranslated words.This paper proposes a leave-one-out expectationmaximization algorithm for unsupervised word alignment to address this problem. The proposed method excludes information derived from the alignment of a sentence pair from the alignment models used to align it. This prevents erroneous alignments within a sentence pair from supporting themselves. Experimental results on Chinese-English and Japanese-English corpora show that the F 1 , precision and recall of alignment were consistently increased by 5.0% -17.2%, and BLEU scores of end-to-end translation were raised by 0.03 -1.30. The proposed method also outperformed l 0 -normalized GIZA++ and Kneser-Ney smoothed GIZA++.

show abstract

Section: Related Workmentioning

confidence: 99%

Leave-one-out Word Alignment without Garbage Collector Effects

Wang

Utiyama

Finch

et al. 2015

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

show abstract

“…Namely, methods like the Witten-Bell smoothing [20] and the KneserNey smoothing [21] are still used in state-of-the-art work [22,23,24]. In general, these methods combine high-order models with lower-order ones.…”

Section: N-gram Smoothing Methodsmentioning

confidence: 99%

Stepwise API usage assistance using n -gram language models

Santos

Prendi

Sousa

et al. 2017

Journal of Systems and Software

View full text Add to dashboard Cite

Reusing software involves learning third-party APIs, a process that is often time-consuming and error-prone. Recommendation systems for API usage assistance based on statistical models built from source code corpora are capable of assisting API users through code completion mechanisms in IDEs. A valid sequence of API calls involving different types may be regarded as a well-formed sentence of tokens from the API vocabulary. In this article we describe an approach for recommending subsequent tokens to complete API sentences using n-gram language models built from source code corpora. The provided system was integrated in the code completion facilities of the Eclipse IDE, providing contextualized completion proposals for Java taking into account the nearest lines of code. The approach was evaluated against existing client code of four widely used APIs, revealing that in more than 90% of the cases the expected subsequent token is within the 10-top-most proposals of our models. The high score provides evidence that the recommendations could help on API learning and exploration, namely through the assistance on writing valid API sentences.

show abstract

“…To avoid overfitting, we find that it is necessary to apply an expected smoothing approach in training. We choose expected Kneser-Ney smoothing technique (Zhang and Chiang, 2014) as it is simple and achieves state-of-the-art performance on the language modeling problem.…”

Section: Biasing Language Modelsmentioning

confidence: 99%