Translation memory retrieval methods

Abstract

Translation Memory (TM) systems are one of the most widely used translation technologies. An important part of TM systems is the matching algorithm that determines what translations get retrieved from the bank of available translations to assist the human translator. Although detailed accounts of the matching algorithms used in commercial systems can't be found in the literature, it is widely believed that edit distance algorithms are used. This paper investigates and evaluates the use of several matching algorithms, including the edit distance algorithm that is believed to be at the heart of most modern commercial TM systems. This paper presents results showing how well various matching algorithms correlate with human judgments of helpfulness (collected via crowdsourcing with Amazon's Mechanical Turk). A new algorithm based on weighted n-gram precision that can be adjusted for translator length preferences consistently returns translations judged to be most helpful by translators for multiple domains and language pairs.

Encoding Gated Translation Memory into Neural Machine Translation

“…where Levenshtein(s a , s b ) is the word-based Levenshtein Distance between s a and s b . The fuzzy match score can also be calculated with other methods, e.g., the method introduced in (Bloodgood and Strauss, 2015). We leave FMS estimated with different methods to our future work.…”

“…Our data come from the Chinese-English United Nations Parallel Corpus (Rafalovitch et al, 2009), which consists of official records and other parliamentary documents. Since large-scale public The fuzzy match score can also be calculated with other methods, e.g., the method introduced in (Bloodgood and Strauss, 2015). We leave FMS estimated with different methods to our future work.…”

Improving the Translation Environment for Professional Translators

“…We investigate whether such abstract, syntax-based matching is able to assess the usefulness of matches in a better way than methods purely based on sequences of words. The fuzzy matching metrics we use are not only the string based metrics such as Levenshtein distance [1], Translation Edit Rate (TER) [2], Percent Match and n-gram precision [3] (a sentence-based metric very similar to BLEU [4]). We apply these metrics also on strings of lemmas and also use METEOR [5].…”

TMop: a Tool for Unsupervised Translation Memory Cleaning

“…Depending on how the matching is performed, its output can be a mix of perfect and partial matches requiring variable amounts of correc-tions by the user. For this reason, most prior works on TM technology focused on improving this aspect (Gupta et al, 2014;Bloodgood and Strauss, 2014;Vanallemeersch and Vandeghinste, 2015;Chatzitheodoroou, 2015;Gupta et al, 2015).…”