When is Char Better Than Subword: A Systematic Study of Segmentation Algorithms for Neural Machine Translation

Abstract

Subword segmentation algorithms have been a de facto choice when building neural machine translation systems. However, most of them need to learn a segmentation model based on some heuristics, which may produce suboptimal segmentation. This can be problematic in some scenarios when the target language has rich morphological changes or there is not enough data for learning compact composition rules. Translating at fully character level has the potential to alleviate the issue, but empirical performances of character-based models has not been fully explored. In this paper, we present an in-depth comparison between character-based and subword-based NMT systems under three settings: translating to typologically diverse languages, training with low resource, and adapting to unseen domains. Experimental results show strong competitiveness of character-based models. Further analyses show that compared to subword-based models, character-based models are better at handling morphological phenomena, generating rare and unknown words, and more suitable for transferring to unseen domains.

Why don't people use character-level machine translation?

“…The results are presented in Table 1. Except for translation into Arabic (which is consistent with the findings of Levy, 2021a andLi et al, 2021), where character methods outperform BPEs, subword methods are always better than characters.…”

“…Therefore, we assume that, if character-level models were a fully-fledged alternative to subword models, at least some systems submitted to the shared tasks would use character-level models. Li et al (2021) evaluated domain robustness by training models on small domain-specific datasets and evaluating them on unrelated domains, claiming the superiority of character-level models in this setup. We argue that this is a very unnatural setup and rather evaluate the domain robustness by evaluating general models on domain-specific test sets.…”

“…Their results show that although character-level and byte-level models are usually worse than BPE models, byte-based models without embedding layers often outperform BPE-based models in the out-of-English direction. Using simiarly small datasets, Li et al (2021) claim that character-level modeling outperforms BPE when translating into fusional, agglutinative, and introflexive languages. Nikolov et al (2018) experimented with character-level models for romanized Chinese.…”

“…Several papers in the past claimed parity of character-based methods with subword models, highlighting advantageous features of such systems. Very recent examples include Gao et al (2020); Banar et al (2020); Li et al (2021). Despite this, character-level methods are rarely used as strong baselines in research papers and shared task submissions, suggesting that character-level models might have drawbacks that are not sufficiently addressed in the literature.…”

Why don’t people use character-level machine translation?

“…For NMT, several papers have claimed parity of character-based methods with subword models, highlighting advantageous features of such systems. Very recent examples include Gao et al (2020); Banar et al (2020); Li et al (2021). Despite this, character-level methods are rarely used as strong baselines in research papers and shared task submissions, suggesting that character-level models might have drawbacks that are not sufficiently addressed in the literature.…”