Using target-language information to train part-of-speech taggers for machine translation

Abstract

Although corpus-based approaches to machine translation (MT) are growing in interest, they are not applicable when the translation involves lessresourced language pairs for which there are no parallel corpora available; in those cases, the rule-based approach is the only applicable solution. Most rule-based MT systems make use of part-of-speech (PoS) taggers to solve the PoS ambiguities in the source-language texts to translate; those MT systems require accurate PoS taggers to produce reliable translations in the target language (TL). The standard statistical approach to PoS ambiguity resolution (or tagging) uses hidden Markov models (HMM) trained in a supervised way from hand-tagged corpora, an expensive resource not always available, or in an unsupervised way through the Baum-Welch expectation-maximization algorithm; both methods use information only from the language being tagged. However, when tagging is considered as an intermediate task for the translation procedure, that is, when the PoS tagger is to be embedded as a module within an MT system, information from the TL can be (unsupervisedly) used in the training phase to increase the translation quality of the whole MT system. This paper presents a method to train HMM-based PoS taggers to be used in MT; the new method uses not only information from the source language (SL), as general-purpose methods do, but also information from the TL and from the remaining modules of the MT system in which the PoS tagger is to be embedded. We find that the translation quality of the MT system embedding a PoS tagger trained in an unsupervised manner through 123 30 F. Sánchez-Martínez et al. this new method is clearly better than that of the same MT system embedding a PoS tagger trained through the Baum-Welch algorithm, and comparable to that obtained by embedding a PoS tagger trained in a supervised way from hand-tagged corpora.

Inferring Shallow-Transfer Machine Translation Rules from Small Parallel Corpora

“…The development of such transfer rules requires qualified people to code them manually; therefore, their automatic inference may save part of this human effort. The method we present is entirely unsupervised and benefits from information in the rest of modules of the MT system in which the inferred rules are applied, in line with the method proposed by Sánchez-Martínez et al (2008) to train part-of-speech taggers in an unsupervised way for their use in MT.…”

PurePos: An Open Source Morphological Disambiguator

“…Candidates for a starting point for our development could also have been one of the following tools: TriTagger [12,11], an open source trigram HMM tagger implemented in Java, has a language specific integrated morphological analyser (for Icelandic) -called IceMorphy [10] -but it does not perform lemmatization and lacks a few clever tricks implemented in HunPos, which seem to boost the performance of that tagger significantly. Another open source trigram HMM tagger [18,17] implemented in C/C++ is included in the Apertium [1] rule based machine translation toolset. It performs full morphological disambiguation by taking morphologically analyzed input and disambiguating it.…”

Part of speech tagging: a systematic review of deep learning and machine learning approaches

“…improves human-to-human communication, enables human-to-machine communication by doing useful processing of texts or speeches. Part-of-speech (POS) tagging is one of the most important addressed areas and main building block and application in the natural language processing discipline [1][2][3]. So, Part of Speech (POS) Tagging is a notable NLP topic that aims in assigning each word of a text the proper syntactic tag in its context of appearance [4][5][6][7][8].…”