Trigram morphosyntactic tagger for Polish

Dębowski, Łukasz

doi:10.1007/978-3-540-39985-8_43

Cited by 18 publications

(4 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Statistical morphological disambiguation using small manually annotated training corpora looks as quite a simple task, when frequencies of grammatical features are generated during the training phase and the most likely sequence of morphological features is found in a new text by the help of various probability methods. Drawing on the experience of morphological annotation systems for other free word order languages (Dębowski, 2004;Hajič et al, 2001;Palanisamy et al, 2006 etc. ), it is obvious that the corpus-based method is most suitable for the developing such systems for Lithuanian.…”

Section: Automatic Morphological Annotation Of the Lithuanian Corpusmentioning

confidence: 99%

Morphological annotation of the Lithuanian corpus

Rimkutė

Daudaraviċius

Utka

2007

Proceedings of the Workshop on Balto-Slavonic Natural Language Processing Information Extraction and Enabling Technologies - AC

View full text Add to dashboard Cite

As the development of information technologies makes progress, large morphologically annotated corpora become a necessity, as they are necessary for moving onto higher levels of language computerisation (e. g. automatic syntactic and semantic analysis, information extraction, machine translation). Research of morphological disambiguation and morphological annotation of the 100 million word Lithuanian corpus are presented in the article. Statistical methods have enabled to develop the automatic tool of morphological annotation for Lithuanian, with the disambiguation precision of 94%. Statistical data about the distribution of parts of speech, most frequent wordforms, and lemmas, in the annotated Corpus of The Contemporary Lithuanian Language is also presented.

show abstract

Section: Automatic Morphological Annotation Of the Lithuanian Corpusmentioning

confidence: 99%

Morphological annotation of the Lithuanian corpus

Rimkutė

Daudaraviċius

Utka

2007

Proceedings of the Workshop on Balto-Slavonic Natural Language Processing Information Extraction and Enabling Technologies - AC

View full text Add to dashboard Cite

show abstract

“…It happens so only for the necessity of the effective search for the most probable hidden states. Some well-known applications of HMMs are automatic speech recognizers (Jelinek, 1997) and trigram partof-speech taggers (Manning and Schütze, 1999;Dębowski, 2004b). It was observed that the error rate of trigram taggers decreases as a negative power of the size of the training data.…”

Section: B Some Properties Of Infinitary Distributionsmentioning

confidence: 99%

On Hilberg's law and its links with Guiraud's law*

̧bowski

2006

Journal of Quantitative Linguistics

View full text Add to dashboard Cite

Hilberg (1990) supposed that finite-order excess entropy of a random human text is proportional to the square root of the text length. Assuming that Hilberg's hypothesis is true, we derive Guiraud's law, which states that the number of word types in a text is greater than proportional to the square root of the text length. Our derivation is based on some mathematical conjecture in coding theory and on several experiments suggesting that words can be defined approximately as the nonterminals of the shortest context-free grammar for the text. Such operational definition of words can be applied even to texts deprived of spaces, which do not allow for Mandelbrot's "intermittent silence" explanation of Zipf's and Guiraud's laws. In contrast to Mandelbrot's, our model assumes some probabilistic long-memory effects in human narration and might be capable of explaining Menzerath's law.

show abstract

“…Several techniques of morphosyntactic tagging for Polish have been explored over the years, including trigrams [4], transformation-based methods 3 (TaKIPI [12]; Pantera [1]), conditional random fields (WCRFT [13]; Concraft [20]), and neural networks (Toygger [9]; KRNNT [22]; MorphoDiTa-pl [19]). The latter now obtain state-of-the-art results 4 in the task of morphosyntactic tagging for Polish [7]. All these taggers adopt a pipeline architecture, where morphosyntactic disambiguation (including guessing) is preceded by sentence segmentation, word segmentation, and morphosyntactic analysis (not necessarily in this order).…”

Section: Introduction and Related Workmentioning

confidence: 99%

Morphosyntactic Disambiguation and Segmentation for Historical Polish with Graph-Based Conditional Random Fields

Waszczuk

Kieraś

Woliński

2018

Text, Speech, and Dialogue

View full text Add to dashboard Cite

The paper presents a system for joint morphosyntactic disambiguation and segmentation of Polish based on conditional random fields (CRFs). The system is coupled with Morfeusz, a morphosyntactic analyzer for Polish, which represents both morphosyntactic and segmentation ambiguities in the form of a directed acyclic graph (DAG). We rely on constrained linear-chain CRFs generalized to work directly on DAGs, which allows us to perform segmentation as a by-product of morphosyntactic disambiguation. This is in contrast with other existing taggers for Polish, which either neglect the problem of segmentation or rely on heuristics to perform it in a pre-processing stage. We evaluate our system on historical corpora of Polish, where segmentation ambiguities are more prominent than in contemporary Polish, and show that our system significantly outperforms several baseline segmentation methods.

show abstract

Trigram morphosyntactic tagger for Polish

Cited by 18 publications

References 4 publications

Morphological annotation of the Lithuanian corpus

Morphological annotation of the Lithuanian corpus

On Hilberg's law and its links with Guiraud's law*

Morphosyntactic Disambiguation and Segmentation for Historical Polish with Graph-Based Conditional Random Fields

Contact Info

Product

Resources

About