Building a large syntactically-annotated corpus of Vietnamese

Abstract

Treebank is an important resource for both research and application of natural language processing. For Vietnamese, we still lack such kind of corpora. This paper presents up-to-date results of a project for Vietnamese treebank construction. Since Vietnamese is an isolating language and has no word delimiter, there are many ambiguities in sentence analysis. We systematically applied a lot of linguistic techniques to handle such ambiguities. Annotators are supported by automaticlabeling tools and a tree-editor tool. Raw texts are extracted from Tuoi Tre (Youth), an online Vietnamese daily newspaper. The current annotation agreement is around 90 percent.

A syntactic component for Vietnamese language processing

“…Recently, a group of Vietnamese computational linguists has been involved in developing a treebank for Vietnamese (Nguyen et al 2009). This is the treebank we used for our extraction system.…”

“…The current scheme contains 13 grammatical relations representing principal functional dependencies between Vietnamese words. All these dependencies use the syntactic categories defined in the Vietnamese treebank (Nguyen et al 2009) and they are divided into three groups.…”

“…The grammar used to evaluate the parser is an LTAG extracted from the Vietnamese Treebank (Nguyen et al 2009) containing 10 163 sentences (225 085 words, about 22.14 words per sentence on average). Figure 14 shows the distribution of the number of sentences ac- We choose a subset of the treebank containing 8808 sentences of length 30 words or less as an evaluation corpus.…”

“…The tagset of the treebank has 38 syntactic labels (18 part-of-speech tags, 17 syntactic category tags, 3 empty categories) and 17 function tags. For details, please refer to Nguyen et al (2009).…”

Creating language resources for under-resourced languages: methodologies, and experiments with Arabic

“…The corpus is nearly one third the size of the published comparable Italian, German and English corpora. Nguyen et al (2009) built a large syntactically annotated corpus for Vietnamese by constructing a treebank. To create the annotated corpus they followed the same approach that was used to create the English Penn Treebank (Marcus et al 1993): automatic parsers annotated the corpus and human annotators corrected any errors.…”