A stochastic parser based on a structural word prediction model

Abstract

]in this paper, we present a stochastic language model using dependency. This model considers a sentence as a word sequence and predicts each word from left to right. The history at each step of prediction is a sequence of partial parse krees covering the preceding words. First ore: model predicts the partial parse trees which have a dependency relation with the next word among them and then predicts the next word fi'om only the trees which have a dependency relation with the next word. Our: model is a generative stochastic model, thus this can be used not only as a parser but also as ~ language model of a speech recognizer. In our experiment, we prepared about 1,000 syntactically annotated Japanese sentences extracted fl'om a financial newspaper and estimated the parameters of our model. We built a parser based on our: model and tested it on approximately 10O sentences of the same newspaper. The accuracy of the dependency relation was 89.9%, the highest, accuracy level obtained by Japanese stochastic parsers.

Sequential dependency analysis for online spontaneous speech processing

“…(1) Analysis of incomplete sentences Conventional dependency analysis methods including those explained above extract the dependency structures from written text (Collins, 1996;Nivre, 2002;Kudo and Matsumoto, 2002;Mori et al, 2000). These methods are designed to analyze complete sentences that are divided by punctuation in written text.…”

Sequential dependency analysis for spontaneous speech understanding

“…Recent work on written texts [3] [4][5] [6] assumes that the complete sentence is known before the dependency analysis starts.…”

A unified single scan algorithm for Japanese base phrase chunking and dependency parsing

“…Most of the modern dependency parsers for Japanese require bunsetsu chunking (base phrase chunking) before dependency parsing (Sekine et al, 2000;Kudo and Matsumoto, 2002;Sassano, 2004). Although wordbased parsers are proposed in (Mori et al, 2000;Mori, 2002), they do not build bunsetsus and are not compatible with other Japanese dependency parsers. Multilingual parsers of participants in the CoNLL 2006 shared task (Buchholz and Marsi, 2006) can handle Japanese sentences.…”