An empirical study of semi-supervised structured conditional models for dependency parsing

Abstract

This paper describes an empirical study of high-performance dependency parsers based on a semi-supervised learning approach. We describe an extension of semisupervised structured conditional models (SS-SCMs) to the dependency parsing problem, whose framework is originally proposed in (Suzuki and Isozaki, 2008). Moreover, we introduce two extensions related to dependency parsing: The first extension is to combine SS-SCMs with another semi-supervised approach, described in (Koo et al., 2008). The second extension is to apply the approach to secondorder parsing models, such as those described in (Carreras, 2007), using a twostage semi-supervised learning approach. We demonstrate the effectiveness of our proposed methods on dependency parsing experiments using two widely used test collections: the Penn Treebank for English, and the Prague Dependency Treebank for Czech. Our best results on test data in the above datasets achieve 93.79% parent-prediction accuracy for English, and 88.05% for Czech.

A robust semi-supervised classification method for transfer learning

“…Recently, semi-supervised classifiers have been proposed based on hybrid generative/discriminative models [18,14,28,29,1]. As reported in [19], generative models unsuitable for real data samples provide worse classification performance than discriminative models.…”

“…In our formulation, classifiers are constructed by combining discriminative and generative models, which is similar to a previous semi-supervised classification framework, namely the Joint probability model Embedding style SemiSupervised Conditional Model (JESS-CM), which substantially improved the performance of natural language processing (NLP) tasks [28,29]. The proposed method provides a classifier training algorithm improved to deal with the difference between source and target distributions.…”

A Neural Probabilistic Structured-Prediction Method for Transition-Based Natural Language Processing

“…However, their parser is a joint model of parsing and POS-tagging, and they use external data in parsing. We list the result of , Koo et al (2008) and Suzuki et al (2009) in Table 7, which make use of large-scale unannotated text to improve parsing accuracies. The input embeddings of our parser are also trained over large raw text, and in this perspective our model is correlated with the semi-supervised models.…”

Generalizing a Strongly Lexicalized Parser using Unlabeled Data

“…Koo et al (2008) and Suzuki et al (2009) use unsupervised wordclusters as features in a dependency parser to get lexical dependencies. This has some notional similarity to categories, since, like categories, clusters are less fine-grained than words but more finegrained than POS-tags.…”