Semi-Supervised Sequence Labeling with Self-Learned Features

Abstract

Abstract-Typical information extraction (IE) systems can be seen as tasks assigning labels to words in a natural language sequence. The performance is restricted by the availability of labeled words. To tackle this issue, we propose a semisupervised approach to improve the sequence labeling procedure in IE through a class of algorithms with self-learned features (SLF). A supervised classifier can be trained with annotated text sequences and used to classify each word in a large set of unannotated sentences. By averaging predicted labels over all cases in the unlabeled corpus, SLF training builds class label distribution patterns for each word (or word attribute) in the dictionary and re-trains the current model iteratively adding these distributions as extra word features. Basic SLF models how likely a word could be assigned to target class types. Several extensions are proposed, such as learning words' class boundary distributions. SLF exhibits robust and scalable behaviour and is easy to tune. We applied this approach on four classical IE tasks: named entity recognition (German and English), part-of-speech tagging (English) and one gene name recognition corpus. Experimental results show effective improvements over the supervised baselines on all tasks. In addition, when compared with the closely related self-training idea, this approach shows favorable advantages.

Learning How to Self-Learn: Enhancing Self-Training Using Neural Reinforcement Learning

“…Semi-supervised learning is useful for improving model performances when a target domain or language lacks of manual resources. Self-training is a commonly used strategy for various natural language processing (NLP) tasks, such as named entity recognition (NER) (Kozareva et al, 2005), part-of-speech (POS) tagging (Wang et al, 2007;Qi et al, 2009) and parsing (McClosky et al, 2006(McClosky et al, , 2008Sagae, 2010).…”

Extracting Researcher Metadata with Labeled Features

“…Our proximity features are similar in spirit to selflabeled features previously studied for tagging problems [10]. Qi, et al proposed an iterative scheme, where feature vectors in each iteration are augmented with the predicted word-level class label distributions from the previous iteration in a semi-supervised manner.…”

Reference Information Extraction and Processing Using Random Conditional Fields

“…Our current work does not feature this aspect, but we regard it as one of the key next steps to be tackled. Consequently, we mention the research performed by Schwartz et al 27 Teufel et al, 28 or Wu et al 29 that deal with using citation contexts for discerning a citation's function and analyzing how this influences or is influenced by the work it points to.…”