Supervised learning for robust term extraction

Abstract

We propose a machine learning method to automatically classify the extracted ngrams from a corpus into terms and non-terms. We use 10 common statistics in previous term extraction literature as features for training. The proposed method, applicable to term recognition in multiple domains and languages, can help 1) avoid the laborious work in the post-processing (e.g. subjective threshold setting); 2) handle the skewness and demonstrate noticeable resilience to domain-shift issue of training data. Experiments are carried out on 6 corpora of multiple domains and languages, including GENIA and ACLRD-TEC(1.0) corpus as training set and four TTC subcorpora of wind energy and mobile technology in both Chinese and English as test set. Promising results are found, which indicate that this approach is capable of identifying both single word terms and multiword terms with reasonably good precision and recall.

Feature-Less End-to-End Nested Term Extraction

“…4. In the table, [5] achieves a satisfying result with Random Forest method in their paper, but the feature preparation is complex and time-consuming. The training data is also re-balanced on the positive and negative instances.…”

“…2. Yuan et al [5], a feature-based machine learning method using n-grams as term candidates and 10 kinds features is pre-processed for each candidate. Our best models for testing are chosen with the loss of development dataset Noted that: Decreasing the term ratio α will increase the precision but degenerate the recall (Fig.…”

“…Machine-learning based ATE [5,6,7,8] is to design and learn different features in the raw text or from syntax information, and then integrate these features into a machine learning method (such as conditional random field, supporting vector classifier). However, different domain, especially language shares different feature patterns, making this method specified to one language or domain.…”

“…2 for more details about term span. [1,3], [1,4], [1,5], [2,2], [2,3], [2,4], [2,5], [3,3], [3,4], [3,5], [4,4], [4,5], [5,5]…”

SemRe-Rank

“…Given training data, machine learning based methods [Astrakhantsev 2014;Conrado et al 2013;Fedorenko et al 2014;Maldonado and Lewis 2016] typically transform training instances into a feature space and train a classifier that can be later used for prediction. The features can be linguistic (e.g., PoS pattern, presence of special characters, etc), or statistical or a combination of both, which often utilise scores calculated by statistical ATE metrics [Maldonado and Lewis 2016;Yuan et al 2017]. However, one of the major problems in applying machine learning to ATE is the availability of reliable training data.…”