Multi-Task Active Learning for Neural Semantic Role Labeling on Low Resource Conversational Corpus

Ikhwantri, Fariz; Louvan, Samuel; Kurniawan, Kemal; Abisena, Bagas; Rachman, Valdi; Mahendra, Rahmad

doi:10.18653/v1/w18-3406

Cited by 11 publications

(11 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…To adapt active learning in a multitask scenario, Reichart et al (2008) proposed a multi-task active learning framework for linguistic annotations with CRF-based models. With the advantages of the RNN model, a multi-task active learning framework was put forward for neural semantic role labeling on low resource conversational corpus (Ikhwantri et al 2018). The above methods are uncertainty-based active learning without demanding diversity explicitly.…”

Section: Related Workmentioning

confidence: 99%

“…The effectiveness of active learning is influenced by the performance of task models. Considering the relevance of different tasks, some researchers have proposed multi-task active learning for linguistic annotations (Reichart et al 2008;Ikhwantri et al 2018). The model in (Ikhwantri et al 2018) exploits the encoder-decoder framework with soft-shared parameters.…”

Section: Introductionmentioning

confidence: 99%

“…Considering the relevance of different tasks, some researchers have proposed multi-task active learning for linguistic annotations (Reichart et al 2008;Ikhwantri et al 2018). The model in (Ikhwantri et al 2018) exploits the encoder-decoder framework with soft-shared parameters. The encoder is trained to learn the shared features beneficial to all tasks.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

MTAAL: Multi-Task Adversarial Active Learning for Medical Named Entity Recognition and Normalization

Zhou

Cai

Zhang

et al. 2021

AAAI

View full text Add to dashboard Cite

Automated medical named entity recognition and normalization are fundamental for constructing knowledge graphs and building QA systems. When it comes to medical text, the annotation demands a foundation of expertise and professionalism. Existing methods utilize active learning to reduce costs in corpus annotation, as well as the multi-task learning strategy to model the correlations between different tasks. However, existing models do not take task-specific features for different tasks and diversity of query samples into account. To address these limitations, this paper proposes a multi-task adversarial active learning model for medical named entity recognition and normalization. In our model, the adversarial learning keeps the effectiveness of multi-task learning module and active learning module. The task discriminator eliminates the influence of irregular task-specific features. And the diversity discriminator exploits the heterogeneity between samples to meet the diversity constraint. The empirical results on two medical benchmarks demonstrate the effectiveness of our model against the existing methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

MTAAL: Multi-Task Adversarial Active Learning for Medical Named Entity Recognition and Normalization

Zhou

Cai

Zhang

et al. 2021

AAAI

View full text Add to dashboard Cite

show abstract

“…Trong lĩnh vực tổng quát, ở hầu hết các công trình, bộ mẫu có được là do khai khoáng từ ngữ liệu [21][22][23] 36 . Khi áp dụng vào văn bản Y Sinh, chúng tôi nhận thấy bài toán SRL có liên quan mật thiết với bài toán Rút trích Thực thể (Named Entity Recognition -NER), vì loại thực thể của đối số quyết định vai trò ngữ nghĩa của đối số (Ví dụ loại thực thể DNA chỉ có thể giữ vai trò "tác nhân" của vị ngữ "encode" chứ không thể giữ vai trò "sản phẩm").…”

Section: Hướng Khớp Mẫuunclassified

A deep-learning model for semantic role labelling in medical documents

Đức

Tứ

Huy

2021

Sci. Tech. Dev. J. - Nat. Sci.

View full text Add to dashboard Cite

We built a model labelling the Predicate Argument Structure (PAS) for biomedical documents. PAS is an important semantic information of any document, because it reveals the main event mentioned in each sentence. Extracting PAS in a sentence is an important premise for the computer to solve a series of other problems related to the semantics in text such as event extraction, named entity extraction, question answering system… The predicate argument structure is domain dependent. Therefore, in Biomedical field, it is required to define a completely new Predicate Argument frame compared to the general field. For a machine learning model to work well with a new argument frame, identifying a new feature set is required. This is difficult, manual and requires a lot of expert labor. To address this challenge, we chose to train our model with Deep Learning method utilizing Bi-directional Long Short Term Memory. Deep learning is a machine learning method that does not require defining the feature sets manually. In addition, we also integrate Highway Connection between hidden neuron layers to minimize derivative loss. Besides, to overcome the problem of small training corpus, we integrate Deep Learning with Multi-task Learning technique. Multi-task Learning helps the main task (PAS tagging) to be complemented with knowledge learnt from a closely related task, the NER. Our model achieved F1 = 75.13% without any manually designed feature, thereby showing the prospect of Deep Learning in this domain. In addition, the experiment results also show that Multi-task Learning is an appropriate technique to overcome the problem of little training data in biomedical fields, by improving the F1 score.

show abstract

“…Active Learning, Task Selection and Sampling Our sampling technique is similar to the ones found in several active learning algorithms (Chen et al, 2006) that are based on Shannon entropy estimations. Reichart et al (2008) and Ikhwantri et al (2018) examined Multi-Task Active Learning (MTAL) using a two task annotation scenario and showed performance gains while needing less labelled data. Our approach is a substantially different variant of MTAL since it was developed for task selection.…”

Section: Related Workmentioning

confidence: 99%

Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data

Pilault¹,

Amine²,

Pal³

2020

Preprint

View full text Add to dashboard Cite

Multi-Task Learning (MTL) has emerged as a promising approach for transferring learned knowledge across different tasks. However, multi-task learning must deal with challenges such as: overfitting to low resource tasks, catastrophic forgetting, and negative task transfer, or learning interference. Additionally, in Natural Language Processing (NLP), MTL alone has typically not reached the performance level possible through per-task fine-tuning of pretrained models. However, many fine-tuning approaches are both parameter inefficient, e.g. potentially involving one new model per task, and highly susceptible to losing knowledge acquired during pretraining. We propose a novel transformer based architecture consisting of a new conditional attention mechanism as well as a set of task conditioned modules that facilitate weight sharing. Through this construction we achieve more efficient parameter sharing and mitigate forgetting by keeping half of the weights of a pretrained model fixed. We also use a new multi-task data sampling strategy to mitigate the negative effects of data imbalance across tasks. Using this approach we are able to surpass single-task fine-tuning methods while being parameter and data efficient. With our base model, we attain 2.2% higher performance compared to a full fine-tuned BERT large model on the GLUE benchmark, adding only 5.6% more trained parameters per task (whereas naive fine-tuning potentially adds 100% of the trained parameters per task) and needing only 64.6% of the data. We show that a larger variant of our single multi-task model approach performs competitively across 26 NLP tasks and yields state-of-the-art results on a number of test and development sets.

show abstract

Multi-Task Active Learning for Neural Semantic Role Labeling on Low Resource Conversational Corpus

Cited by 11 publications

References 19 publications

MTAAL: Multi-Task Adversarial Active Learning for Medical Named Entity Recognition and Normalization

MTAAL: Multi-Task Adversarial Active Learning for Medical Named Entity Recognition and Normalization

A deep-learning model for semantic role labelling in medical documents

Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data

Contact Info

Product

Resources

About