Untitled

Lu, Peng; Bai, Ting; Langlais, Philippe

doi:10.18653/v1/n19-1249

Cited by 7 publications

(5 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, understanding the relationships between tasks has been identified as crucial, as highlighted by studies focusing on the adaptive sharing of multi-level distributed representations . Furthermore, the advent of novel LSTM cells encapsulating both shared and task-specific parameters, as proposed in SC-LSTM (Lu et al, 2019), shows promise in improving the performance of a target task by judicious selection of auxiliary tasks, and hence, adds a new dimension to the ongoing discourse. Additionally, the reparameterization of convolutions for incremental multi-task learning has provided a pathway to manage task-specific parameters efficiently, especially when introducing new tasks to the MTL framework (Kanakis et al, 2020).…”

Section: Multi-task Learningmentioning

confidence: 99%

An Auxiliary Task Boosted Multi-task Learning Method for Service Account Retrieval with Limited Human Annotation

Yao,

Zhang,

Yang

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track

View full text Add to dashboard Cite

Service accounts, including organizations' official accounts and mini-programs, provide various convenient services for users, and have become crucial components of a number of applications. Therefore, retrieving service accounts quickly and accurately is vital. However, this task suffers from the problem of limited human annotation, i.e., manually assessing account functionality and assigning ratings based on user experience is both laborintensive and time-consuming. To this end, this paper proposes a novel approach, the Auxiliary task Boosted Multi-Task Learning method (AuxBoost-MTL). Specifically, the proposed method introduces multiple auxiliary tasks, which are able to utilized the log data from our application as supervision, and enhance the performance of the main task, service account retrieval. Furthermore, we introduce an Adaptive Hierarchical Fusion Module (AHF module) into our approach. This module is designed to adaptively perform hierarchical fusion of embeddings from auxiliary tasks into the main task, thereby enhancing the model efficacy. Experiments on two real-world industrial datasets demonstrate the effectiveness of our proposed approach.

show abstract

Section: Multi-task Learningmentioning

confidence: 99%

An Auxiliary Task Boosted Multi-task Learning Method for Service Account Retrieval with Limited Human Annotation

Yao,

Zhang,

Yang

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track

View full text Add to dashboard Cite

show abstract

“…Many proposals in literature have focused on MTL for NER [33] or, POS [34], [35] or, both NER & POS [8], [9], [28], [36]- [39], but these solutions require huge RAM/ROM for on-device inferencing. Existing efforts towards employing MTL for tasks such as sequence labeling and semantic tasks have primarily focused on accuracy, leading to models that are huge for ondevice use where resources are constrained [6], [40].…”

Section: Related Workmentioning

confidence: 99%

“…There exists literature on learning joint representations for both NER and POS using MTL and they have shown promising results in avoiding over-fitting but have been applied for serverside processing [8], [9]. Further, works on building on-device inference models for text classification [10], [11] highlight the advantages of on-device inferencing of neural models.…”

Section: Introductionmentioning

confidence: 99%

LiteMuL: A Lightweight On-Device Sequence Tagger using Multi-task Learning

Kumari

Agarwal

Challa

et al. 2021

2021 IEEE 15th International Conference on Semantic Computing (ICSC)

View full text Add to dashboard Cite

Named entity detection and Parts-of-speech tagging are the key tasks for many NLP applications. Although the current state of the art methods achieved near perfection for long, formal, structured text there are hindrances in deploying these models on memory-constrained devices such as mobile phones. Furthermore, the performance of these models is degraded when they encounter short, informal, and casual conversations. To overcome these difficulties, we present LiteMuL -a lightweight on-device sequence tagger that can efficiently process the user conversations using a Multi-Task Learning (MTL) approach. To the best of our knowledge, the proposed model is the first on-device MTL neural model for sequence tagging. Our LiteMuL model is about 2.39 MB in size and achieved an accuracy of 0.9433 (for NER), 0.9090 (for POS) on the CoNLL 2003 dataset. The proposed LiteMuL not only outperforms the current state of the art results but also surpasses the results of our proposed on-device task-specific models, with accuracy gains of up to 11% and model-size reduction by 50%-56%. Our model is competitive with other MTL approaches for NER and POS tasks while outshines them with a low memory footprint. We also evaluated our model on custom-curated user conversations and observed impressive results.

show abstract

“…Sequence Labeling (SL) is a basic paradigm in the natural language processing (NLP) field, which assigns a predefined label to every meaningful unit (e.g., word or character) in a given sequence (He et al, 2020;Lu et al, 2019;Li et al, 2021;. Many NLP tasks belong to this category, such as Named Entity Recognition (NER) (Zhu and Li, 2022;Shen et al, 2022;He and Tang 2022;Zhou et al 2022b), Chinese Word Segmentation (CWS) (Fu et al, 2020;Maimaiti et al, 2021), Part-Of-Speech (POS) tagging (Zhou et al, 2022a;Nguyen and Verspoor, 2018).…”

Section: Introductionmentioning

confidence: 99%

DiffusionSL: Sequence Labeling via Tag Diffusion Process

Huang,

Cao,

Zhao

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Sequence Labeling (SL) is long-standing in Natural Language Processing (NLP). Traditionally, discriminative models have been widely used to capture the conditional distribution of sequence tags, rather than generative models. In this paper, we present DiffusionSL, a framework that utilizes a conditional discrete diffusion model for generating discrete tag data, resulting in a Tag Diffusion Process. We treat the natural language sequence as the conditional signal and the sequence tags as the generation target, iteratively refining the noisy tags to obtain clean ones. To address the discreteness issue, we propose the Bit-Tag Converter (BTConverter) to model the target in continuous data space. Furthermore, we introduce the Bit Diffusion Transformer (BitDiT) to model the process of noise elimination. Leveraging the powerful iterative refinement capability of the diffusion model, DiffusionSL achieves superior performance against previous state-ofthe-art (SOTA) baselines and outperforms gpt-3.5-turbo significantly across multiple benchmark datasets and various tasks 1 .

show abstract

Untitled

Cited by 7 publications

References 35 publications

An Auxiliary Task Boosted Multi-task Learning Method for Service Account Retrieval with Limited Human Annotation

An Auxiliary Task Boosted Multi-task Learning Method for Service Account Retrieval with Limited Human Annotation

LiteMuL: A Lightweight On-Device Sequence Tagger using Multi-task Learning

DiffusionSL: Sequence Labeling via Tag Diffusion Process

Contact Info

Product

Resources

About