LETS: A Label-Efficient Training Scheme for Aspect-Based Sentiment Analysis by Using a Pre-Trained Language Model

Abstract

Recently proposed pre-trained language models can be easily fine-tuned to a wide range of downstream tasks. However, a large-scale labelled task-specific dataset is required for fine-tuning creating a bottleneck in the development process of machine learning applications. To foster a fast development by reducing manual labelling efforts, we propose a Label-Efficient Training Scheme (LETS). The proposed LETS consists of three elements: (i) task-specific pre-training to exploit unlabelled task-specific corpus data, (ii) label augmentation to maximise the utility of labelled data, and (iii) active learning to label data strategically. In this paper, we apply LETS to a novel aspect-based sentiment analysis (ABSA) use-case for analysing the reviews of the health-related program supporting people to improve their sleep quality. We validate the proposed LETS on a custom health-related program-reviews dataset and another ABSA benchmark dataset. Experimental results show that the LETS can reduce manual labelling efforts 2-3 times compared to labelling with random sampling on both datasets. The LETS also outperforms other state-of-the-art active learning methods. Furthermore, the experimental results show that LETS can contribute to better generalisability with both datasets compared to other methods thanks to the task-specific pre-training and the proposed label augmentation. We expect this work could contribute to the natural language processing (NLP) domain by addressing the issue of the high cost of manually labelling data. Also, our work could contribute to the healthcare domain by introducing a new potential application of NLP techniques.

Interdisciplinary research unlocking innovative solutions in healthcare

“…Another data-efficient learning strategy is proposed to not only reduce manual labelling efforts effectively but also maximise the utility of data (Shim et al, 2021). The proposed learning strategy consist of three elements: (i) task-specific pre-training to exploit unlabelled task-specific corpus data, (ii) label augmentation to maximise the utility of labelled data and (iii) active learning to label data strategically.…”

A Study of Speech Recognition with Deep Learning

“…Sentiment analysis can use a piece of text to predict whether the students' feelings towards the teacher in class are positive or negative, and then the teacher and school can see if the teacher is having problems or whether the students' attitudes towards the teacher have changed for the better or worse over time. One of the challenges with sentiment analysis is the lack of a particularly large labeled training set in [16], but with the use of word embeddings, it is possible to build a good sentiment analyzer relying on a moderately sized labeled training set in [17]. For sentiment analysis tasks, it is common that the training set may have between 10,000 and 100,000 words of data.…”

Application of the Deep Pretrained Language Model Processing Method in Social Network Sentiment Analysis

“…Sentiment analysis of social networks, also known as opinion mining, uses natural language processing technology to mine users' views, attitudes, and emotions in social networks [1]. Many researchers have conducted sentiment analysis on the contents of social networking sites [2] and have mined the emotions and potential opinions of users on social networking sites.…”