WeSAL: Applying Active Supervision to Find High-quality Labels at Industrial Scale

Nashaat, Mona; Ghosh, Aindrila; Miller, James H.; Quader, Shaikh

doi:10.24251/hicss.2020.028

Cited by 5 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, motivated by the shortcomings of these approaches, in this article, we present ArtAI an improved prediction scheme to predict the long-time popularity of news articles without the need for ground-truth observations. The scheme extends our previous work [11], which is a labeling framework that combines Weak Supervision with Active Learning to create large-scale, high-quality training data. However, ArtAI applies a novel selection policy to engage the end-users in the process.…”

Section: Introductionmentioning

confidence: 78%

Improving News Popularity Estimation via Weak Supervision and Meta-active Learning

Nashaat¹,

Miller²

2021

Proceedings of the Annual Hawaii International Conference on System Sciences

Self Cite

View full text Add to dashboard Cite

Social news has fundamentally changed the mechanisms of public perception, education, and even dis-information. Apprising the popularity of social news articles can have significant impact through a diversity of information redistribution techniques. In this article, an improved prediction algorithm is proposed to predict the long-time popularity of social news articles without the need for ground-truth observations. The proposed framework applies a novel active learning selection policy to obtain the optimal volume of observations and achieve superior predictive performance. To assess the proposed framework, a large set of experiments are undertaken; these indicate that the new solution can improve prediction performance by 28% (precision) while reducing the volume of required ground truth by 32%.

show abstract

Section: Introductionmentioning

confidence: 78%

Improving News Popularity Estimation via Weak Supervision and Meta-active Learning

Nashaat¹,

Miller²

2021

Proceedings of the Annual Hawaii International Conference on System Sciences

Self Cite

View full text Add to dashboard Cite

show abstract

“…Previous work has proposed a variety of methods for giving users (who are in our case the product moderators) control over classifiers by making use of a pipeline that allows them to provide feedback about training data labels and classification results. In WeSAL (Nashaat et al, 2018(Nashaat et al, , 2020 user feedback improves the labels that sets of rules assign to data points. In contrast, our focus is on feedback that allows moderators to improve the rules directly.…”

Section: Related Workmentioning

confidence: 99%

Exploring Inspiration Sets in a Data Programming Pipeline for Product Moderation

Winkler¹,

Brugman²,

Berkel³

et al. 2021

Proceedings of the 4th Workshop on E-Commerce and NLP

View full text Add to dashboard Cite

We carry out a case study on the use of data programming to create data to train classifiers used for product moderation on a large e-commerce platform. Data programming is a recently-introduced technique that uses human-defined rules to generate training data sets without tedious item-by-item hand labeling. Our study investigates methods for allowing product moderators to quickly modify the rules given their knowledge of the domain and, especially, of textual item descriptions. Our results show promise that moderators can use this approach to steer the training data, making possible fast and close control of classifiers that detect policy violations.

show abstract

“…In the investigations presented by Biegel et al (2021) and Nashaat et al (2020), DP is integrated with AL, resulting in enhanced performance across various domains, including visual relationship analysis, spam detection, renewal sales prediction, bank operations, credit card fraud detection, occupancy sensing, and MNIST classification [30,31]. However, the application of AL in combination with DP for TS data has yet to be explored in the literature.…”

Section: Introductionmentioning

confidence: 99%

An active learning enhanced data programming (ActDP) framework for ECG time series

Gupta,

Kumar

2024

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

Supervised machine learning requires the estimation of multiple parameters by using large amounts of labelled data. Getting labelled data generally requires a substantial allocation of resources in terms of both cost and time. In such scenarios, weak supervised learning techniques like data programming (DP) and active learning (AL) can be advantageous for time-series classification tasks. These paradigms can be used to assign data labels in an automated manner, and time-series classification can subsequently be carried out on the labelled data. This work proposes a novel framework titled active learning enhanced data programming (ActDP). It uses DP and AL for ECG classification using single-lead data. ECG classification is pivotal in cardiology and healthcare for diagnosing a broad spectrum of heart conditions and arrhythmias. To establish the usefulness of this proposed ActDP framework, the experiments have been conducted using the MIT-BIH dataset with 94,224 ECG beats. DP assigns a probabilistic label to each ECG beat using nine novel polar labelling functions and a generative model in this work. Further, AL improves the result of DP by replacing the labels for sampled ECG beats of a generative model with ground truth. Subsequently, a discriminative model is trained on these labels for each iteration. The experimental results show that by incorporating AL to DP in the ActDP framework, the accuracy of ECG classification strictly increases from 85.7 % to 97.34 % in 58 iterations. Comparatively, the proposed framework (ActDP) has demonstrated a higher classification accuracy of 97.34 % In contrast, DP with data augmentation (DA) achieves an accuracy of 92.2 %, while DP without DA results in an accuracy of 85.7 %, majority vote yields an accuracy of 50.2 %, and the generative model achieves an accuracy of only 66.5 %.

show abstract

WeSAL: Applying Active Supervision to Find High-quality Labels at Industrial Scale

Cited by 5 publications

References 18 publications

Improving News Popularity Estimation via Weak Supervision and Meta-active Learning

Improving News Popularity Estimation via Weak Supervision and Meta-active Learning

Exploring Inspiration Sets in a Data Programming Pipeline for Product Moderation

An active learning enhanced data programming (ActDP) framework for ECG time series

Contact Info

Product

Resources

About