Hybridization of Active Learning and Data Programming for Labeling Large Industrial Datasets

Nashaat, Mona; Ghosh, Aindrila; Miller, James; Quader, Shaikh; Marston, Chad; Puget, Jean‐François

doi:10.1109/bigdata.2018.8622459

Cited by 15 publications

(24 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unsupervised learning techniques, such as subspace clustering, have been shown to find influential points from a cluster [51]. A hybrid method that connects active learning and data programming [48] has shown improvements in the reduction of noisy data in large scale workspaces [15]. Similar to our work, active learning approaches [23], [63], [78] have been effective while training biased and highly varied datasets.…”

Section: Related Worksupporting

confidence: 54%

Sampling Approach Matters: Active Learning for Robotic Language Acquisition

Pillai

Raff

Ferraro

et al. 2020

2020 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

Ordering the selection of training data using active learning can lead to improvements in learning efficiently from smaller corpora. We present an exploration of active learning approaches applied to three grounded language problems of varying complexity in order to analyze what methods are suitable for improving data efficiency in learning. We present a method for analyzing the complexity of data in this joint problem space, and report on how characteristics of the underlying task, along with design decisions such as feature selection and classification model, drive the results. We observe that representativeness, along with diversity, is crucial in selecting data samples.

show abstract

Section: Related Worksupporting

confidence: 54%

Sampling Approach Matters: Active Learning for Robotic Language Acquisition

Pillai

Raff

Ferraro

et al. 2020

2020 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

show abstract

“…Previous work has proposed a variety of methods for giving users (who are in our case the product moderators) control over classifiers by making use of a pipeline that allows them to provide feedback about training data labels and classification results. In WeSAL (Nashaat et al, 2018(Nashaat et al, , 2020 user feedback improves the labels that sets of rules assign to data points. In contrast, our focus is on feedback that allows moderators to improve the rules directly.…”

Section: Related Workmentioning

confidence: 99%

Exploring Inspiration Sets in a Data Programming Pipeline for Product Moderation

Winkler¹,

Brugman²,

Berkel³

et al. 2021

Proceedings of the 4th Workshop on E-Commerce and NLP

View full text Add to dashboard Cite

We carry out a case study on the use of data programming to create data to train classifiers used for product moderation on a large e-commerce platform. Data programming is a recently-introduced technique that uses human-defined rules to generate training data sets without tedious item-by-item hand labeling. Our study investigates methods for allowing product moderators to quickly modify the rules given their knowledge of the domain and, especially, of textual item descriptions. Our results show promise that moderators can use this approach to steer the training data, making possible fast and close control of classifiers that detect policy violations.

show abstract

“…These ideas have been used to expand intent training data for conversational agents (Mallinar et al 2019) and to quickly label large industrial data sets (Nashaat et al 2018). We build on these works by using an iterative procedure to automatically construct weak models and show that the Snorkel generative model can select more relevant sentences for labeling than search-based methods alone.…”

Section: Data Programming Weak Supervision and Machine Teachingmentioning

confidence: 99%

Iterative Data Programming for Expanding Text Classification Corpora

Mallinar¹,

Shah

et al. 2020

AAAI

View full text Add to dashboard Cite

Real-world text classification tasks often require many labeled training examples that are expensive to obtain. Recent advancements in machine teaching, specifically the data programming paradigm, facilitate the creation of training data sets quickly via a general framework for building weak models, also known as labeling functions, and denoising them through ensemble learning techniques. We present a fast, simple data programming method for augmenting text data sets by generating neighborhood-based weak models with minimal supervision. Furthermore, our method employs an iterative procedure to identify sparsely distributed examples from large volumes of unlabeled data. The iterative data programming techniques improve newer weak models as more labeled data is confirmed with human-in-loop. We show empirical results on sentence classification tasks, including those from a task of improving intent recognition in conversational agents.

show abstract

Hybridization of Active Learning and Data Programming for Labeling Large Industrial Datasets

Cited by 15 publications

References 38 publications

Sampling Approach Matters: Active Learning for Robotic Language Acquisition

Sampling Approach Matters: Active Learning for Robotic Language Acquisition

Exploring Inspiration Sets in a Data Programming Pipeline for Product Moderation

Iterative Data Programming for Expanding Text Classification Corpora

Contact Info

Product

Resources

About