Leveraging organizational resources to adapt models to new data modalities

Abstract

As applications in large organizations evolve, the machine learning (ML) models that power them must adapt the same predictive tasks to newly arising data modalities (e.g., a new video content launch in a social media application requires existing text or image models to extend to video). To solve this problem, organizations typically create ML pipelines from scratch. However, this fails to utilize the domain expertise and data they have cultivated from developing tasks for existing modalities. We demonstrate how organizational resources, in the form of aggregate statistics, knowledge bases, and existing services that operate over related tasks, enable teams to construct a common feature space that connects new and existing data modalities. This allows teams to apply methods for data curation (e.g., weak supervision and label propagation) and model training (e.g., forms of multimodal learning) across these different data modalities. We study how this use of organizational resources composes at production scale in over 5 classification tasks at Google, and demonstrate how it reduces the time needed to develop models for new modalities from months to weeks or days.

TagRuler: Interactive Tool for Span-Level Data Programming by Demonstration

“…We implement semantic rules via similarity based on contextual token embeddings (BERT [5] and ELMo [16]). In addition to the active learning approach previously implemented in Ruler, TagRuler proposes a novel active learning component focusing on false positives, thus contributing to one of the main challenges in data programming: surfacing difficult (or borderline) labeled examples [20]. In this active learning approach, unlabeled examples that have higher potential to identify false positives will have higher probability to be sampled as next instance to be labeled.…”

“…TagRuler is a system designed to learn from expert knowledge, and expert manual annotation is expensive, so it is important to obtain informative data with as few annotations as possible. The active learning approach focuses on contributing to this main challenge in data programming: generating difficult (or borderline) examples [20]. TagRuler samples the text to be displayed in Figure 2 (A) after each annotation using an active learning technique that leverages the trained label model and a small labeled development set.…”