Ember: No-Code Context Enrichment via Similarity-Based Keyless Joins

Abstract

Structured data, or data that adheres to a pre-defined schema, can suffer from fragmented context: information describing a single entity can be scattered across multiple datasets or tables tailored for specific business needs, with no explicit linking keys (e.g., primary key-foreign key relationships or heuristic functions). Context enrichment, or rebuilding fragmented context, using keyless joins is an implicit or explicit step in machine learning (ML) pipelines over structured data sources. This process is tedious, domain-specific, and lacks support in now-prevalent no-code ML systems that let users create ML pipelines using just input data and high-level configuration files. In response, we propose Ember, a system that abstracts and automates keyless joins to generalize context enrichment. Our key insight is that Ember can enable a general keyless join operator by constructing an index populated with task-specific embeddings. Ember learns these embeddings by leveraging Transformer-based representation learning techniques. We describe our core architectural principles and operators when developing Ember, and empirically demonstrate that Ember allows users to develop nocode pipelines for five domains, including search, recommendation and question answering, and can exceed alternatives by up to 39% recall, with as little as a single line configuration change.

Data augmentation: A comprehensive survey of modern approaches

“…Techniques for data integration [9], [10], [11], [36], [39], [40], [41], [42] generally aim to automatically discover, select and aggregate related data in order to extend a given dataset. Many of the approaches deal with tabular data.…”

“…Neural style transfer [6], generative modeling techniques such as variational autoencoders (VAEs) [7] and generative adversarial networks (GANs) [8] have also been extensively used to generate synthetic data for training deep learning models. Another way to augment training data is by integrating existing data from several sources (e.g., in [9], [10], [11]). This is a useful way to leverage the large quantities of data available in various forms on the internet and other sources.…”

From BERT to GPT-3 codex

“…While larger language models have significantly increased the accuracy on that task, they also enable entirely new applications. Here, the tutorial will cover recent research leveraging language models for tasks such as data preparation and integration [2,74,75], fact checking from data [10, 25, 33-40, 81, 82], or database tuning [78][79][80][85][86][87].…”

“…Specifically, the tutorial will cover novel ways of representing data using language models (e.g., by storing data as natural language facts [77] or by integrating data within the language model [26]). Also, it will discuss the use of language models in the execution engine (e.g., to implement operators [74,77] or to synthesize code for data processing [84]).…”

CodexDB: Generating Code for Processing SQL Queries using GPT-3 Codex

“…CodexDB relates to prior work exploiting machine learning [6,7,13] and specifically Transformers [20,21] in the context of database systems. It connects broadly to prior work using GPT-3 for program synthesis [5,11,12].…”