Materialization Optimizations for Feature Selection Workloads

Abstract

There is an arms race in the data management industry to support analytics, in which one critical step is feature selection, the process of selecting a feature set that will be used to build a statistical model. Analytics is one of the biggest topics in data management, and feature selection is widely regarded as the most critical step of analytics; thus, we argue that managing the feature selection process is a pressing data management challenge. We study this challenge by describing a feature-selection language and a supporting prototype system that builds on top of current industrial, R-integration layers. From our interactions with analysts, we learned that feature selection is an interactive, human-in-the-loop process, which means that feature selection workloads are rife with reuse opportunities. Thus, we study how to materialize portions of this computation using not only classical database materialization optimizations but also methods that have not previously been used in database optimization, including structural decomposition methods (like QR factorization) and warmstart. These new methods have no analog in traditional SQL systems, but they may be interesting for array and scientific database applications. On a diverse set of data sets and programs, we find that traditional database-style approaches that ignore these new opportunities are more than two orders of magnitude slower than an optimal plan in this new tradeoff space across multiple R-backends. Furthermore, we show that it is possible to build a simple cost-based optimizer to automatically select a near-optimal execution plan for feature selection.

Learning Generalized Linear Models Over Normalized Data

“…Application in a System: Recent systems such as Columbus [20,33] and MLBase [21] provide a high-level language that includes both relational and ML operations. Such systems optimize the execution of logical ML computations by choosing among alternative physical plans using cost models.…”

“…There is increasing research and industrial interest in building systems that achieve closer integration of ML with data processing. These include systems that combine linear algebra-based languages with data management platforms [4,15,34], systems for Bayesian inference [9], systems for graph-based ML [23], and systems that combine dataflow-based languages for ML with data management platforms [21,22,33]. None of these systems address the problem of learning over joins, but we think our work is easily applicable to the last group of systems.…”

“…Request permissions from permissions@acm.org. and academia aim to integrate ML capabilities with data processing in RDBMSs, Hadoop, and other systems [2,4,9,15,18,21,22,33,34]. Almost all such implementations of ML algorithms require that the input dataset be a single table.…”

“…Second, as the base tables evolve, maintaining the materialized output of the join could become an overhead. Finally, analysts often perform exploratory analysis of different subsets of features and data [20,33]. Materializing temporary tables after joins for learning on each subset could slow the analyst and inhibit exploration [7].…”

Time-Saving Approach for Optimal Mining of Association Rules

“…There are different types of DML such as: Tasks : ( for further clarification please refer to MLbase [18,21], (fixed task) Columbus [25], DeepDive [20])…”