Learning Generalized Linear Models Over Normalized Data

Abstract

Enterprise data analytics is a booming area in the data management industry. Many companies are racing to develop toolkits that closely integrate statistical and machine learning techniques with data management systems. Almost all such toolkits assume that the input to a learning algorithm is a single table. However, most relational datasets are not stored as single tables due to normalization. Thus, analysts often perform key-foreign key joins before learning on the join output. This strategy of learning after joins introduces redundancy avoided by normalization, which could lead to poorer end-to-end performance and maintenance overheads due to data duplication. In this work, we take a step towards enabling and optimizing learning over joins for a common class of machine learning techniques called generalized linear models that are solved using gradient descent algorithms in an RDBMS setting. We present alternative approaches to learn over a join that are easy to implement over existing RDBMSs. We introduce a new approach named factorized learning that pushes ML computations through joins and avoids redundancy in both I/O and computations. We study the tradeoff space for all our approaches both analytically and empirically. Our results show that factorized learning is often substantially faster than the alternatives, but is not always the fastest, necessitating a cost-based approach. We also discuss extensions of all our approaches to multi-table joins as well as to Hive.

On Functional Aggregate Queries with Additive Inequalities

“…In-database machine learning algorithms is a growing class of algorithms that aims to learn in time sublinear in the input data a.k.a. the design matrix [22,2,11,3,18,19]. The trick is that the design matrix J often happens to be the output of some database query Q whose size could be much larger than the size of its input tables T 1 , .…”

“…Example: Let ǫ = 1 and A be an array having the weights (10,12,13,14,15,16,17,18,19), then the ǫ-sketch of A, denoted A ′ , will be an array that has the weights of indices (1,2,4,8) in A (the weights of index 16 or higher are assumed to be ∞); thus, A ′ = {10, 12, 14, 14, 18, 18, 18, 18}.…”

A Layered Aggregate Engine for Analytics Workloads

“…Further examples in this category are: Orion [30] and Hamlet [31], which support generalized linear models and Naïve Bayes classi cation; recent e orts on scaling linear algebra using existing distributed database systems [32]; the declarative language BUDS [20], whose compiler can perform deep optimizations of the user's program; and Morpheus [14]. Morpheus factorizes the computation of linear algebra operators summation, matrix-multiplication, pseudo-inverse, and element-wise operations over training datasets de ned by key-foreign key star or chain joins.…”

Helix

“…The database community has identified various opportunities for optimizing DPR. Several approaches identify as a key bottleneck in DPR and optimize it [37,15,49,38]. Kumar et al [37] optimizes generalized linear models directly over factorized / normalized representations of relational data, avoiding key-foreign key joins.…”