Applying Machine Learning Models to Scalable DataFrames with Grizzly

Kläbe, Steffen; Hagedorn, Stefan

doi:10.18420/btw2021-10

2021

DOI: 10.18420/btw2021-10

|View full text |Cite

Applying Machine Learning Models to Scalable DataFrames with Grizzly

Steffen Kläbe¹,

Stefan Hagedorn²

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Article1

Relationship

Self Cite0

Independent1

Authors

Journals

Cited by 1 publication

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

InferDB: In-Database Machine Learning Inference Using Indexes

Salazar-Díaz,

Glavic,

Rabl

2024

Proc. VLDB Endow.

View full text Add to dashboard Cite

The performance of inference with machine learning (ML) models and its integration with analytical query processing have become critical bottlenecks for data analysis in many organizations. An ML inference pipeline typically consists of a preprocessing workflow followed by prediction with an ML model. Current approaches for in-database inference implement preprocessing operators and ML algorithms in the database either natively, by transpiling code to SQL, or by executing user-defined functions in guest languages such as Python. In this work, we present a radically different approach that approximates an end-to-end inference pipeline (preprocessing plus prediction) using a light-weight embedding that discretizes a carefully selected subset of the input features and an index that maps data points in the embedding space to aggregated predictions of an ML model. We replace a complex preprocessing workflow and model-based inference with a simple feature transformation and an index lookup. Our framework improves inference latency by several orders of magnitude while maintaining similar prediction accuracy compared to the pipeline it approximates.

show abstract

InferDB: In-Database Machine Learning Inference Using Indexes

Salazar-Díaz,

Glavic,

Rabl

2024

Proc. VLDB Endow.

View full text Add to dashboard Cite

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Applying Machine Learning Models to Scalable DataFrames with Grizzly

Cited by 1 publication

References 0 publications

InferDB: In-Database Machine Learning Inference Using Indexes

InferDB: In-Database Machine Learning Inference Using Indexes

Contact Info

Product

Resources

About