SAVIME: A Database Management System for Simulation Data Analysis and Visualization

Lustosa, Hermano; Porto, Fábio; Valduriez, Patrick

doi:10.5753/sbbd.2019.8810

Cited by 2 publications

(2 citation statements)

References 10 publications

(10 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have integrated the DJEnsemble approach into SAV-IME, an in-memory columnar multidimensional array data management system [20]. Given its multidimensional array data model, SAVIME suits storing spatio-temporal datasets, such as the meteorology dataset discussed in section 6.1.1.…”

Section: Savimementioning

confidence: 99%

DJEnsemble: On the Selection of a Disjoint Ensemble of Deep Learning Black-Box Spatio-Temporal Models

Souto,

Pereira,

Zorrilla

et al. 2020

Preprint

View full text Add to dashboard Cite

Consider a set of black-box models -each of them independently trained on a different dataset -answering the same predictive spatio-temporal query. Being built in isolation, each model traverses its own life-cycle until it is deployed to production. As such, these competitive models learn data patterns from different datasets and face independent hyperparameter tuning. In order to answer the query, the set of black-box predictors has to be ensembled and allocated to the spatio-temporal query region. However, computing an optimal ensemble is a complex task that involves selecting the appropriate models and defining an efficient allocation function that maps the model frame to the query region.In this paper, we present a cost-based approach for the automatic selection and allocation of a disjoint ensemble of black-box predictors to answer predictive spatio-temporal queries. Our approach is divided into two parts-offline and online. During the offline part, we preprocess the predictive domain data -transforming it into a regular grid -and the black-box models-computing their spatio-temporal learning function. In the online part, we compute a DJEnsemble plan which minimizes a multivariate cost function based on estimates for the prediction error and the execution cost -producing a model spatial allocation matrix -and run the optimal ensemble plan. We conduct a set of extensive experiments that evaluate the DJEnsemble approach and highlight its efficiency. We show that our cost model produces plans with performance close to the actual best plan. When compared against the traditional ensemble approach, DJEnsemble achieves up to 4X improvement in execution time and almost 9X improvement in prediction accuracy. To the best of our knowledge, this is the first work to solve the problem of optimizing the allocation of black-box models to answer predictive spatio-temporal queries.

show abstract

Section: Savimementioning

confidence: 99%

DJEnsemble: On the Selection of a Disjoint Ensemble of Deep Learning Black-Box Spatio-Temporal Models

Souto,

Pereira,

Zorrilla

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…This paper is an extended version of the work presented in [Lustosa et al 2019], and includes the ML feature not covered in the original. It is organized as follows.…”

Section: Introductionmentioning

confidence: 99%

SAVIME: An Array DBMS for Simulation Analysis and ML Models Prediction

Lustosa

Silva

et al. 2021

jidm

Self Cite

View full text Add to dashboard Cite

Limitations in current DBMSs prevent their wide adoption in scientific applications. In order to make them benefit from DBMS support, enabling Declarative data analysis and visualization over scientific data, we present an in-memory array DBMS system called SAVIME. In this work we describe the system SAVIME, along with its data model. Our preliminary evaluation show how SAVIME, by using a simple storage definition language (SDL) can outperform the state-of-the-art array database system, SciDB, during the process of data ingestion. We also show that it is possible to use SAVIME as a storage alternative for a numerical solver without affecting its scalability, making it useful for modern ML based applications.

show abstract

SAVIME: A Database Management System for Simulation Data Analysis and Visualization

Cited by 2 publications

References 10 publications

DJEnsemble: On the Selection of a Disjoint Ensemble of Deep Learning Black-Box Spatio-Temporal Models

DJEnsemble: On the Selection of a Disjoint Ensemble of Deep Learning Black-Box Spatio-Temporal Models

SAVIME: An Array DBMS for Simulation Analysis and ML Models Prediction

Contact Info

Product

Resources

About