Towards Self-Adapting Data Migration in the Context of Schema Evolution in NoSQL Databases

Hillenbrand, Andrea; Störl, Uta; Levchenko, Maksym; Nabiyev, Shamil; Klettke, Meike

doi:10.1109/icdew49219.2020.00013

Cited by 11 publications

(6 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further research is focusing now on applying the gained insights in order to develop a self-adaptive migration strategy which realizes the heuristics on the level of a migration strategy. This self-adaptive migration strategy could be parameterized according to the preferences on the tradeoff between migration costs and latency and their opportunity costs, based on a theoretical analysis already published in [23]. This appears especially interesting when selfadaptive strategies would be implemented in cloud solutions, which would compete with on-premise solutions in terms of a better cost efficiency, energy reduction, and sustainability.…”

Section: Discussionmentioning

confidence: 99%

“…Completely searching the solution space would by far exceed computational means. We use a probabilistic Monte Carlo method of repeated sampling to investigate the scenario characteristics while bringing the complexity of the problem under control [16,31] We parameterize each scenario and calculate the metrics for each migration strategy by means of the tool-based migration advisor MigCast, which we presented in earlier work [22,23]. Our MigCast tool stands on the tradition of advisors in database technology [6] by maintaining a cost model and forecasting migration costs for alternative data migration strategies.…”

Section: Lazy Strategy Predictive Strategy Incremental Strategymentioning

confidence: 99%

“…MigCast in Monte Carlo runs MigCast repeatedly, a tool-based advisor that we published in earlier work [22,23]. By means of repeated samplings, we contribute a systematic exploration of the search space of the scenario characteristics, thereby clarifying possible and likely impacts of schema evolution on the metrics.…”

Section: Architecture and Approachmentioning

confidence: 99%

“…Interestingly, with the chosen size of the prediction set and thereby the investment of migration costs, the resulting latency of the predictive strategy proportionally fits in relative to the other strategies. A detailed investigation on the correlation of different prediction set sizes and resulting metrics is part of our future research, following up on [23].…”

Section: Workload Intensitymentioning

confidence: 99%

See 3 more Smart Citations

MigCast in Monte Carlo: The Impact of Data Model Evolution in NoSQL Databases

Hillenbrand,

Störl,

Nabiyev

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

During the development of NoSQL-backed software, the data model evolves naturally alongside the application code. Especially in agile development, new application releases are deployed frequently causing schema changes. Eventually, decisions have to be made regarding the migration of versioned legacy data which is persisted in the cloud-hosted production database. We solve this schema evolution problem and present the results of near-exhaustive calculations by means of which software project stakeholders can manage the operative costs for data model evolution and adapt their software release strategy accordingly in order to comply with service-level agreements regarding the competing metrics of migration costs and latency. We clarify conclusively how data model evolution in NoSQL databases impacts the metrics while taking all relevant characteristics of migration scenarios into account. As calculating all possible combinatorics in the search space of migration scenarios would by far exceed computational means, we used a probabilistic Monte Carlo method of repeated sampling, serving as a well-established means to bring the complexity of data model evolution under control. Our experiments show the qualitative and quantitative impact on the performance of migration strategies with respect to intensity and distribution of data entity accesses, the kinds of schema changes, and the characteristics of the underlying data model.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Lazy Strategy Predictive Strategy Incremental Strategymentioning

confidence: 99%

Section: Architecture and Approachmentioning

confidence: 99%

Section: Workload Intensitymentioning

confidence: 99%

See 2 more Smart Citations

MigCast in Monte Carlo: The Impact of Data Model Evolution in NoSQL Databases

Hillenbrand,

Störl,

Nabiyev

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Application architects are adopting NoSQL storage technologies that have been designed to provide good horizontal scalability for data-intensive computing and analytics applications. Not only are architects faced with the curse of choosing the technology [1] that best matches their data model, consistency and availability requirements, yet it is far from trivial to compare the relative impact and trade-offs between different performance and scalability tactics that must be applied when configuring the chosen storage technology [2,3]. Standard benchmarks can help evaluators in testing different storage systems and in facilitating performance comparisons.…”

Section: Introductionmentioning

confidence: 99%

Automated Configuration of NoSQL Performance and Scalability Tactics for Data-Intensive Applications

Preuveneers

Joosen

2020

Informatics

View full text Add to dashboard Cite

This paper presents the architecture, implementation and evaluation of a middleware support layer for NoSQL storage systems. Our middleware automatically selects performance and scalability tactics in terms of application specific workloads. Enterprises are turning to NoSQL storage technologies for their data-intensive computing and analytics applications. Comprehensive benchmarks of different Big Data platforms can help drive decisions which solutions to adopt. However, selecting the best performing technology, configuring the deployment for scalability and tuning parameters at runtime for an optimal service delivery remain challenging tasks, especially when application workloads evolve over time. Our middleware solves this problem at runtime by monitoring the data growth, changes in the read-write-query mix at run-time, as well as other system metrics that are indicative of sub-optimal performance. Our middleware employs supervised machine learning on historic and current monitoring information and corresponding configurations to select the best combinations of high-level tactics and adapt NoSQL systems to evolving workloads. This work has been driven by two real world case studies with different QoS requirements. The evaluation demonstrates that our middleware can adapt to unseen workloads of data-intensive applications, and automate the configuration of different families of NoSQL systems at runtime to optimize the performance and scalability of such applications.

show abstract