Synchronization Options for Data Warehouse Designs

Italiano, Isabel Cristina; Ferreira, João Eduardo

doi:10.1109/mc.2006.104

Cited by 23 publications

(7 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Whilethefrequencyofupdatescanbeincreasedtohourlyormore,theriskofnetworkcongestion or unreliability becomes even greater. Italiano and Ferreira (2006) point out that the method of updatingallstoreddataatregularperiodsmaynolongerbeavalidapproachduetodifferentbusiness modelsthathavetheirownupdaterequirementswhichcanchangeovertime.Therearealsodata warehouseenvironments,suchasforU.S.HomelandSecurityandterrorismtracking,wheremore stochasticmethodsmaybeappropriateduetheextremelylargenumberofdisparatesourcesofdata (Kim,2005).Therefore,ourproposedsolutionisastochasticcompromisebetweenreal-timeand batchupdatemethods.…”

Section: Literature Reviewmentioning

confidence: 99%

An Efficient Stochastic Update Propagation Method in Data Warehousing

Bordoloi

Kapoor

Jacks

2018

Journal of Database Management

View full text Add to dashboard Cite

This paper develops an approach to update propagation for data warehousing environments where data storage (and retrieval) is required as a sum of data at individual source nodes in an organization's distributed information system network. Our procedure is expected to result in less network traffic (as compared with the real-time method) due to update propagation required because of changes in source data, in particular due to Big Data. The method allows system users to place limits on the discrepancy between the source data and the data in the data warehouse which could result due to a time lag between source data changes and the update operation. Finally, the pre-specified limits on the discrepancy are maintained while accounting for two crucial factors in distributed systems: the fact that some nodes are situated on more congested network links and that some of the links on the network are less reliable than others.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

An Efficient Stochastic Update Propagation Method in Data Warehousing

Bordoloi

Kapoor

Jacks

2018

Journal of Database Management

View full text Add to dashboard Cite

show abstract

“…The second research stream focuses on improving data timeliness by quickly delivering the data changes or the requested information quickly to information users. The second stream features mainly study on how to improve the efficiency of update policies [10,17,18], query processing algorithms [9,19,20], and system response speed [21,22].…”

Section: A Current Research On Data Timelinessmentioning

confidence: 99%

A Markov-Based Update Policy for Constantly Changing Database Systems

Zong

Jiang

2017

IEEE Trans. Eng. Manage.

View full text Add to dashboard Cite

In order to maximize the value of an organization's data assets, it is important to keep data in its databases upto-date. In the era of big data, however, constantly changing data sources make it a challenging task to assure data timeliness in enterprise systems. For instance, due to the high frequency of purchase transactions, purchase data stored in an enterprise resource planning system can easily become outdated, affecting the accuracy of inventory data and the quality of inventory replenishment decisions. Despite the importance of data timeliness, updating a database as soon as new data arrives is typically not optimal because of high update cost. Therefore, a critical problem in this context is to determine the optimal update policy for database systems. In this study, we develop a Markov decision process model, solved via dynamic programming, to derive the optimal update policy that minimizes the sum of data staleness cost and update cost. Based on realworld enterprise data, we conduct experiments to evaluate the performance of the proposed update policy in relation to benchmark policies analyzed in the prior literature. The experimental results show that the proposed update policy outperforms fixed interval update policies and can lead to significant cost savings.

show abstract

“…In literature, most of the work has been done on this approach. Tools proposed in other studies (Bouzeghoub et al, 1999;Vassiliadis et al, 2001;Simitsis et al, 2005;Italiano & Ferreira, 2006;Santos & Bernardio, 2008) are built on this approach. ETL-DDF provides a RTDW solution that is based on the MCM incremental maintenance for directly inserting real-time data into the MCM (Dayal et al, 2009).…”

Section: Related Workmentioning

confidence: 99%

The data warehouse virtualization framework for operational business intelligence

Farooq

2012

Expert Systems

View full text Add to dashboard Cite

In order to explore the most current information and react faster to changing business conditions, organizations consider real‐time data warehousing a powerful technique to achieve operational business intelligence (BI). We propose in this paper a novel real‐time data warehouse (RTDW) framework based on the virtualization concept. Our approach introduces a conceptual modelling technique, known as ring modelling, for real‐time data management and multidimensional analysis. This technique produces a flexible semi‐structured data model that accommodates unknown business process data and relationships as they evolve, handles schema changes and aggregate‐management efficiently, and scales well with the large size of increasing data volumes. With the help of a telecommunication business example, We evaluated our proposed approach in an extensive experimental study where we compared our approach Ring Model with existing structured multidimensional conceptual models (MCMs), i.e. relational OLAP and multidimensional OLAP, and with semi‐structured MCM, i.e. XML Cubes, in terms of scalability, data storage estimations, data updates loading time, and query response times. Our performance results show that encouraging speedups are achieved.

show abstract

Synchronization Options for Data Warehouse Designs

Cited by 23 publications

References 12 publications

An Efficient Stochastic Update Propagation Method in Data Warehousing

An Efficient Stochastic Update Propagation Method in Data Warehousing

A Markov-Based Update Policy for Constantly Changing Database Systems

The data warehouse virtualization framework for operational business intelligence

Contact Info

Product

Resources

About