Data Updating and Query in Real-Time Data Warehouse System

Zhu, Youchan; Lei, An; Liu, Shuangxi

doi:10.1109/csse.2008.78

Cited by 20 publications

(7 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The authors of [6] suggest a solution based in SOA (Service Oriented Architecture). This approach does data extraction initially, through a web service, and stores the results in caches.…”

Section: To S-dw Loading Timesmentioning

confidence: 99%

“…From the point of view of data integration, constraints and indexes considerably slow the process down, as well as the refreshing of all those structures with the new data. The appropriate solution for these problems in traditional warehouses is to take the whole warehouse offline, disable/drop the constraints and indexes that cause loading slowdown, load the whole data and refresh the datasets, and then rebuild the auxiliary structures and constraints (e.g., [10,12,13,14]). …”

mentioning

confidence: 99%

See 1 more Smart Citation

Real-Time Data Warehousing: A Rewrite/Merge Approach

Cuzzocrea

Ferreira

Furtado

2014

Data Warehousing and Knowledge Discovery

View full text Add to dashboard Cite

Abstract. This paper focuses on Real-Time Data Warehousing systems, a relevant class of Data Warehouses where the main requirement consists in executing classical data warehousing operations (e.g., loading, aggregation, indexing, OLAP query answering, and so forth) under real-time constraints. This makes classical DW architectures not suitable to this goal, and puts the basis for a novel research area which has tight relationship with emerging Cloud architectures. Inspired by this motivation, in this paper we proposed a novel framework for supporting Real-Time Data Warehousing which makes use of a rewrite/merge approach. We also provide an extensive experimental campaign that confirms the benefits deriving from our framework. IntroductionData Warehouses (e.g., [11]) are more and more demanding for high-performance which allow them to deal with real-time paradigms (e.g., [16]), which may turn to be extremely useful in next-generation Big Data research. Indeed, there exists a plethora of emerging applications where Real-Time Data Warehousing plays a leading role, such as: sensor networks, real-time business intelligence, real-time Cloud applications, and so forth. The traditional data warehouse architecture model assumes that new data loading occurs only at certain times, when the warehouse is taken offline, and the data is integrated during a more or less lengthy time interval. This offline procedure is required for three main reasons: there should be no interference between the loading process and the query sessions running on the data warehouse. Therefore, there is no significant slowdown. Looking at data formats, a data warehouse is typically a set of interconnected data marts, schemas (stars), with constraints (e.g., foreign keys, not null constraints, primary keys), lots of indexes (e.g., b-trees, bitmap indexes), materialized views and other summary or derived data, which are created to speedup query answering. From the point of view of data integration, constraints and indexes considerably slow the process down, as well as the refreshing of all those structures with the new data. The appropriate solution for these problems in traditional warehouses is to take the whole warehouse offline, disable/drop the constraints and indexes that cause loading slowdown, load the whole data and refresh the datasets, and then rebuild the auxiliary structures and constraints (e.g., [10,12,13,14]).

show abstract

“…The authors of [6] suggest a solution based in SOA (Service Oriented Architecture). This approach does data extraction initially, through a web service, and stores the results in caches.…”

Section: To S-dw Loading Timesmentioning

confidence: 99%

mentioning

confidence: 99%

Real-Time Data Warehousing: A Rewrite/Merge Approach

Cuzzocrea

Ferreira

Furtado

2014

Data Warehousing and Knowledge Discovery

View full text Add to dashboard Cite

show abstract

“…XML form can also be used as the intermediate data form in data transmission process. It provides a unified data accessing, transmission format [5]. And it shields the difference of the data formats stored in the different relational databases; facilitates the integration of the heterogeneous data source [5].…”

Section: Application Of Xml In Real-time Data Warehouse With Multi-lementioning

confidence: 99%

“…Reference [5] furthered this contention and developed a method of using Web Services as the base data capture structure. Being intermediate data format, XML plays an important role in the data transformation of Web Services.…”

Section: Application Of Xml In Real-time Data Warehouse With Multi-lementioning

confidence: 99%

Application of XML in real-time data warehouse

et al. 2009

View full text Add to dashboard Cite

At present, XML is one of the most widely-used technologies of data-describing and data-exchanging, and the needs for real-time data make real-time data warehouse a popular area in the research of data warehouse. What effects can we have if we apply XML technology to the research of real-time data warehouse? XML technology solves many technologic problems which are impossible to be addressed in traditional real-time data warehouse, and realize the integration of OLAP (On-line Analytical Processing) and OLTP (Online transaction processing) environment. Then real-time data warehouse can truly be called "real time".

show abstract

“…In [3] author discussed the technique in order to implement real-t ime data warehouse. Based on Service Oriented Architecture (SOA) author presented practical real-t ime data warehouse architecture.…”

Section: B Efforts Related To Semi Real Time Etlmentioning

confidence: 99%

Data Load Distribution by Semi Real Time Data Warehouse

Javed¹,

Nawaz²

2010

2010 Second International Conference on Computer and Network Technology

View full text Add to dashboard Cite

Today many organizations used data warehouse for strategic decision making. Today's realtime business stresses the potential to process increasingly volumes of data at very high speed in order to stay competitive in market. Data Warehouse is populated by data extraction, transformation and loading from different data sources by software utilities called ETL (Extraction, transformation & loading). ETL process is a time consuming process as it has to process large volume of data. ETL processes must have certain completion time window and ETL process must have to finish within this time window. In this paper we discusses a technique to distribute the volume of data to be extracted, transformed and loaded into data warehouse by merging both conventional and real-time techniques, so ETL process finishes its job within its time window by utilizing ETL idle time.

show abstract

Data Updating and Query in Real-Time Data Warehouse System

Cited by 20 publications

References 0 publications

Real-Time Data Warehousing: A Rewrite/Merge Approach

Real-Time Data Warehousing: A Rewrite/Merge Approach

Application of XML in real-time data warehouse

Data Load Distribution by Semi Real Time Data Warehouse

Contact Info

Product

Resources

About