Data engineering approach to efficient data warehouse: Life cycle development revisited

Daneshpour, Negin; Barfourosh, Ahmad Abdollahzadeh

doi:10.1109/csicsse.2011.5963983

Cited by 2 publications

(2 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a typical DW project ETL consumes a large fraction of time, money and effort to do the correct functionality and adequate performance. Due to different applications objectives and characteristic of their data, to maintain ETL will be a main concern [7].…”

Section: Fig: 1 Data Mining Enhanced (Idss) For Better Dssmentioning

confidence: 99%

“…Data scrubbing is the major starting step to have a high quality data warehouse to make valid decisions in decision support systems in a reasonable time. Data scrubbing some time called Data Cleaning or Data Cleansing.Some difference between Data Cleaning and Data Cleansing describe in [7], Figure 2 illustrated Data Scrubbing and ETL tools. …”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Comparison Study of Data Scrubbing Algorithms and Frameworks in Data Warehousing

Housien¹,

Zhang²,

Abdulhadi³

2013

IJCA

View full text Add to dashboard Cite

In these days, many organizations tend to use a Data Warehouse to meet the requirements to develop decisionmaking processes and achieve their goals better and satisfy their customers. It enables Executives to access the information they need in a timely manner for making the right decision for any work. Decision Support System (DSS) is one of the means that applied in data mining . Its robust and better decision depends on an important and conclusive factor called Data Quality (DQ), to obtain a high data quality using Data Scrubbing (DS) which is one of data Extraction Transformation and Loading (ETL) tools. Data Scrubbing is very important and necessary in the Data Warehouse (DW). There are growing relationships to get high DQ and effective DS. The use of DS algorithms is a solution to the constraints that limit the DQ which leads to weak decisions and the burden of the high financial costs. These constraints are: dirty data, noise data, missing values, inconsistency, uncertain data, ambiguous, conflicting, duplicated records and similar columns. The Sources and causes of these constraints are many, including: input error, merge data from different sources, difference in representing the same information, etc. In addition there are more than 35 sources and causes of the poor-quality data that arise at the stage of the ETL process. This paper present comparison and analysis for DS algorithms and the pros and cons of each algorithm, accuracy and time complexity. Additionally, it present a comparative and analysis of the Data Scrubbing Frameworks and determine the best framework.

show abstract

Section: Fig: 1 Data Mining Enhanced (Idss) For Better Dssmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%