A shared context approach for supporting experts in data ETL (Extraction, Transformation and Loading) processes

Tahir, Hassane; Brézillon, Patrick

doi:10.1109/isda.2011.6121741

Cited by 7 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, it failed to handle the whole ETL development lifecycle. Tahir and Brézillon [19] presented how to contextualize ETL process based on different expert viewpoints. They showed how making shared context explicit can help improve the ETL process and avoid conflicts between experts having different viewpoints.…”

Section: A Process-based Etl Approachesmentioning

confidence: 99%

An Integrated Approach for Massive Sequential Data Processing in Civil Infrastructure Operation and Maintenance

Liu

et al. 2018

IEEE Access

View full text Add to dashboard Cite

This paper presents an extract-transform-load (ETL) approach based on multilayer task execution for processing massive sequential data collected from infrastructure operation and maintenance. The proposed approach consists of ETL task partition, execution mode selection, and ETL modeling. The task partition focuses on dividing the ETL process into four tasks to be executed in accordance with different organizational forms of data. Sequenced or non-sequenced load mode is optional, which is independent of the data standardization. In addition, the ETL modeling phase implements conceptual, logical, and physical modeling for the multi-dimensional model. Our main objective is to integrate massive sequential data, enhancing decision-making performance for the intelligent management platform. Traffic data for two years were collected from various systems and acquisition tools of different providers to evaluate the data integration capability of the proposed approach. Furthermore, Kettle software was used to perform transformation and job modules for the multilayer tasks. In addition, a machine learning algorithm was used to generate traffic warning in the tunnels based on the integrated data. The proposed approach is promising for the management and analysis of massive sequential data generated in operation, the maintenance of transportation tunnels, and effective decision-making.INDEX TERMS Civil infrastructure, massive data integration, sequential analysis, maintenance and operation management.

show abstract

Section: A Process-based Etl Approachesmentioning

confidence: 99%

An Integrated Approach for Massive Sequential Data Processing in Civil Infrastructure Operation and Maintenance

Liu

et al. 2018

IEEE Access

View full text Add to dashboard Cite

show abstract

“…These unstructured data are often related with the DW entities and relationships, constituting the context. Meanwhile, in [30] the authors consider the context in the ETL stage, grouping the users that share interests in this stage and contextualizing the ETL process according to the different points of views of the experts.…”

Section: How Contexts Are Defined and Used In Dws?mentioning

confidence: 99%

“…For example, the works in [19][21] [27] focus on data quality in DW sources, and the works in [21][44] [63] focus on quality, especially data cleaning, in ETL processes. In particular, the works in [30][64] focus on the contexts of the ETL processes and the data sources, respectively.…”

Section: Dq Assessment In Dws Based On Contextsmentioning

confidence: 99%

Context-based Data Quality Metrics in Data Warehouse Systems

Serra¹,

Marotta²

2017

CLEIej

View full text Add to dashboard Cite

The fact that Data Quality (DQ) depends on the context, in which data are produced, stored and used, is widely recognized in the research community. Data Warehouse Systems (DWS), whose main goal is to give support to decision making based on data, have had a huge growth in the last years, in research and industry. DQ in this kind of systems becomes essential. This work presents a proposal for identifying DQ problems in the domain of DWS, considering the different contexts that exist in each system component. This proposal may act as a first conceptual framework that guides the DQ-responsible in the management of DQ in DWS. The main contributions of this work are a thorough literature review about how contexts are used for evaluating DQ in DWS, and a proposal for assessing DQ in DWS through contextbased DQ metrics.

show abstract

Data quality in data warehouse systems: A context-based approach

Serra

Marotta

2016

2016 XLII Latin American Computing Conference (CLEI)

View full text Add to dashboard Cite

A shared context approach for supporting experts in data ETL (Extraction, Transformation and Loading) processes

Cited by 7 publications

References 12 publications

An Integrated Approach for Massive Sequential Data Processing in Civil Infrastructure Operation and Maintenance

An Integrated Approach for Massive Sequential Data Processing in Civil Infrastructure Operation and Maintenance

Context-based Data Quality Metrics in Data Warehouse Systems

Data quality in data warehouse systems: A context-based approach

Contact Info

Product

Resources

About