Revised Framework for ETL Workflow Management for Efficient Business Decision-Making

Abstract

Abstract-Business decision-making is not a simple task. There are many reasons for that but the main reason is data comes from heterogeneous operational sources of an organization. Therefore, it is difficult to organize and maintain especially if a huge volume of data is involved. A data warehouse is helpful in this regard as it can assists in business decision-making. Data collection and loading it into a data warehouse is difficult job because data sources are not in consistent form. This job usually consists of three main processes that involve extraction, transformation and loading. To extract the data from different sources, then transform it into a unified format and consequently load it into the warehouse, ETL (Extract, transform and load) tools are required. Nowadays, the majority of ETL tools organize workflow. An ETL workflow can be considered as a group of ETL jobs with dependencies between them. In this research paper a revised ETL workflow management framework which is based upon different considerations is proposed. These considerations along with the addition of the components in the workflow scheduling layer would help in making more effective and quality business decisions.

Scheduling of Extract, Transform, and Load (ETL) Procedures with Genetic Algorithm

“…Load Plans in Oracle Data Integrator platform). Related articles approach the ETL scheduling optimisation problem by minimizing the total execution time (Shan & Yue-Ping, 2013) and allocated memory (Karagiannis, Vassiliadis, & Simitsis, 2013) or by straightforward rearranging of the procedures by their priorities (Malik, Shamim, Bibi, Khan, & Gorsi, 2013).…”