A New Approach for Conceptual Extraction-Transformation-Loading Process Modeling

Biswas, Neepa; Chattapadhyay, Samiran; Mahapatra, Gautam; Chatterjee, Santanu; Mondal, Kartick Chandra

doi:10.4018/ijaci.2019010102

Cited by 19 publications

(5 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… Biswas et al (2019) proposed another modeling approach for conceptualizing the ETL process using a standard systems modeling language (SysML), which is a new modeling language standardized by the Object Management Group (OMG). Their article additionally focused on creating an automated executable SysML model based on an activity diagram.…”

Section: Related Workmentioning

confidence: 99%

A domain-specific language for managing ETL processes

Popović,

Ivković,

Trajković

et al. 2024

PeerJ Computer Science

View full text Add to dashboard Cite

Maintenance of Data Warehouse (DW) systems is a critical task because any downtime or data loss can have significant consequences on business applications. Existing DW maintenance solutions mostly rely on concrete technologies and tools that are dependent on: the platform on which the DW system was created; the specific data extraction, transformation, and loading (ETL) tool; and the database language the DW uses. Different languages for different versions of DW systems make organizing DW processes difficult, as minimal changes in the structure require major changes in the application code for managing ETL processes. This article proposes a domain-specific language (DSL) for ETL process management that mitigates these problems by centralizing all program logic, making it independent from a particular platform. This approach would simplify DW system maintenance. The platform-independent language proposed in this article also provides an easier way to create a unified environment to control DW processes, regardless of the language, environment, or ETL tool the DW uses.

show abstract

Section: Related Workmentioning

confidence: 99%

A domain-specific language for managing ETL processes

Popović,

Ivković,

Trajković

et al. 2024

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…A new approach to the design of ETL processes using models and meta-models is described. Furthermore, a new approach to modeling the ETL process using the modeling language (SysML) is proposed by authors in [22]. A simulation of an expected case study of e-commerce is presented.…”

Section: Related Workmentioning

confidence: 99%

Towards a Scalable and Efficient ETL

Gueddoudj,

Chikh

2023

IJCDS

View full text Add to dashboard Cite

Extract, transform, and load (ETL) processes are crucial for building repositories of data from a variety of self-contained sources. Despite their complexity and cost, ETL processes have demonstrated some maturity for traditional, XML, and graph data sources. However the main challenge for ETL processes is double: (1) they do not scale when brought down to managing large and highly varied data sources, involving web-data. (2) the deployment of the target data warehouse in a polystore. The paper reviews various research efforts along this line of research. The paper then proposes a conceptual modeling of these processes using BPMN (Business Process Modeling Notation). These processes are automatically converted to scripts to be implemented within Spark framework. The solution is packaged according a new distributed architecture (Open ETL) that supports both batch and stream processing. To make our new approach more concrete and evaluable, a real case study using the LUBM benchmark, which involves heterogeneous data sources is considered.

show abstract

“…The XMI (XML Metadata Interchange) executable code corresponding to the ETL model is then generated. In a follow-up work, based on the model-based system engineering (MBSE)-oriented approach, [40] tried to justify the system validation by applying a simulation process. For this purpose, the Cameo Simulation Toolkit (CST) was used to simulate the proposed conceptual model through an activity simulation engine, which allows for execution of the generated activity diagrams.…”

Section: Summary Of Etl Process Modeling Approaches Based On Umlmentioning

confidence: 99%

“…Indeed, the works proposed by [6,22] were based on the class diagram to conceptually model the proposed ETL process. The works of [33,34,39] were based on the activity diagram, while [35,40] used both activity and requirement diagrams to express the requirements, based on the text and their relationship provided by the SysML. In addition, the only contribution that used the object diagram was that of [39].…”

Section: Comparison Of Uml-based Approachesmentioning

confidence: 99%

Data Warehousing Process Modeling from Classical Approaches to New Trends: Main Features and Comparisons

Dhaouadi

Bousselmi

Gammoudi

et al. 2022

Data

View full text Add to dashboard Cite

The extract, transform, and load (ETL) process is at the core of data warehousing architectures. As such, the success of data warehouse (DW) projects is essentially based on the proper modeling of the ETL process. As there is no standard model for the representation and design of this process, several researchers have made efforts to propose modeling methods based on different formalisms, such as unified modeling language (UML), ontology, model-driven architecture (MDA), model-driven development (MDD), and graphical flow, which includes business process model notation (BPMN), colored Petri nets (CPN), Yet Another Workflow Language (YAWL), CommonCube, entity modeling diagram (EMD), and so on. With the emergence of Big Data, despite the multitude of relevant approaches proposed for modeling the ETL process in classical environments, part of the community has been motivated to provide new data warehousing methods that support Big Data specifications. In this paper, we present a summary of relevant works related to the modeling of data warehousing approaches, from classical ETL processes to ELT design approaches. A systematic literature review is conducted and a detailed set of comparison criteria are defined in order to allow the reader to better understand the evolution of these processes. Our study paints a complete picture of ETL modeling approaches, from their advent to the era of Big Data, while comparing their main characteristics. This study allows for the identification of the main challenges and issues related to the design of Big Data warehousing systems, mainly involving the lack of a generic design model for data collection, storage, processing, querying, and analysis.

show abstract

A New Approach for Conceptual Extraction-Transformation-Loading Process Modeling

Cited by 19 publications

References 27 publications

A domain-specific language for managing ETL processes

A domain-specific language for managing ETL processes

Towards a Scalable and Efficient ETL

Data Warehousing Process Modeling from Classical Approaches to New Trends: Main Features and Comparisons

Contact Info

Product

Resources

About