Dimensions of AutomatedETL Management: A Contemporary Literature Review

Abstract

Emerging dynamics of data systems and reliance on quality data sources, data processing for informed and strategic decision-making enhance the scope of using the ETL solutions. In the current scenario, one of the critical aspects focused on software engineering is about focusing on using the data management tools that can help gain insights for functional and operational aspects. While many academic and industrial research studies have focused on data management dynamics and the application of ETL tools as a profound solution, there is an imperative need to upscaling the ETL efficiency over real-time applications. In this literature review, the scope of the current ETL frameworks, limitations, and scope are discussed. Categorically, the objective is to explore if the machine learning models are adapted in the ETL systems. However, from the literature review, it is evident that many academic studies have advocated using machine learning models to improve and optimize the use of ETL solutions. But very few tools in the market are using the comprehensive range of machine learning models in ETL processing. Focusing on the current constraints and the scope for improvement, this study advocates the need for designing and developing machine learning-based models for ETL-based data management optimization. If such processes could be developed, it can help the organizations have potential systems in place for decision-making.