A Multi-Agent Framework for Data Extraction,Transformation and Loading in Data Warehouse

Abstract

Abstract-The rapid growth in size of data sets poses challenge to extract and analyze information in timely manner for better prediction and decision making. Data warehouse is the solution for strategic decision making. Data warehouse serves as a repository to store historical and current data. Extraction, Transformation and Loading (ETL) process gather data from different sources and integrate it into data warehouse. This paper proposes a multi-agent framework that enhance the efficiency of ETL process. Agents perform specific task assigned to them. The identification of errors at different stages of ETL process become easy. This was difficult and time consuming in traditional ETL process. Multi-agent framework identify data sources, extract, integrate, transform, and load data into data warehouse. A monitoring agent remains active during this process and generate alerts if there is issue at any stage.

Performance Enhancement of Predictive Analytics for Health Informatics Using Dimensionality Reduction Techniques and Fusion Frameworks

“…Feature selection creates a subset of features using correlation analysis or weighting methods [14]. Feature extraction transforms the original HDD into a low-dimensional representation by eliminating the redundant features [15]. Feature extraction can be performed using linear approaches [16], such as Principal Component Analysis (PCA), Independent Compo-nent Analysis (ICA).…”

Employing Machine Learning‐Based Predictive Analytical Approaches to Classify Autism Spectrum Disorder Types

“…Machine learning is a growing field, which uses mathematical learning, statistical estimation, and information theories to find useful patterns in the large amount of data [13][14][15][16]. Recently, deep learning is the most trending area of research, which is the subset of machine learning and uses the neural network architectures to model the high-level abstraction in data [17].…”