Data Mapper: An Operator for Expressing One-to-Many Data Transformations

Abstract

Abstract. Transforming data is a fundamental operation in application scenarios involving data integration, legacy data migration, data cleaning, and extract-transform-load processes. Data transformations are often implemented as relational queries that aim at leveraging the optimization capabilities of most RDBMSs. However, relational query languages like SQL are not expressive enough to specify an important class of data transformations that produce several output tuples for a single input tuple. This class of data transformations is required for solving the data heterogeneities that occur when source data represents an aggregation of target data. In this paper, we propose and formally define the data mapper operator as an extension of the relational algebra to address one-to-many data transformations. We supply an algebraic rewriting technique that enables the optimization of data transformation expressions that combine filters expressed as standard relational operators with mappers. Furthermore, we identify the two main factors that influence the expected optimization gains.

Data Load Distribution by Semi Real Time Data Warehouse

“…The next step is to transform extracted data by applying different transformation techniques. Data transformation is a basic process in application scenarios concerning integration of data, migration of legacy data, data cleansing or data scrubbing, and extractiontransformation-loading procedures [17]. Transformation and loading are done with the help of oracle warehouse builder.…”

“…A little research efforts are also carried out for the ETL processes optimization [16,8], together with works associated with distinct operators like the DataMapper [17]. Also research work in the field of data fusion [9,13], schema mapping and data cleaning [20,23] that is linked with ETL processes.…”

ONE-TO-MANY DATA TRANSFORMATION OPERATIONS - Optimization and Execution on an RDBMS

“…A key insight is that, when applied to a tuple, a mapper function can produce a set of values in the domain of its target attributes Dom(A i ), rather than a single value. Further details can be found in (Carreira et al, 2005b).…”

“…Herein, due to space limitations, we briefly sketch the main ideas. Please refer to (Carreira et al, 2005b;Carreira et al, 2006) for details about rewriting rules and their corresponding proofs of correctness.…”

Study and application of a multimedia content transformation method based on model-view-controller 2x pattern

“…Data transformation is a fundamental operation in application scenarios involving data integration, legacy data migration, data cleaning, and extract-transform-load processes [1]. Among the mass data stored in current corporations, the traditional structural data that is operated by RDBMS only occupies fifteen percent.…”