Capturing Interactive Data Transformation Operations Using Provenance Workflows

Omitola, Tope; Freitas, André; Curry, Edward; Ó’Riain, Seán; Gibbins, Nicholas; Shadbolt, Nigel

doi:10.1007/978-3-662-46641-4_3

Cited by 7 publications

(8 citation statements)

References 7 publications

(1 reference statement)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Future work include the refinement of the vocabulary based on feedback from users. The provenance model proposed in this paper was already implemented to describe interactive data transformations from the Google Refine platform [13]. The verification of the interoperability between Google Refine and an open source ETL platform is planned.…”

Section: Discussionmentioning

confidence: 99%

Representing Interoperable Provenance Descriptions for ETL Workflows

Freitas

Kämpgen

Oliveira

et al. 2015

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. The increasing availability of data on the Web provided by the emergence of Web 2.0 applications and, more recently by Linked Data, brought additional complexity to data management tasks, where the number of available data sources and their associated heterogeneity drastically increases. In this scenario, where data is reused and repurposed on a new scale, the pattern expressed as Extract-Transform-Load (ETL) emerges as a fundamental and recurrent process for both producers and consumers of data on the Web. In addition to ETL, provenance, the representation of source artifacts, processes and agents behind data, becomes another cornerstone element for Web data management, playing a fundamental role in data quality assessment, data semantics and facilitating the reproducibility of data transformation processes. This paper proposes the convergence of these two Web data management concerns, introducing a principled provenance model for ETL processes in the form of a vocabulary based on the Open Provenance Model (OPM) standard and focusing on the provision of an interoperable provenance model for ETL environments. The proposed ETL provenance model is instantiated in a real-world sustainability reporting scenario.

show abstract

Section: Discussionmentioning

confidence: 99%

Representing Interoperable Provenance Descriptions for ETL Workflows

Freitas

Kämpgen

Oliveira

et al. 2015

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…The process of data transformation in all aspects of keeping lossless to less is hard but not impossible to achieve. Must needed requirement is to be carefully checking the availability of mapping mechanism to become able in constructing a new one in case of noncompliance with transformation process to complete . To achieve similar capabilities, the proposed model being introduced is using XML as an intermediate level of data transformation process.…”

Section: Spatial Data Transformation Methodologymentioning

confidence: 99%

A generic methodology for geo‐related data semantic annotation

Malik

Habib

Khalid

et al. 2018

Concurrency and Computation

View full text Add to dashboard Cite

Summary Geo‐related data, also known as spatial data, is represented using a vector used for representing longitude, elevation, and latitude. Specially built systems, well known as Geographical Information Systems (GIS), made use of such data for querying, manipulating, navigation, and analyzing. In the current era of data, science needs to involve smart interactive investigation involving Internet of Data (IoD) on predicting upcoming changes and spatial updates on the map is growing rapidly. To resolve issues concerning real‐time spatial data, transformation using semantic annotation can provide a better way to translate spatial relationships. These spatial relationships will support spatial analysis by linking different cause and effect with the help of reasoning mechanism. This research's major focus is on a data transformation methodology for geo‐related semantic annotation. Spatial dataset gets stored in a database and then transformed into Extensible Markup Language (XML) and Resource Description Framework (RDF). Even for bi‐directional transformation to work properly, we need to map different schema level transformations. A deep research is conducted to consider available mappings, implementations, and updates to further improving data fusion for having better compatibility. Then, transformed data as results get analyzed and discussed based on the data mapping rules formulated. It is aimed to show the importance of reducing the response time of investigation and offer compatibility between the web and semantically enriched spatial data.

show abstract

“…Besides, the data transformation effort is usually positioned as one element in a workflow which needs to be integrated with different processes to generate LOD. Last but not least, to several authors the data transformation effort can be tracked to provide provenance data in order to ensure the quality of LOD [6][13] [23]. Cordeiro and others [6] proposed an innovative approach to support the exposure, sharing and association of government data resources in the form of LOD.…”

Section: Provenance Initiativesmentioning

confidence: 99%

“…The proposed model was built upon the structure of OPM, being designed to extend the basic semantics and structure of OPMbased provenance workflows. The second author formalized and emphasized the potential of interactive data transformation (IDT) tools to support data transformation efforts in LOD, [23] also propose an OPM ontology-based provenance model for capturing, representing and storing different kinds of provenance of a given process managed by an IDT platform. The proposed approach organizes the provenance management in a three-layer approach that handles the provenance data as RDF triples.…”

Section: Provenance Initiativesmentioning

confidence: 99%

“…This approach was used in the provenance management architecture presented by [23] as described at the end of section 3. The adopted approach represents provenance data in three layers: the bottom layer uses the Open Provenance Model Vocabulary (OPMV) to describe basic provenance related to the workflow, the second layer uses Cogs Vocabulary to describe provenance related to the ETL operations and the third layer uses a specific vocabulary to describe provenance related to domain specific objects of the workflow.…”

Section: Data Preparation and Transformation Process Provenancementioning

confidence: 99%

See 1 more Smart Citation

Lop

Mendonça

Cruz

Cerda

et al. 2013

Proceedings of the Fifth Workshop on Semantic Web Information Management

View full text Add to dashboard Cite

The Web of Data has emerged as a means to expose, share, reuse, and connect information on the Web identified by URIs using RDF as a data model, following Linked Data Principles. However, the reuse of third party data can be compromised without proper data quality assessments. In this context, important questions emerge: how can one trust on published data and links? Which manipulation, modification and integration operations have been applied to the data before its publication? What is the nature of comparisons or transformations applied to data during the interlinking process? In this scenario, provenance becomes a fundamental element. In this paper, we describe an approach for generating and capturing Linked Open Provenance (LOP) to support data quality and trustworthiness assessments, which covers preparation and format transformation of traditional data sources, up to dataset publication and interlinking. The proposed architecture takes advantage of provenance agents, orchestrated by an ETL workflow approach, to collect provenance at any specified level and also link it with its corresponding data. We also describe a real use case scenario where the architecture was implemented to evaluate the proposal.

show abstract

Capturing Interactive Data Transformation Operations Using Provenance Workflows

Cited by 7 publications

References 7 publications

Representing Interoperable Provenance Descriptions for ETL Workflows

Representing Interoperable Provenance Descriptions for ETL Workflows

A generic methodology for geo‐related data semantic annotation

Lop

Contact Info

Product

Resources

About