A Semantic-Based Approach to Attain Reproducibility of Computational Environments in Scientific Workflows: A Case Study

Santana-Perez, Idafen; Silva, Rafael Ferreira da; Rynge, Mats; Deelman, Ewa; Pérez-Hernández, María S.; Corcho, Óscar

doi:10.1007/978-3-319-14325-5_39

Cited by 11 publications

(14 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Tools and ideas were presented, such as Pegasus [11] and Kameleon [5]: these tools target reproducible experiments, mainly addressing the problem exploiting virtual machines to reproduce the environment where an experiment is held. The main focus of these research groups keeps being on the virtual machine approach combined with cloud computing, as can be seen in [1] and [10]. More recent tools approach the reproducibility issue by building a workflow using web services or creating portable experimental software packages, like Taverna, VisTrails, Knime, Kepler and CDE.…”

Section: Infrastructure For Reproducible and Trusted Experimentsmentioning

confidence: 98%

Trusted High-Performance Computing in the Classroom

Burkhart

Guerrera

Maffia

2014

2014 Workshop on Education for High Performance Computing

View full text Add to dashboard Cite

A well-designed high-performance computing (HPC) course not only presents theoretical parallelism concepts but also includes practical work on parallel systems. Today's machine models are diverse and as a consequence multiple programming models exist. The challenge for HPC course lecturers is to decide what to include and what to exclude, respectively. We have experience in teaching HPC in a multi-paradigm style. The practical course parts include message-passing programming using MPI, directive-based shared memory programming using OpenMP, partitioned global address space based programming using Chapel, and domain-specific programming using a high-level framework. If these models are taught in an isolated mode, students would have problems in assessing the strengths and weaknesses of the approaches presented. We propose a projectbased approach which introduces a specific problem to be solved (in our case a stencil computation) and asks for solutions using the programming approaches introduced. Our course has been successfully taught several times but a major problem has always been checking the individual student solutions, especially to decide which performance results reported one can trust. In order to overcome these deficiencies, we have built a pedagogical tool which enhances the trust in students' work. In the paper we present the infrastructure and tools that make student experiments easily reproducible by lecturers. We introduce a taxonomy for general benchmark experiments, describe the distributed architecture of our development and analysis environment, and, as a case study, discuss performance experiments when solving a stencil problem in multiple programming models.

show abstract

Section: Infrastructure For Reproducible and Trusted Experimentsmentioning

confidence: 98%

Trusted High-Performance Computing in the Classroom

Burkhart

Guerrera

Maffia

2014

2014 Workshop on Education for High Performance Computing

View full text Add to dashboard Cite

show abstract

“…A research study (Belhajjame et al, 2012) conducted to evaluate the reproducibility of scientific workflows has shown that around 80% of the workflows cannot be reproduced, and 12% of them are due to the lack of information about the execution environment (Santana-P´erez and P´erez-Hern´andez, 2014). This information affects a workflow execution in multiple ways.…”

Section: Motivationmentioning

confidence: 99%

Cloud infrastructure provenance collection and management to reproduce scientific workflows execution

Hasham

Munir

McClatchey

2018

Future Generation Computer Systems

View full text Add to dashboard Cite

The emergence of Cloud computing provides a new computing paradigm for scientific workflow execution. It provides dynamic, on-demand and scal-able resources that enable the processing of complex workflow-based experiments. With the ever growing size of the experimental data and increasingly complex processing workflows, the need for reproducibility has also become essential. Provenance has been thought of a mechanism to verify a workflow and to provide workflow reproducibility. One of the obstacles in reproduc-ing an experiment execution is the lack of information about the execution infrastructure in the collected provenance. This information becomes crit-ical in the context of Cloud in which resources are provisioned on-demand and by specifying resource configurations. Therefore, a mechanism is re-quired that enables capturing of infrastructure information along with the provenance of workflows executing on the Cloud to facilitate the re-creation of execution environment on the Cloud. This paper presents a framework, ReCAP, along with the proposed mapping approaches that aid in captur-ing the Cloudaware provenance information and help in re-provisioning the execution resource on the Cloud with similar configurations. Experimental evaluation has shown the impact of different resource configurations on the workflow execution performance, therefore justifies the need for collecting such provenance information in the context of Cloud. The evaluation has also demonstrated that the proposed mapping approaches can capture Cloud information in various Cloud usage scenarios without causing performance overhead and can also enable the re-provisioning of resources on Cloud. Experiments were conducted using workflows from different scientific domains such as astronomy and neuroscience to demonstrate the applicability of this research for different workflows.

show abstract

“…In order to evaluate this aspect of workflow reproducibility, an algorithm has been proposed that compares the outputs produced by two given workflows. It uses the MD5 hashing algorithm [30] on the outputs and compares the hash value to verify the produced outputs. The MD5 algorithm outputs a string that is often used in many systems as the single data identifier, due to a small collision probability [31].…”

Section: B Workflow Output Comparisonmentioning

confidence: 99%

Reproducibility of scientific workflows execution using cloud-aware provenance (ReCAP)

Hasham

Munir

2018

Computing

View full text Add to dashboard Cite

Provenance of scientific workflows has been considered a mean to provide workflow reproducibility. However, the provenance approaches adopted so far are not applicable in the context of Cloud because the provenance trace lacks the Cloud information. This paper presents a novel approach that collects the Cloud-aware provenance and represents it as a graph. The workflow execution reproducibility on the Cloud is determined by comparing the workflow provenance at three levels i.e., workflow structure, execution infrastructure and workflow outputs. The experimental evaluation shows that the implemented approach can detect changes in the provenance traces and the outputs produced by the workflow.

show abstract

A Semantic-Based Approach to Attain Reproducibility of Computational Environments in Scientific Workflows: A Case Study

Cited by 11 publications

References 16 publications

Trusted High-Performance Computing in the Classroom

Trusted High-Performance Computing in the Classroom

Cloud infrastructure provenance collection and management to reproduce scientific workflows execution

Reproducibility of scientific workflows execution using cloud-aware provenance (ReCAP)

Contact Info

Product

Resources

About