Collaborative Scientific Workflows

Lu, Shiyong; Zhang, Jia

doi:10.1109/icws.2009.150

Cited by 24 publications

(16 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…c) It could also enable a new generation of scientific workflows -collaborative scientific workflows [18], in which user interaction and collaboration patterns are firstclass entities for scientific workflow management. User interaction and collaboration intensive scientific workflows have been difficult to implement in a Grid environment as it is more suitable for batch-based scientific workflows.…”

Section: Opportunitiesmentioning

confidence: 99%

Opportunities and Challenges in Running Scientific Workflows on the Cloud

Zhao

Fei

Raicu

et al. 2011

2011 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery

Self Cite

View full text Add to dashboard Cite

Abstract-Cloud computing is gaining tremendous momentum in both academia and industry. The application of Cloud computing, however, has mostly focused on Web applications and business applications; while the recognition of using Cloud computing to support large-scale workflows, especially dataintensive scientific workflows on the Cloud is still largely overlooked. We coin the term "Cloud Workflow", to refer to the specification, execution, provenance tracking of large-scale scientific workflows, as well as the management of data and computing resources to enable the execution of scientific workflows on the Cloud. In this paper, we analyze why there has been such a gap between the two technologies, and what it means to bring Cloud and workflow together; we then present the key challenges in running Cloud workflow, and discuss the research opportunities in realizing workflows on the Cloud.

show abstract

Section: Opportunitiesmentioning

confidence: 99%

Opportunities and Challenges in Running Scientific Workflows on the Cloud

Zhao

Fei

Raicu

et al. 2011

2011 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery

Self Cite

View full text Add to dashboard Cite

show abstract

“…Among other benefits, decomposing a complex application into a workflow simplifies the design effort, enables the reuse of computational modules, and allows their parallel and/or pipelined execution. The concept of workflow applications is used widely in scientific research in variety of domains such as bioinformatics, physics, astronomy, ecology and earthquake science (Lu and Zhang, 2009, Lim et al, 2010, Juve and Deelman, 2010. Given the computing, storage and networking technologies, coupled with the increased capacity to perform collaborative scientific research, there is an increased need to produce efficient scientific workflows.…”

Section: Introductionmentioning

confidence: 99%

Task And Data Allocation Strategies for Big Data Workflows

Ebrahimi¹,

Mohan²,

Kashlev³

et al. 2015

STBD

Self Cite

View full text Add to dashboard Cite

The makespan of a big data workflow is the time elapsed between the start of the first task and the completion of the last task in the workflow. This time includes the delivery of the final data product to the desired location within the network. Due to the large number of inputs and intermediate outputs of a big data workflow activity, the makespan of the workflow is significantly influenced by how its tasks and datasets are allocated in a distributed computing environment. Therefore, reducing makespans of big data workflows can be achieved by incorporating a data and task allocation strategy into an execution planning phase performed by a workflow management system. This creates a pressing need for an investigation of such strategies. To address this need, this paper provides a formal definition of the makespan minimization problem for big data workflows and proposes efficient workflow execution planning strategies. In particular, two algorithms, WEP-A and WEP-B, following different strategies are proposed. WEP-A follows a phased approach to the generation of an execution plan whereas WEP-B uses an evolutionary algorithm-based optimization strategy to find a valid plan with the shortest makespan. Both of these strategies are evaluated through extensive simulation experiments by varying workflow graphs and resources in the workflow environment. The results of the experiments demonstrate that WEP-B performs better than WEP-A on a set of benchmark examples. For more complex and large workflows, the improvements due to evolutionary optimization in WEP-B are likely to be even more pronounced.

show abstract

“…Modern data-intensive scientific research projects often require intensive collaboration among scientists with diverse expertise. Collaborative workflow composition is one important step in the lifecycle of collaborative scientific workflow management [2], [3].…”

Section: Introductionmentioning

confidence: 99%

A Granular Concurrency Control for Collaborative Scientific Workflow Composition

Fei

Zhang

2011

2011 IEEE International Conference on Services Computing

Self Cite

View full text Add to dashboard Cite

Abstract-Collaborative scientific workflow composition has recently been proposed to support collaborative scientific research projects, which require intensive collaboration among scientists with diverse expertise. A collaborative scientific workflow management system allows participating scientists to design and compose common scientific workflows concurrently. Concurrency control has become one of the key challenges in collaborative scientific workflow composition, which aims to facilitate collaboration while ensuring the correctness and consistency of the results generated from concurrent operations. We have previously proposed a locking scheme to support simple scientific workflow compositions, in which workflows are flat (not hierarchical). In this paper, we take a step forward to propose a new granular scientific workflow locking scheme, which captures dependency relationships between workflows, to support general hierarchical workflow compositions. We also conduct several experiments to evaluate the performance of the concurrency control.

show abstract

Collaborative Scientific Workflows

Abstract: In recent years, a number of scientific workflow management systems (SWFMSs)

Cited by 24 publications

References 33 publications

Opportunities and Challenges in Running Scientific Workflows on the Cloud

Opportunities and Challenges in Running Scientific Workflows on the Cloud

Task And Data Allocation Strategies for Big Data Workflows

A Granular Concurrency Control for Collaborative Scientific Workflow Composition

Contact Info

Product

Resources

About