A new optimization phase for scientific workflow management systems

Holl, Sonja; Zimmermann, Olav; Palmblad, Magnus; Mohammed, Yassene; Hofmann‐Apitius, Martin

doi:10.1016/j.future.2013.09.005

Cited by 18 publications

(10 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Especially we consider the performance of different resources when we calculate the percentages in Eq. (4).Through this way we can obtain both short overall completion time and excellent load balance level in most workflow situations.…”

Section: Algorithm Analysismentioning

confidence: 94%

“…Scientific workflow is very suitable and convenient to model such process and express the entire data processing steps and dependencies for the users [3]. A scientific workflow is similar to the general workflow, which organizes a set of tasks in an order according to their dependent relations, but it is data-oriented not control-oriented which means it pays more attention to the intensive operations on data sets, and the dependencies emphatically describe the flow of data streams [4]. The technology of scientific workflow has been successfully introduced in the scientific fields such as bioinformatics, genetics, astrophysics, and geophysics which provide an environment to aid the scientific discovery process through the combination of available tools for scientific data management, analysis, simulation, and visualization [5].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A scientific workflow management system architecture and its scheduling based on cloud service platform for manufacturing big data analytics

Song

Huang

2015

Int J Adv Manuf Technol

View full text Add to dashboard Cite

To improve the efficiency and productivity of modern manufacturing, the requirements for enterprises are discussed. The new emerged technologies such as cloud computing and internet of things are analyzed and the bottlenecks faced by enterprises in manufacturing big data analytics are investigated. Scientific workflow technology as a method to solve the problems is introduced and an architecture of scientific workflow management system based on cloud manufacturing service platform is proposed. The functions of each layer in the architecture are described in detail and implemented with an existing workflow system as a case study. The workflow scheduling algorithm is the key issue of management system, and the related work is reviewed. This paper takes the general problems of existing algorithms as the motivation to propose a novel scheduling algorithm called MP (max percentages) algorithm. The simulation results indicate that the proposed algorithm has performed better than the other five classic algorithms with respect to both the total completion time and load balancing level.

show abstract

Section: Algorithm Analysismentioning

confidence: 94%

Section: Introductionmentioning

confidence: 99%

A scientific workflow management system architecture and its scheduling based on cloud service platform for manufacturing big data analytics

Song

Huang

2015

Int J Adv Manuf Technol

View full text Add to dashboard Cite

show abstract

“…In general, workflow lifecycle is composed of the following phases (Holl et al, 2013): (1) Design and refinement: involves the design of a new workflow or refinement of an existing workflow, through selection and composition of components, and definition of data and component dependencies. (2) Planning and sharing: involves the creation of an executable workflow from the abstract design, through mapping the abstract elements into concrete applications, defining data sources, and sharing the designed workflow with each individual agent.…”

Section: Workflow Optimizationmentioning

confidence: 99%

Optimization and Control

Nof

Ceroni

Jeong

et al. 2015

Revolutionizing Collaboration Through E-Work, E-Business, and E-Service

View full text Add to dashboard Cite

Abstract. Considering the foundations, tools, and emerging discoveries of collaborative e-Work, as discussed in Chapters 1 and 2, it is realized that optimization and control are focused primarily on the core elements of e-Systems; agents, protocols, and workflows. In this chapter, we will show that these elements compose a solid framework for optimization and control of collaboration in emerging distributed e-Work systems. In order to be able to efficiently pass the benefits on to more complex constructs such as autonomous agents systems, production units configuration, highly reactive control protocols, and so on, these elements must be optimized as well. In order to show the evidence of the latest developments in optimization and control involving agent, protocol, and workflow theories, this chapter reviews the state-of-the-art techniques for achieving optimal design and operational control, and collaboration engineering. This chapter covers the incentives to construct autonomous agent-based systems, the key e-Criteria emerging from the transformation from traditional centralized work systems to decentralized e-Work systems, and several real-life applications of agent-based systems. Basic agent-based optimization and control architectures are reviewed along with pioneering bioinspired mechanisms based on swarm intelligence and natural evolution, and their impact on the intelligence and autonomy of agents. Several techniques for protocol and workflow optimization are also discussed. Emerging Impacts of AgentsWith the transformation from traditional monolithic systems to decentralized e-Work networks, optimization and control of e-Systems are facing new challenges. The emerging requirements for timely response to the growing complexity and dynamicity of e-Systems call for higher intelligence and autonomy of agents for smarter interaction and collaboration (Rosenschein and Ephrati, 1993). Classic monolithic and hierarchical systems are no longer sufficient for addressing such requirements, and often lead to situations where the entire system collapses by a single error or conflict at one point of the system hierarchy 116 4 Optimization and Control (Colombo et al., 2006). Various e-Criteria have been defined to formalize the emerging needs of e-Systems at different levels, and with different contextual relationships (Figure 4.1): adaptability, flexibility, reconfigurability, changeability, robustness, and scalability (Putnik et al., 2013). The prerequisites for addressing these requirements, however, mean to enable and delegate intelligence and autonomy to the lowest-level individual units collaborating within the e-System, namely the agents. Fig. 4.1 (a) Flexibility is interpreted as the ability of a system to change its behavior without changing its configuration, and reconfigurability is interpreted as the ability to change the behavior of a system by changing its configuration (both implying a changeable system). Scalability implies changing configuration by adding/removing the configuration elements, which is...

show abstract

“…Currently, data flows are typically designed manually, although commercial tools may provide some simple, static, cost-oblivious rule-based optimizations [6,7]. Interestingly, there is an increasingly large portion of flow designers that are not IT experts [1], which raises doubts about the optimality of such manual designs.…”

Section: Introductionmentioning

confidence: 99%

Optimization of Data-intensive Flows

Gounaris

2014

Proceedings of the 17th International Workshop on Data Warehousing and OLAP

View full text Add to dashboard Cite

Modern data analysis is increasingly employing data-intensive flows for processing very large volumes of data. As the data flows become more and more complex and operate in a highly dynamic environment, we argue that we need to resort to automated cost-based optimization solutions rather than relying on efficient designs by human experts. We further demonstrate that the current state-of-the-art in flow optimizations needs to be extended and we propose a promising direction for optimizing flows at the logical level, and more specifically, for deciding the sequence of flow tasks.

show abstract

A new optimization phase for scientific workflow management systems

Cited by 18 publications

References 39 publications

A scientific workflow management system architecture and its scheduling based on cloud service platform for manufacturing big data analytics

A scientific workflow management system architecture and its scheduling based on cloud service platform for manufacturing big data analytics

Optimization and Control

Optimization of Data-intensive Flows

Contact Info

Product

Resources

About