Designing for Recommending Intermediate States in A Scientific Workflow Management System

Chakroborti, Debasish; Roy, Banani; Nath, Sristy Sumana

doi:10.1145/3457145

Cited by 3 publications

(4 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When utilizing relation similarity to progressively adjust the matching degree with uncertainty of module pairs under pairwise workflows, the efficacy of MANSOR is influenced by the number of modules present in a given workflow (termed as "workflow scale"), represented by num. To assess the impact of workflow scale on MANSOR's performance, we partition the user rating dataset into subsets based on the number of modules (≤ 5, [6, 10], [11,15], [16,20], ≥ 21) and evaluate MANSOR's effectiveness under varying workflow scales. The outcomes of these experiments are depicted in Figure 6.…”

Section: Performance Analyses Wrt the Number Of Modules In One Workflowmentioning

confidence: 99%

“…Since a fresh scientific experiment may be relevant to numerous experiments conducted by others in the majority of cases, the scientific workflows of these experiments will not differ significantly 14 . In particular, the researcher can substitute important modules of original workflows with functionally similar modules that satisfy his/her needs to create a personalized workflow, which is cost‐effective and error‐avoiding compared to designing from scratch 15 . Thus, discovering similar and appropriate modules from various scientific workflows is a crucial and significant research topic.…”

Section: Introductionmentioning

confidence: 99%

“…14 In particular, the researcher can substitute important modules of original workflows with functionally similar modules that satisfy his/her needs to create a personalized workflow, which is cost-effective and error-avoiding compared to designing from scratch. 15 Thus, discovering similar and appropriate modules from various scientific workflows is a crucial and significant research topic.…”

mentioning

confidence: 99%

See 2 more Smart Citations

MANSOR: A module alignment method based on neighbor information for scientific workflow

Cao

Qian

et al. 2023

Concurrency and Computation

View full text Add to dashboard Cite

SummaryFinding similar scientific workflow modules that can substitute essential components of the privacy workflows from public repositories to create a personalized workflow is growing in popularity. This is a cost‐effective and error‐free strategy for the scientific community. Currently, module alignment approaches heavily depend on syntactic information such as modules' names and types. However, the contextual semantic information of modules, which encompasses inputs, outputs, and datalinks connecting modules, can also convey their functions. Unfortunately, this information is scarcely utilized during the module alignment process. In this work, we propose a module alignment method based on neighbor information for scientific workflow (MANSOR). Specifically, we present a rule‐based attribute similarity computation approach for calculating initial module‐pair similarity in two workflows. The relation similarity is then employed to iteratively fine‐tune the matching degree with uncertainty of module pairs by considering the contextual semantics until reaching a steady state. After pairwise comparison of workflows, the module alignment results are determined based on their module‐pair similarity. Experimental results on the human‐curated corpus of ratings indicate that MANSOR outperforms the existing state‐of‐the‐art approaches with statistical significance.

show abstract

Section: Performance Analyses Wrt the Number Of Modules In One Workflowmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

MANSOR: A module alignment method based on neighbor information for scientific workflow

Cao

Qian

et al. 2023

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…Data management has recently been considered in workflow management systems (WfMS) to ensure efficient data storage and workflow execution. 30,31 Chakroborti et al's 31 aim was to reinforce the process of workflow construction with mechanisms for the recommendation of intermediate data and the handling of intermediate states in the built workflow w.r.t. the user requirements.…”

Section: Background On Service Managementmentioning

confidence: 99%

On the use of big data frameworks in big service management

Ghedass,

Ben Charrada

2023

J Software Evolu Process

View full text Add to dashboard Cite

Over the last few years, big data have emerged as a paradigm for processing and analyzing a large volume of data. Coupled with other paradigms, such as cloud computing, service computing, and Internet of Things, big data processing takes advantage of the underlying cloud infrastructure, which allows hosting and managing massive amounts of data, while service computing allows to process and deliver various data sources as on‐demand services. This synergy between multiple paradigms has led to the emergence of big services, as a cross‐domain, large‐scale, and big data‐centric service model. Apart from the adaptation issues (e.g., need of high reaction to changes) inherited from other service models, the massiveness and heterogeneity of big services add a new factor of complexity to the way such a large‐scale service ecosystem is managed in case of execution deviations. Indeed, big services are often subject to frequent deviations at both the functional (e.g., service failure, QoS degradation, and IoT resource unavailability) and data (e.g., data source unavailability or access restrictions) levels. Handling these execution problems is beyond the capacity of traditional web/cloud service management tools, and the majority of big service approaches have targeted specific management operations, such as selection and composition. To maintain a moderate state and high quality of their cross‐domain execution, big services should be continuously monitored and managed in a scalable and autonomous way. To cope with the absence of self‐management frameworks for large‐scale services, the goal of this work is to design an autonomic management solution that takes the whole control of big services in an autonomous and distributed lifecycle process. We combine autonomic computing and big data processing paradigms to endow big services with self‐* and parallel processing capabilities. The proposed management framework takes advantage of the well‐known MapReduce programming model and Apache Spark and manages big service's related data using knowledge graph technology. We also define a scalable embedding model that allows processing and learning latent big service knowledge in a distributed manner. Finally, a cooperative decision mechanism is defined to trigger non‐conflicting management policies in response to the captured deviations of the running big service. Big services' management tasks (monitoring, embedding, and decision), as well as the core modules (autonomic managers' controller, embedding module, and coordinator), are implemented on top of Apache Spark as MapReduce jobs, while the processed data are represented as resilient distributed dataset (RDD) structures. To exploit the shared information exchanged between the workers and the master node (coordinator), and for further resolution of conflicts between management policies, we endowed the proposed framework with a lightweight communication mechanism that allows transferring useful knowledge between the running map‐reduce tasks and filtering inappropriate intermediate data (e.g., conflicting actions). The experimental results proved the increased quality of embeddings and the high performance of autonomic managers in a parallel and cooperative setting, thanks to the shared knowledge.

show abstract

Challenges of Provenance in Scientific Workflow Management Systems

Alam

Roy

2022

2022 IEEE/ACM Workshop on Workflows in Support of Large-Scale Science (WORKS)

View full text Add to dashboard Cite

Designing for Recommending Intermediate States in A Scientific Workflow Management System

Cited by 3 publications

References 30 publications

MANSOR: A module alignment method based on neighbor information for scientific workflow

MANSOR: A module alignment method based on neighbor information for scientific workflow

On the use of big data frameworks in big service management

Challenges of Provenance in Scientific Workflow Management Systems

Contact Info

Product

Resources

About