Toward an End-to-End Framework for Modeling, Monitoring and Anomaly Detection for Scientific Workflows

Mandal, Anirban; Ruth, Paul; Baldin, Ilya; Król, Dariusz; Juve, Gideon; Mayani, Rajiv; Silva, Rafael Ferreira da; Deelman, Ewa; Meredith, Jeremy; Vetter, Jeffrey S.; Lynch, V. E.; Mayer, Ben; Wynne, James; Blanco, Mark P.; Carothers, C.D.; LaPre, Justin M.; Tierney, Brian

doi:10.1109/ipdpsw.2016.202

Cited by 14 publications

(6 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As to future work, it is now important to explore the implications of these conclusions to other kinds of scientific workflow analytics. Specifically, previous papers for anomalies detection for scientific workflow [23,22,24,25,26,25,31,30,29] that not based on TCP data transfers should be also reinvestigated as none have done tuning study to avoid falling in the same retracted category.…”

Section: Discussionmentioning

confidence: 99%

“…Several existing works [23,22,24,25,26] in end-to-end monitoring of workflow applications and systems are essential building blocks to detect such problems. However, several techniques for anomaly detection are often based on thresholds and simple statistics (e.g., moving averages) [27], which fail to understand longitudinal patterns, i.e., relationship between features.…”

Section: B Anomaly Detection In Scientific Workflowsmentioning

confidence: 99%

See 1 more Smart Citation

Mining Workflows for Anomalous Data Transfers

Papadimitriou

Kiran

et al. 2021

2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR)

Self Cite

View full text Add to dashboard Cite

Modern scientific workflows are data-driven and are often executed on distributed, heterogeneous, high-performance computing infrastructures. Anomalies and failures in the workflow execution cause loss of scientific productivity and inefficient use of the infrastructure. Hence, detecting, diagnosing, and mitigating these anomalies are immensely important for reliable and performant scientific workflows. Since these workflows rely heavily on high-performance network transfers that require strict QoS constraints, accurately detecting anomalous network performance is crucial to ensure reliable and efficient workflow execution. To address this challenge, we have developed X-FLASH, a network anomaly detection tool for faulty TCP workflow transfers. X-FLASH incorporates novel hyperparameter tuning and data mining approaches for improving the performance of the machine learning algorithms to accurately classify the anomalous TCP packets. X-FLASH leverages XGBoost as an ensemble model and couples XGBoost with a sequential optimizer, FLASH, borrowed from search-based Software Engineering to learn the optimal model parameters. X-FLASH found configurations that outperformed the existing approach up to 28%, 29%, and 40% relatively for F-measure, G-score, and recall in less than 30 evaluations. From (1) large improvement and (2) simple tuning, we recommend future research to have additional tuning study as a new standard, at least in the area of scientific workflow anomaly detection.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: B Anomaly Detection In Scientific Workflowsmentioning

confidence: 99%

Mining Workflows for Anomalous Data Transfers

Papadimitriou

Kiran

et al. 2021

2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Although human adaptation is a desired feature that remains an open problem in SWMSs, monitoring is widely supported in several existing SWMSs [4,35]. For example, Pegasus [36] provides a framework to monitor workflow executions and has rich capabilities for online performance monitoring, troubleshooting, and debugging.…”

Section: Related Workmentioning

confidence: 99%

Data reduction in scientific workflows using provenance monitoring and user steering

Souza

Silva

Coutinho

et al. 2020

Future Generation Computer Systems

View full text Add to dashboard Cite

Scientific workflows need to be iteratively, and often interactively, executed for large input datasets. Reducing data from input datasets is a powerful way to reduce overall execution time in such workflows. When this is accomplished online (i.e., without requiring the user to stop execution to reduce the data, and then resume), it can save much time. However, determining which subsets of the input data should be removed becomes a major problem. A related problem is to guarantee that the workflow system will maintain execution and data consistent with the reduction. Keeping track of how users interact with the workflow is essential for data provenance purposes. In this paper, we adopt the "human-in-the-loop" approach, which enables users to steer the running workflow and reduce subsets from datasets online. We propose an adaptive workflow monitoring approach that combines provenance data monitoring and computational steering to support users in analyzing the evolution of key parameters and determining the subset of data to remove. We extend a provenance data model to keep track of users' interactions when they reduce data at runtime. In our experimental validation, we develop a test case from the oil and gas domain, using a 936-cores cluster. The results on this test case show that the approach yields reductions of 32% of execution time and 14% of the data processed.

show abstract

“…Their solution is specific to one domain and yet to be further developed. The work of Reference present an online monitoring system for anomaly detection on workflows. There a handful of examples on the active monitoring of experiments, however the aspect of supervising the components in a Science Gateway is not well explored.…”

Section: Related Workmentioning

confidence: 99%

Optimizing computational resource management for the scientific gateways ecosystems based on the service‐oriented paradigm

et al. 2020

View full text Add to dashboard Cite

Science Gateways provide portals for experiments execution, regardless of the users' computational background. Nowadays its construction and performance need enhancement in terms of resource provision and task scheduling. We present the Modular Distributed Architecture to support the Protein Structure Prediction (MDAPSP), a Service-Oriented Architecture for management and construction of Science Gateways, with resource provisioning on a heterogeneous environment. The Decision Maker, central module of MDAPSP, defines the best computational environment according to experiment parameters. The proof of concept for MDAPSP is presented in WorkflowSim, with two novel schedulers. Our results demonstrate good Quality of Service (QoS), capable of correctly distributing the workload, fair response times, providing load balance, and overall system improvement. The study case relies on PSP algorithms and the Galaxy framework, with monitoring experiments to show the bottlenecks and critical aspects. K E Y W O R D Soptimization, protein structure prediction, science gateways, scientific workflow, service-oriented architecture, WorkflowSim 1 Softw: Pract Exper. 2020;50:899-924. wileyonlinelibrary.com/journal/spe © 2020 John Wiley & Sons, Ltd. 899 900 MARTINS DE OLIVEIRA et al.Another limitation of scientific gateways is the resource provision for its machines, that affect usability and performance. The problem with resource provisioning is a quest for the best configuration, requiring the smallest investment and minimal overheads. In general, inexperienced users cannot make good decisions on what machines to contract. Limited economic resources complicate the scenario.Cloud computing paradigm is among the principal resources available, as it has many advantages: scalability, pay-as-you-go features, on-demand provisioning, and management by third parties. Multiple providers infrastructure and decentralization of resources are expanding and are expected to create impact on diverse areas. 4 Resource providing and management are one of the challenging aspects of this expansion. 5 Despite its success, cloud is not the only option for provisioning. 6 Computer clusters are reliable infrastructures for conducting experiments, mainly High-Performance Computing (HPC) and concurrent algorithms. The advantages of cluster machines are local management, freedom of configuration, reliable storage, and fast network transfers. 3 For those reasons, we investigate the scientific gateway Galaxy, because it is an adaptable system, capable to operate over different infrastructures, including local workstations.Galaxy is a well-known framework for scientific experiments, with workflow management and reproducibility support. Its tools offer practical solutions and addition of new algorithms according to the evolution of the system. The focus is Biomedical science, protein prediction, gene sequencing, and other bioinformatics-related areas. Its advantages are simple interface, reliability, powerful tools, and an active community. 7 However, as any other...

show abstract

Toward an End-to-End Framework for Modeling, Monitoring and Anomaly Detection for Scientific Workflows

Cited by 14 publications

References 15 publications

Mining Workflows for Anomalous Data Transfers

Mining Workflows for Anomalous Data Transfers

Data reduction in scientific workflows using provenance monitoring and user steering

Optimizing computational resource management for the scientific gateways ecosystems based on the service‐oriented paradigm

Contact Info

Product

Resources

About