Enhanced Resource Management Enabling Standard Parameter Sweep Jobs for Scientific Applications

Abstract

Parameter sweeps are used by researchers with scientific domain-specific tools or workflows to submit a large collection of computational jobs whereby each single job of it only varies in certain parts. They require a more fine-grained distribution of jobs across resources, which also raise a significant challenge for efficient resource management in middleware environments that have been not specifically designed to perform parameter sweeps. This paper offers insights into parameter sweep solutions that support multidisciplinary science environments via abstraction from resource management complexities using middleware. The solutions are based on use case requirements, enable efficient submission, enhanced usability, and standard compliance. We also apply a use case taken from the life science domain to demonstrate usefulness and efficiency of the solutions.