In‐memory staging and data‐centric task placement for coupled scientific simulation workflows

Abstract

Coupled scientific simulation workflows are composed of heterogeneous component applications that simulate different aspects of the physical phenomena being modeled and that interact and exchange significant volumes of data at runtime. As the data volumes and generation rates keep growing, the traditional disk I/O-based data movement approach becomes cost prohibitive, and workflow requires more scalable and efficient approach to support the data movement. Moreover, the cost of moving large volume of data over system interconnection network becomes dominating and significantly impacts the workflow execution time. Minimize the amount of network data movement and localize data transfers are critical for reducing such cost. To achieve this, workflow task placement should exploit data locality to the extent possible and move computation closer to data. In this paper, we investigate applying in-memory data staging and data-centric task placement to reduce the data movement cost in large-scale coupled simulation workflows. Specifically, we present a distributed data sharing and task execution framework that (1) co-locates in-memory data staging on application compute nodes to store data that needs to be shared or exchanged and (2) uses data-centric task placement to map computations onto processor cores that a large portion of the data exchanges can be performed using the intra-node shared memory. We also present the implementation of the framework and its experimental evaluation on Titan Cray XK7 petascale supercomputer. KEYWORDScoupled simulations, data-intensive application workflows, data-centric task mapping, data staging 1 INTRODUCTION Emerging coupled simulation workflows are composed of multiple applications that interact and exchange data at runtime and have the potential to achieve higher accuracy and accelerate the data to insight process. Multi-physics multi-model simulation workflow simulates different aspects of the phenomena being modeled by coupling multiple physical models. For example, in the Community Earth System Model 1 application workflow, separate simulations are coupled to model the interaction of the earth's ocean, atmosphere, land surface, and sea ice. Meanwhile, online data analytics workflow supports analyzing raw simulation data while it is being generated. For example, online analytics workflow for combustion simulation S3D 2 extracts and streams simulation data to a number of analysis operations, eg, visualization, descriptive statistics, which execute concurrently and in parallel with the simulation.However, running coupled simulation workflows on extreme-scale computing system is non-trivial. One challenge of running coupled simulation workflow at scale is providing efficient data movement mechanism to extract data from 1 simulation, transfer, and redistribute the data to another simulation or analysis. Traditionally, workflow applications share and exchange data using distributed file systems. 3-6 But the increasing performance gap between computation and disk I/O introduces significan...

SIM-SITU: A Framework for the Faithful Simulation of in-situ Workflows

“…Studying such strategies would be a first step in evaluating the impact of data transfers and network performance on the execution of scientific workflows. We also plan to extend the simulation capacities and realism of Sim-Situ by developing more complex versions of the Data Transport Layer component that mimic the behavior of popular implementations such as DataSpaces [14] or Dimes [15]. The objective is to provide Sim-Situ users with the capacity to select which flavor of the DTL they want to use in the simulation of their in-situ workflows.…”

“…A SimGrid plugin is a standalone piece of code that composes some of basic concepts exposed by the SimGrid API to offer a higher level of abstraction useful in a specific context. In this paper, the proposed plugin relies on data structures accessed through a Producer-Consumer synchronization mechanism commonly used in actual DTLs [14,15]. Using this plugin only requires to include an extra header file in the application code to use the functions it provides.…”

SIM-SITU: A Framework for the Faithful Simulation of in situ Processing

“…Among the fifteen in situ processing systems characterized in [2], eleven follow the applicationaware approach, i.e., rely on an API for the integration of analysis/visualization with a numerical simulation. Moreover, five of these systems focus on data movements between components and implement a data transport layer [15]- [19]. As this is considered as the most flexible, extensible, and efficient way to implement in situ processing, we decided to simulate such a DTL in SIM-SITU.…”

Simulations of Future Particle Accelerators: Issues and Mitigations

“…Such an in-memory or cross-node data transfer can be made to respect physics constraints (conservation, divergence free conditions, etc.) and is also much more efficient than file I/O based transfers [42][43][44]. This need can be best served with the common low-level data layouts and interfaces as mentioned above and/or by promoting/enabling interoperable mesh/particle capabilities among existing tools in the ecosystem.…”