Python-based in situ analysis and visualization

Abstract

This work focuses on enabling the use of Python-based methods for the purpose of performing in situ analysis and visualization. This approach facilitates access to and use of a rapidly growing collection of Python-based, third-party libraries for analysis and visualization, as well as lowering the barrier to entry for userwritten Python analysis codes. Beginning with a simulation code that is instrumented to use the SENSEI in situ interface, we present how to couple it with a Python-based data consumer, which may be run in situ, and in parallel at the same concurrency as the simulation. We present two examples that demonstrate the new capability. One is an analysis of the reaction rate in a proxy simulation of a chemical reaction on a 2D substrate, while the other is a coupling of an AMR simulation to Yt, a parallel visualization and analysis library written in Python. In the examples, both the simulation and Python in situ method run in parallel on a large-scale HPC platform. CCS CONCEPTS• Software and its engineering → Massively parallel systems;• Theory of computation → Parallel computing models; • Computing methodologies → Massively parallel algorithms; Massively parallel and high-performance simulations; KEYWORDSPython, in situ analysis, in situ visualization ACM Reference Format:

Diva: A Declarative and Reactive Language for In-Situ Visualization

“…The high-level scripting language based method directly integrates the popular general-purpose scripting language Python into a software framework. This method either requires the framework to expose functionalities through a set of wrapped Python objects or an embedded Python interpreter, or to allow users to directly inject customized scripts in the execution pipeline [27]. As a result, user supplied scripts can fully apply the powerful analysis capabilities of Python to do the heavy lifting in the process of in situ visualization and analysis.…”