The SENSEI Generic In Situ Interface

Ayachit, Utkarsh; Whitlock, Brad; Wolf, Matthew; Loring, Burlen; Geveci, Berk; Lonie, David; Bethel, E. Wes

doi:10.1109/isav.2016.013

Cited by 68 publications

(29 citation statements)

References 9 publications

(13 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given the variety of in transit systems, there is an effort to unify them under a common API known as SENSEI [2]. Current implementation of the SENSEI library could make use of many of the well-known transport mechanisms [1].…”

Section: Previous Workmentioning

confidence: 99%

See 1 more Smart Citation

HDF5 as a vehicle for in transit data movement

Loring

et al. 2019

Proceedings of the Workshop on in Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization

Self Cite

View full text Add to dashboard Cite

For in transit processing, one of the fundamental challenges is the efficient movement of data from producers to consumers. Exploiting the flexibility offered by the SENSEI generic in situ framework, we have developed a number of different in transit data transport mechanisms. In this work, we focus on the transport mechanism that leverages the HDF5 parallel I/O library, and investigate the performance characteristics of this transport mechanism. For in transit use cases at scale on HPC platforms, one might expect that an in transit data transport mechanism that uses faster layers of the storage hierarchy, such as DRAM memory, would always outperform a transport that uses slower layers of the storage hierarchy, such as an NVRAM-based persistent storage presented as a distributed file system. However, our test results show that the performance of the transport using NVRAM is competitive with the transport that uses socket-based data movement across varying levels of producer and consumer concurrency. CCS CONCEPTS • Software and its engineering → Massively parallel systems; • Theory of computation → Parallel computing models; • Computing methodologies → Massively parallel algorithms; Massively parallel and high-performance simulations;

show abstract

Section: Previous Workmentioning

confidence: 99%

“…. $15.00 https://doi.org/10.1145/3364228.3364237 the SENSEI generic in situ interface [2] as one of several potential in transit transport mechanisms that can be selected through an XML-based configuration file.…”

Section: Introductionmentioning

confidence: 99%

HDF5 as a vehicle for in transit data movement

Loring

et al. 2019

Proceedings of the Workshop on in Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization

Self Cite

View full text Add to dashboard Cite

show abstract

“…The specification of this interface is an important step and may require a significant effort. Alternatively, a generic interface, such as SENSEI [27], may be used to replace the adaptor. Figure 1 illustrates the integration among libMesh-sedimentation with Catalyst and DfAnalyzer.…”

Section: In Situ Data Extraction and Visualization Using Paraview Catmentioning

confidence: 99%

In situ visualization and data analysis for turbidity currents simulation

Camata

Silva

Valduriez

et al. 2018

Computers & Geosciences

View full text Add to dashboard Cite

International audienceTurbidity currents are underflows responsible for sediment deposits that generate geological formations of interest for the oil and gas industry. libMesh-sedimentation is an application built upon the libMesh library to simulate turbidity currents. In this work, we present the integration of libMesh-sedimentation with in situ visualization and in transit data analysis tools. DfAnalyzer is a solution based on provenance data to extract and relate strategic simulation data in transit from multiple data for online queries. We integrate libMesh-sedimentation and ParaView Catalyst to perform in situ data analysis and visualization. We present a parallel performance analysis for two turbidity currents simulations showing that the overhead for both in situ visualization and in transit data analysis is negligible. We show that our tools enable monitoring the sediments appearance at runtime and steer the simulation based on the solver convergence and visual information on the sediment deposits, thus enhancing the analytical power of turbidity currents simulations

show abstract

“…Virtually all major in situ infrastructures, e.g., ADIOS [8], ParaView Catalyst [3,15], VisIt Libsim [5,17], SENSEI [2] through Libsim and Catalyst, and Ascent [6], provide some form of support for Python use by employing one of two approaches. In one, a framework exposes capabilities through a set of wrapped objects or functions that can be used in Python programs.…”

Section: Previous Workmentioning

confidence: 99%

“…SENSEI [2] is a lightweight, generic in situ interface providing normalized access to a diverse set of analysis back ends via a simple API and rich data model, and that has been shown to scale to over 1M cores [1]. With no coding changes, a simulation instrumented with the SENSEI interface has runtime-switchable access to multiple in situ backends, such as Libsim, Catalyst, ADIOS, VTK-m, and a host of others.…”

Section: Previous Workmentioning

confidence: 99%

Python-based in situ analysis and visualization

Loring

Myers

Camp

et al. 2018

Proceedings of the Workshop on in Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization

Self Cite

View full text Add to dashboard Cite

This work focuses on enabling the use of Python-based methods for the purpose of performing in situ analysis and visualization. This approach facilitates access to and use of a rapidly growing collection of Python-based, third-party libraries for analysis and visualization, as well as lowering the barrier to entry for userwritten Python analysis codes. Beginning with a simulation code that is instrumented to use the SENSEI in situ interface, we present how to couple it with a Python-based data consumer, which may be run in situ, and in parallel at the same concurrency as the simulation. We present two examples that demonstrate the new capability. One is an analysis of the reaction rate in a proxy simulation of a chemical reaction on a 2D substrate, while the other is a coupling of an AMR simulation to Yt, a parallel visualization and analysis library written in Python. In the examples, both the simulation and Python in situ method run in parallel on a large-scale HPC platform. CCS CONCEPTS• Software and its engineering → Massively parallel systems;• Theory of computation → Parallel computing models; • Computing methodologies → Massively parallel algorithms; Massively parallel and high-performance simulations; KEYWORDSPython, in situ analysis, in situ visualization ACM Reference Format:

show abstract

The SENSEI Generic In Situ Interface

Cited by 68 publications

References 9 publications

HDF5 as a vehicle for in transit data movement

HDF5 as a vehicle for in transit data movement

In situ visualization and data analysis for turbidity currents simulation

Python-based in situ analysis and visualization

Contact Info

Product

Resources

About