Speeding up Python-based Lagrangian Fluid-Flow Particle Simulations via Dynamic Collection Data Structures

Abstract

Array-like collection data structures are widely established in Python's scientific computing-ecosystem for high-performance computations. The structure maps well to regular, gridded lattice structures that are common to computational problems in physics and geosciences. High performance is, however, only guaranteed for static computations with a fixed computational domain. We show that for dynamic computations within an actively changing computational domain, the array-like collections provided by NumPy and its derivatives are a bottleneck for large computations. In response, we describe the integration of naturally-dynamic collection data structures (e.g. double-linked lists) into NumPy simulations and ctypes-based C-bindings. Our benchmarks verify and quantify the performance increase attributed to the change of the collection data structure. Our application scenario, a Lagrangian (oceanic) fluid-flow particle simulation within the Parcels framework, demonstrates the speed-up yield in a realistic setting and demonstrates the novel capabilities that are facilitated by optimised collection data structures.

Efficiently Simulating Lagrangian Particles in Large-Scale Ocean Flows – Data Structures and their Impact on Geophysical Applications

“…A scalability study was out of scope of the displayed experiments. That said, all available information, including available previous studies Kehl et al (2021), suggest that this behaviour scales proportionally with the number of particles.…”

“…Hence, implementing different particle set collections potentially reduces those delays. This change has already been benchmarked by Kehl et al (2021) for pure particle advection scenarios, while this paper investigates the I/O delay reduction in oceanographic scenarios with more extensively attributed particles.…”

“…In contrast to previous studies (Kehl et al, 2021), this article benchmarks the technical developments in operational oceanic simulations. The selected scenarios cover a range of computational conditions, as illustrated and referenced below.…”

“…This paper investigates the performance improvement of Lagrangian ocean simulations. A previous performance study on synthetic data (Kehl et al, 2021) indicated high I/O load as bottleneck for Lagrangian simulations. Hence, this study quantifies the I/O load on real oceanographic simulations.…”

A Novel Method for Simulating Micro-Scale Water Droplet Movements

“…There are three types of methods for fluid simulation: the eulerian method [1][2][3][4][5], the Lagrangian method [6][7][8][9][10] and the mixed method [11][12][13][14]. All of these methods are based on Navier-Stokes equations and work well in large-scale water simulations without regard to the water droplets' adhesion force, surface tension and other factors.…”