The development of supercomputers and multi-scale computational fluid dynamics (CFD) models based on adaptive mesh refinement (AMR) enabled fast, large-scale, and high fidelity CFD simulations. Interactive in situ steering is an effective tool for debugging, searching for optimal solutions, and analyzing inverse problems in such CFD simulations. We propose an interactive in situ steering framework for large-scale CFD simulations on GPU supercomputers. This framework employs in situ particle-based volume rendering (PBVR), in situ data sampling, and a file-based control that enables interactive and asynchronous communication of steering parameters, compressed visualization particle data, and sampled monitoring data between supercomputers and user PCs. The parallelized PBVR is processed on the host CPU to avoid interference with CFD simulations on the GPU. We apply the proposed framework to a real-time plume dispersion analysis code CityLBM, which computes the lattice Boltzmann method on the block AMR grid using GPU supercomputers. In the numerical experiment, we address an inverse problem to find a pollutant source from the observation data at monitoring points and demonstrate the effectiveness of the human-in-the-loop approach via the in situ steering framework.
Graphical abstract