Particle simulations are important workloads in high performance and parallel computing. Due to massive particle migration between parallel processes during simulations, efficient and balanced parallel computing is a practical challenge in large-scale realistic particle applications. This paper proposes a novel approach to enable highly-efficient dynamic load balance in a coupled DSMC/PIC solver for large-scale numerical simulations of the plasma plume. We employ dual unstructured grids of different granularity, with a coarse grid for DSMC simulations of flow fields and an embedded fine grid for PIC simulations of electric fields, to facilitate coupled DSMC/PIC calculation and grid partition for parallel computing. We then design and implement a centralized as well as a distributed communication strategies to dynamically migrate particles among arbitrary parallel processes. During the timestep iterations, we present a lightweight dynamic load balancer, composed of a load imbalance factor, a weighted load model and an efficient grid remapping mechanism, to adaptively rebalance the simulation among parallel processes with little extra overheads. We perform 3D unsteady simulations of the plasma plume induced by the pulsed vacuum arc in a cylindrical nozzle with hydrogen atoms and ions to validate the coupled solver. Parallel performance results scaling up to thousands of processes with billions of particles demonstrate the efficiency and effectiveness of our dynamic load balancer and parallel implementation.

No abstract

The transport characteristics of the unsteady flow field in rarefied plasma plumes is crucial for a pulsed vacuum arc in which the particle distribution varies from 1016 to 1022 m−3. The direct simulation Monte Carlo (DSMC) method and particle-in-cell (PIC) method are generally combined to study this kind of flow field. The DSMC method simulates the motion of neutral particles, while the PIC method simulates the motion of charged ions. A hybrid DSMC/PIC algorithm is investigated here to determine the unsteady axisymmetric flow characteristics of vacuum arc plasma plume expansion. Numerical simulations are found to be consistent with the experiments performed in the plasma mass and energy analyzer (EQP). The electric field is solved by Poisson’s equation, which is usually computationally expensive. The compressed sparse row (CSR) format is used to store the huge diluted matrix and PETSc library to solve Poisson’s equation through parallel calculations. Double weight factors and two timesteps under two grid sets are investigated using the hybrid DSMC/PIC algorithm. The fine PIC grid is nested in the coarse DSMC grid. Therefore, METIS is used to divide the much smaller coarse DSMC grid when dynamic load imbalances arise. Two parameters are employed to evaluate and distribute the computational load of each process. Due to the self-adaption of the dynamic-load-balancing parameters, millions of grids and more than 150 million particles are employed to predict the transport characteristics of the rarefied plasma plume. Atomic Ti and Ti2+ are injected into the small cylinders. The comparative analysis shows that the diffusion rate of Ti2+ is faster than that of atomic Ti under the electric field, especially in the z-direction. The fully diffuse reflection wall model is adopted, showing that neutral particles accumulate on the wall, while charged ions do not—due to their self-consistent electric field. The maximum acceleration ratio is about 17.94.