Efficient use of sparsity by direct solvers applied to 3D controlled-source EM problems

Abstract

Controlled-source electromagnetic (CSEM) surveying becomes a widespread method for oil and gaz exploration, which requires fast and efficient software for inverting large-scale EM datasets. In this context, one often needs to solve sparse systems of linear equations with a large number of sparse right-hand sides, each corresponding to a given transmitter position. Sparse direct solvers are very attractive for these problems, especially when combined with low-rank approximations which significantly reduce the complexity and the cost of the factorization. In the case of thousands of right-hand sides, the time spent in the sparse triangular solve tends to dominate the total simulation time and here we propose several approaches to reduce it. A significant reduction is demonstrated for marine CSEM application by utilizing the sparsity of the right-hand sides (RHS) and of the solutions that results from the geometry of the problem. Large gains are achieved by restricting computations at the forward substitution stage to exploit the fact that the RHS matrix might have empty rows (vertical sparsity) and/or empty blocks of columns within a non-empty row (horizontal sparsity). We also adapt the parallel algorithms that were designed for the factorization to solve-oriented algorithms and describe performance optimizations particularly relevant for the very large numbers of right-hand sides of the CSEM application. We show that both the operation count and the elapsed time for the solution phase can be significantly reduced. The total time of CSEM simulation can be divided by approximately a factor of 3 on all the matrices from our set (from 3 to 30 million unknowns, and from 4 to 12 thousands RHSs). Exploitation efficace du creux dans les solveurs directs sur des problèmes d'électromagnétisme 3D à source contrôléeRésumé : L'électromagnétisme à source controlée (CSEM) est une méthode de plus en plus utilisée pour l'exploration du gaz et du pétrole. L'inversion des données électromagnétiques nécessite souvent la résolution de systèmes d'équations linéaires avec un grand nombre de seconds membres creux, chacun correspondant à la position d'une source/d'un émetteur dans l'application. Les solveurs creux directs sont très attrayants pour ce type de problèmes, surtout lorsqu'ils sont combinés avec des approximations de rang faible qui réduisent la complexité et le coût de la factorisation. Comme il peut y avoir des milliers de seconds membres, le coût de la phase de résolution devient alors prédominant. Dans cet article, nous montrons qu'il est possible d'exploiter la géométrie du problème et la structure creuse qui apparaît à la fois dans les seconds membres et dans la matrice des solutions et que cela peut avoir un impact important sur les performances des phases de résolutions triangulaires. Pour cela, nous organisons les calculs de manière à minimiser le nombre d'opérations, tout en traitant les seconds membres par blocs qui tiennent en mémoire et qui visent à conserver au mieux le parallélisme lors de la phae de résolution....

Fast multi-source nanophotonic simulations using augmented partial factorization

“…Beyond photonics, APF can be used for mapping the angle dependence of radar cross-sections, for microwave imaging 50 , for full waveform inversion 51 and controlled-source electromagnetic surveys 38 in geophysics and for quantum transport simulations 52 . More generally, APF can efficiently evaluate matrices of the form CA −1 B in numerical linear algebra, not limited to partial differential equations.…”

“…37 ), which does not utilize the structure of equation (2). While advanced algorithms have been developed to exploit the sparsity of the inputs and the outputs during forward and backward substitutions 38 or through domain decomposition 39 , they We compute the scattering matrix with up to 2W/λ = 1,000 plane-wave inputs from either the left or right and with all of the M ′ = 2,000 outgoing plane waves. b, Computing time versus the number M of input angles using APF and other methods: conventional FDFD method using MaxwellFDFD with direct 40 or iterative 41 solvers for the full-basis solutions, RCWA using S4 (ref.…”

Fast multi-source nanophotonic simulations using augmented partial factorization

“…Predicting the thermal emission from nanostructures [86] requires a large number of frequency-domain simulations with different dipole sources, which can also be vastly accelerated using APF. Beyond optics, APF can be used for mapping the angle dependence of radar crosssections, for microwave imaging [87], for full waveform inversion [88] and controlled-source electromagnetic surveys [61] in geophysics, and for quantum transport simulations [89]. More generally, APF can efficiently evaluate matrices of the form CA −1 B for other applications of numerical linear algebra, not limited to wave physics.…”

“…(2). While advanced algorithms have been developed to exploit the sparsity of the input and the output during forward and backward substitutions [61] or through domain decomposition [62], they still require an O(M ) substitution stage, with a modest speed-up (a factor of 3 when M is several thousands) and no memory usage reduction. APF is simpler yet much more efficient as it obviates the forward and backward substitution steps and the need for LU factors.…”

Three-dimensional finite-difference finite-element frequency-domain wave simulation with multi-level optimized additive Schwarz domain-decomposition preconditioner: A tool for FWI of sparse node datasets

“…In this section, we assess two direct solvers and orthogonalization schemes of the Krylov basis. We test the Intel MKL PARDISO solver (Alappat et al, 2020;Böllhofer et al, 2019Böllhofer et al, , 2020 and the MUMPS solver (Amestoy et al, 2018(Amestoy et al, , 2019bMUMPS team, 2021) to factorize the local matrices tied to each subdomain. When using MUMPS, we test both the full-rank (FR) and the block low-rank (BLR) version of the solver, referred to as MUMPS F R and MUMPS BLR , respectively (Amestoy et al, 2016(Amestoy et al, , 2019a.…”