Parallel time integration with multigrid

Abstract

We consider optimal-scaling multigrid solvers for the linear systems that arise from the discretization of problems with evolutionary behavior. Typically, solution algorithms for evolution equations are based on a time-marching approach, solving sequentially for one time step after the other. Parallelism in these traditional time-integration techniques is limited to spatial parallelism. However, current trends in computer architectures are leading towards systems with more, but not faster, processors. Therefore, faster compute speeds must come from greater parallelism. One approach to achieve parallelism in time is with multigrid, but extending classical multigrid methods for elliptic operators to this setting is not straightforward. In this paper, we present a non-intrusive, optimal-scaling time-parallel method based on multigrid reduction (MGR). We demonstrate optimal-ity of our multigrid-reduction-in-time algorithm (MGRIT) for solving diffusion equations in two and three space dimensions in numerical experiments. Furthermore, through both parallel performance models and actual parallel numerical results, we show that we can achieve significant speedup in comparison to sequential time marching on modern architectures. 1. Introduction. One of the major challenges facing the computational science community with future architectures is that faster compute speeds must come from increased concurrency, since clock speeds are no longer increasing but core counts are going up sharply. As a consequence, traditional time marching is becoming a huge sequential bottleneck in time integration simulations in the following way: improving simulation accuracy by scaling up the spatial resolution requires a similar (or greater) increase in the temporal resolution, which is also required to maintain stability in explicit methods. As a result, numerical time integration involves many more time steps leading to long overall compute times, since parallelizing only in space limits concurrency. Solving for multiple time steps in parallel and, therefore, increasing concurrency would remove this time integration bottleneck. Because time is sequential in nature, the idea of simultaneously solving for multiple time steps is not intuitive. Yet it is possible, with work on this topic going back to as early as 1964 [33]. However, most research on this subject has been done within the past 30 years including [2, 7-10, 14-22, 25, 28, 31, 32, 38, 40-44]. One approach to achieve parallelism in time is with multigrid methods. The parareal in time method, introduced by Lions, Maday, and Turinici in [25], can be interpreted as a two-level multigrid method [16], even though the leading idea came from a spatial domain decomposition approach. The algorithm is optimal, but concurrency is limited since the coarse-grid solve is still sequential. Considering true multilevel (not two-level) schemes, only a few methods exhibit full multigrid optimality and concurrency such as [21, 42,43], and most are designed for specific problems or discretizations. Furt...

50 Years of Time Parallel Time Integration

“…We see in the convergence estimate (24) the same term appear as in the parareal convergence estimate (21). This term is typical for the convergence of waveform relaxation methods we will see next, and thus the comment of Saha, Stadel and Tremaine in the quote at the beginning of Subsection 2.4 is justified.…”

“…Such a result on a 512 processor machine was shown for the space-time multigrid waveform relaxation algorithm in [79], compared to space parallel multigrid waveform relaxation and standard time stepping, see also [47] for results on an even larger machine, and [5]. A very recent comparison can be found in the short note [21], where the authors show that above a certain number of processors time-parallel algorithms indeed outperform classical ones. Time parallel methods are currently a very active field of research, and many new developments extending the latest directions we have seen, like Parareal, Schwarz-, Dirichlet-Neumann and Neumann-Neumann waveform relaxation, PFASST and full space-time multigrid, and RIDC and ParaExp, are to be expected over the coming years.…”

“…Next looking in (21), we see that the method's order increases at each iteration k by p, the order of the coarse propagator, as already shown by Lions, Maday and Turinici in their proposition for the Euler method. We have however also a precise estimate of the constant in front in (21), and this constant contracts faster than linear, since it is an algebraic power of C 1 T divided by k! (the exponential term is not growing as the iteration k progresses).…”

Algorithm 965

“…PyPFASST [Emmett 2013] is a python implementation of a modified parareal solver for ODEs and PDEs [Emmett and Minion 2012]. XBRAID [Schroder et al 2015] is a C library that implements a multigrid-reduction-in-time algorithm [Falgout et al 2014], where multiple time-grids of different granularity are distributed across processors using MPI. PFASST++ [Emmett et al 2015] is a C++ implementation of the " parallel full approximation scheme in space and time (PFASST) algorithm [Emmett and Minion 2014].…”

Multigrid method based on a space-time approach with standard coarsening for parabolic problems

“…This led us to look for the increase of the concurrency by using time-parallel and full spacetime methods. There are different multigrid approaches on space-time grids, among which the space-time multigrid [2] , the space-time concurrent multigrid waveform relaxation [3] , the multigrid-reduction-in-time [4] , the space-time parallel multigrid [5] , the parallel full approximation scheme in space and time [6] and the parareal method [7,8] are included. In this work, also a multigrid method to deal with the entire space-time problem, opposite to the standard time-stepping strategy, is proposed for the solution of the heat equation with one and two spatial dimensions.…”