Consistent treatment of incompletely converged iterative linear solvers in reverse-mode algorithmic differentiation

Abstract

Algorithmic differentiation (AD) is a widely-used approach to compute derivatives of numerical models. Many numerical models include an iterative process to solve non-linear systems of equations. To improve efficiency and numerical stability, AD is typically not applied to the linear solvers. Instead, the differentiated linear solver call is replaced with hand-produced derivative code that exploits the linearity of the original call. In practice, the iterative linear solvers are often stopped prematurely to recompute the linearisation of the non-linear outer loop. We show that in the reverse-mode of AD, the derivatives obtained with partial convergence become inconsistent with the original and the tangent-linear models, resulting in inaccurate adjoints. We present a correction term that restores consistency between adjoint and tangent-linear gradients if linear systems are only partially converged. We prove the consistency of this correction term and show in numerical experiments that the accuracy of adjoint gradients of an incompressible flow solver applied to an industrial test case is restored when the correction term is used.

Geometry and Topology Optimization of Switched Reluctance Machines: A Review

“…It repeatedly applies the chain rule to the program sequence of elementary arithmetic operations and functions. The AD method has two operating modes, namely forward accumulation and reverse accumulation [99]. The two modes compute the gradient of the function with a seed vector that has the same number of the function inputs or outputs for forward accumulation and reverse accumulation, respectively [99].…”

“…The AD method has two operating modes, namely forward accumulation and reverse accumulation [99]. The two modes compute the gradient of the function with a seed vector that has the same number of the function inputs or outputs for forward accumulation and reverse accumulation, respectively [99]. The AD method was used to find the sensitivity of the electromagnetic force to different geometric parameters of a linear actuator in [100].…”

Pitfalls of Discrete Adjoint Fixed-Points Based on Algorithmic Differentiation

“…Both (7) and (11) are duality preserving with respect to tangent linearizations based on their respective definition of residual, provided the transposition of M is consistent.…”

Approximate Linearization of Fixed Point Iterations: Error Analysis of Tangent and Adjoint Problems Linearized about Non-Stationary Points

“…The final set shows the impact of exactly linearizing the residual operator at every stage of an RK scheme when the gradients are only computed at the first stage and then frozen throughout the analysis. All cases were run on an unstructured triangular mesh consisting of 4212 elements shown in Figure (1), in M = .7 flow with α = 2 o , with 2 Hickes-Henne bump functions [6] used as design variables to perturb the airfoil surface.…”

“…Padway and Mavriplis [16] showed by numerical experiment that for an approximately linearized quasi-Newton fixed point iteration, convergence of the non-linear problem led to a decrease in error from the approximate linearization. The issue of inexact linearization of a linear system solve has been investigated previously to produce consistent automatic differentiation of linear system solves in segregated solvers [1]. These linearizations would be necessary in the "piggy-back" iterations of the one-shot adjoint method [4] when applied to implicit nonlinear solvers.…”