Algorithms and design for a second-order automatic differentiation module

Abate, Jason; Bischof, Christian; Roh, Lucas; Carle, Alan

doi:10.1145/258726.258770

Cited by 22 publications

(21 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Second-order information was tackled in automatic differentiation (AD) by Abate et al (1997), Giering and Kaminski (1998a,b), Gay (1996), Hovland (1995), Bischof (1995), Burger et al (1992), Griewank and Corliss (1991), and Griewank (1993Griewank ( , 2000Griewank ( , 2001, to cite but a few. Several AD packages such as the tangent linear and adjoint model compiler (TAMC) of Giering and Kaminski (1998a) allow calculation of the Hessian of the cost functional.…”

Section: Volume 130 M O N T H L Y W E a T H E R R E V I E Wmentioning

confidence: 99%

Second-Order Information in Data Assimilation*

2002

View full text Add to dashboard Cite

In variational data assimilation (VDA) for meteorological and/or oceanic models, the assimilated fields are deduced by combining the model and the gradient of a cost functional measuring discrepancy between model solution and observation, via a first-order optimality system. However, existence and uniqueness of the VDA problem along with convergence of the algorithms for its implementation depend on the convexity of the cost function. Properties of local convexity can be deduced by studying the Hessian of the cost function in the vicinity of the optimum. This shows the necessity of second-order information to ensure a unique solution to the VDA problem.In this paper a comprehensive review of issues related to second-order analysis of the problem of VDA is presented along with many important issues closely connected to it. In particular issues of existence, uniqueness, and regularization through second-order properties are examined. The focus then shifts to second-order information related to statistical properties and to issues related to preconditioning and optimization methods and second-order VDA analysis. Predictability and its relation to the structure of the Hessian of the cost functional is then discussed along with issues of sensitivity analysis in the presence of data being assimilated. Computational complexity issues are also addressed and discussed.Automatic differentiation issues related to second-order information are also discussed along with the computational complexity of deriving the second-order adjoint.Finally an application aimed at illustrating the use of automatic differentiation for deriving the second-order adjoint as well as the Hessian/vector product applied to minimizing a cost functional of a meteorological problem using the truncated-Newton method is presented. Results verifying numerically the computational cost of deriving the second-order adjoint as well as results related to the spectrum of the Hessian of the cost functional are displayed and discussed.

show abstract

Section: Volume 130 M O N T H L Y W E a T H E R R E V I E Wmentioning

confidence: 99%

Second-Order Information in Data Assimilation*

2002

View full text Add to dashboard Cite

show abstract

“…Since the evaluation of the function computed by the original code is tightly interleaved with the firstand second-order derivative computation, we perform a redundant evaluation of the original function and its gradients, enabling parallel processing without synchronization. The new implementation of the parallelization strategy using ADIFOR 3.0 [18] and the Hessian module [1] is able to generate ready-to-use parallel code for computing first-and second-order derivatives. Finally, we report on an application of the proposed strategy to the MSIS-86 atmospheric model leading to an augmented model capable of evaluating secondorder derivatives in parallel.…”

Section: Discussionmentioning

confidence: 99%

“…In practice, however, one would save half of the operations and storage by exploiting the symmetry of the Hessians. For example, in [1], the Hessians are stored in the LAPACK packed symmetric scheme.…”

Section: Automatic Differentiationmentioning

confidence: 99%

“…In this work, we specificly extend the strategy presented in [12] to the parallel computation of Hessians, and report on a recent implementation using the AD-IFOR 3.0 [18] automatic differentiation software and the language-independent Hessian module presented in [1] which is interfacing with ADIFOR and ADIC [8].…”

Section: Automatically Parallelizing Hessian Computationsmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Generation of Parallel Code for Hessian Computations

Bücker

Rasch

Vehreschild

2008

OpenMP Shared Memory Parallel Programming

View full text Add to dashboard Cite

Abstract. Given a program to compute some function, automatic differentiation can be used to mechanically generate another program capable of evaluating first-and higher-order derivatives of that function. A new strategy for the computation of Hessians by automatic differentiation is proposed where the generated code is automatically parallelized using OpenMP. The approach is applied to compute second-order derivatives of an atmospheric reference model and performance results on a Sun Fire E6900 system are reported.

show abstract

“…In both cases we a r e i n terested in the ratio 2 Direct and substitution methods usually require more than M gradient e v aluations to determine the Hessian matrix, and thus the increase in the value of 2 relative t o 1 in Table 3.2 was expected. Still, it is reassuring that the median value of 2 is reasonably small.…”

Section: Computing Hessian Matricesmentioning

confidence: 98%

Automatic Differentiation Tools in Optimization Software

Moré¹

2002

Automatic Differentiation of Algorithms

View full text Add to dashboard Cite

We discuss the role of automatic di erentiation tools in optimization software. We emphasize issues that are important to large-scale optimization and that have p r o ved useful in the installation of nonlinear solvers in the NEOS Server. Our discussion centers on the computation of the gradient and Hessian matrix for partially separable functions and shows that the gradient and Hessian matrix can be computed with guaranteed bounds in time and memory requirements.

show abstract

Algorithms and design for a second-order automatic differentiation module

Cited by 22 publications

References 8 publications

Second-Order Information in Data Assimilation*

Second-Order Information in Data Assimilation*

Automatic Generation of Parallel Code for Hessian Computations

Automatic Differentiation Tools in Optimization Software

Contact Info

Product

Resources

About