Beautiful differentiation

Elliott, Conal

doi:10.1145/1596550.1596579

Cited by 36 publications

(7 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent AD systems, such as MYIA, TANGENT, and those in footnote 1, as well as the HASKELL AD package available on Cabal (Kmett, 2010), the "Beautiful Differentiation" system (Elliott, 2009), and the "Compiling to Categories" system (Elliott, 2017), have been implemented for higher-order languages like SCHEME, ML, HASKELL, F7, PYTHON, LUA, and JULIA. One by one, many of these systems have come to discover the kind of perturbation confusion reported by Siskind & Pearlmutter (2005) and have come to implement the tagging mechanisms reported by and .…”

Section: Resultsmentioning

confidence: 99%

Perturbation confusion in forward automatic differentiation of higher-order functions

Manzyuk¹,

Pearlmutter²,

Radul

et al. 2019

J. Funct. Prog.

View full text Add to dashboard Cite

Automatic Differentiation (AD) is a technique for augmenting computer programs to compute derivatives. The essence of AD in its forward accumulation mode is to attach perturbations to each number, and propagate these through the computation by overloading the arithmetic operators. When derivatives are nested, the distinct derivative calculations, and their associated perturbations, must be distinguished. This is typically accomplished by creating a unique tag for each derivative calculation and tagging the perturbations. We exhibit a subtle bug, present in fielded implementations which support derivatives of higher-order functions, in which perturbations are confused despite the tagging machinery, leading to incorrect results. The essence of the bug is this: a unique tag is needed for each derivative calculation, but in existing implementations unique tags are created when taking the derivative of a function at a point. When taking derivatives of higher-order functions, these need not correspond! We exhibit a simple example: a higher-order function f whose derivative at a point x, namely f ′ (x), is itself a function which calculates a derivative. This situation arises naturally when taking derivatives of curried functions. Two potential solutions are presented, and their deficiencies discussed. One uses eta expansion to delay the creation of fresh tags in order to put them into oneto-one correspondence with derivative calculations. The other wraps outputs of derivative operators with tag substitution machinery. Both solutions seem very difficult to implement without violating the desirable complexity guarantees of Forward AD.

show abstract

Section: Resultsmentioning

confidence: 99%

Perturbation confusion in forward automatic differentiation of higher-order functions

Manzyuk¹,

Pearlmutter²,

Radul

et al. 2019

J. Funct. Prog.

View full text Add to dashboard Cite

show abstract

“…Symbolic differentiation [36] is used for many purposes. It is used to symbolically expand known derivatives:…”

Section: Differentiationmentioning

confidence: 99%

“…Symbolic Jacobian matrices consisting of derivatives have many applications, for example to speed up simulation runtime [19]. Such a matrix is often computed using automatic differentiation [36] that combines symbolic differentiation with other techniques to achieve fast computation. If there is no symbolic Jacobian available, a numerical Jacobian might instead be estimated by the numerical solvers.…”

Section: Differentiationmentioning

confidence: 99%

Tools and Methods for Analysis, Debugging, and Performance Improvement of Equation-Based Models

Sjölund¹

2015

Linköping Studies in Science and Technology. Dissertations

View full text Add to dashboard Cite

Equation-based object-oriented (EOO) modeling languages such as Modelica provide a convenient, declarative method for describing models of cyber-physical systems. Because of the ease of use of EOO languages, large and complex models can be built with limited effort. However, current state-of-the-art tools do not provide the user with enough information when errors appear or simulation results are wrong. It is of paramount importance that such tools should give the user enough information to correct errors or understand where the problems that lead to wrong simulation results are located. However, understanding the model translation process of an EOO compiler is a daunting task that not only requires knowledge of the numerical algorithms that the tool executes during simulation, but also the complex symbolic transformations being performed.As part of this work, methods have been developed and explored where the EOO tool, an enhanced Modelica compiler, records the transformations during the translation process in order to provide better diagnostics, explanations, and analysis. This information is used to generate better error-messages during translation. It is also used to provide better debugging for a simulation that produces unexpected results or where numerical methods fail.Meeting deadlines is particularly important for real-time applications. It is usually essential to identify possible bottlenecks and either simplify the model or give hints to the compiler that enable it to generate faster code. When profiling and measuring execution times of parts of the model the recorded information can also be used to find out why a particular system model executes slowly.Combined with debugging information, it is possible to find out why this system of equations is slow to solve, which helps understanding what can be done to simplify the model. A tool with a graphical user interface has been developed to make debugging and performance profiling easier. Both debugging and profiling have been combined into a single view so that performance metrics are mapped to equations, which are mapped to debugging information.The algorithmic part of Modelica was extended with meta-modeling constructs (MetaModelica) for language modeling. In this context a quite general approach to debugging and compilation from (extended) Modelica to C code was developed. That makes it possible to use the same executable format for simulation executables as for compiler bootstrapping when the compiler written in MetaModelica compiles itself.Finally, a method and tool prototype suitable for speeding up simulations has been developed. It works by partitioning the model at appropriate places and compiling a simulation executable for a suitable parallel platform. Populärvetenskaplig sammanfattningSom ingenjör lär man sig att förstå världen genom ekvationer. För att beskriva eller modellera ett fysikaliskt system, till exempel vid utveckling av en produkt, vill ingenjören använda sig av de ekvationer som han eller hon lärt sig från böcker eller fö...

show abstract

“…Symbolic differentiation (Elliott, 2009) is used for many purposes. It is used to symbolically expand known derivatives (19) or as an operation during index reduction.…”

Section: Differentiationmentioning

confidence: 99%

“…If there is no symbolic Jacobian available, a numerical one might instead be estimated by the numerical solvers. Such a matrix is often computed using automatic differentiation (Elliott, 2009) which combines symbolic and/or automatic differentiation with other techniques to achieve fast computation.…”

Section: Differentiationmentioning

confidence: 99%

Integrated Debugging of Modelica Models

Pop

Sjölund

Asghar

et al. 2014

MIC

View full text Add to dashboard Cite

The high abstraction level of equation-based object-oriented (EOO) languages such as Modelica has the drawback that programming and modeling errors are often hard to find. In this paper we present integrated static and dynamic debugging methods for Modelica models and a debugger prototype that addresses several of those problems. The goal is an integrated debugging framework that combines classical debugging techniques with special techniques for equation-based languages partly based on graph visualization and interaction.To our knowledge, this is the first Modelica debugger that supports both equation-based transformational and algorithmic code debugging in an integrated fashion.

show abstract

Beautiful differentiation

Cited by 36 publications

References 12 publications

Perturbation confusion in forward automatic differentiation of higher-order functions

Perturbation confusion in forward automatic differentiation of higher-order functions

Tools and Methods for Analysis, Debugging, and Performance Improvement of Equation-Based Models

Integrated Debugging of Modelica Models

Contact Info

Product

Resources

About