High accuracy least-squares solutions of nonlinear differential equations

Mortari, Daniele; Johnston, Hunter; Smith, Lidia

doi:10.1016/j.cam.2018.12.007

Cited by 62 publications

(79 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where g(t) represents a "freely chosen" function, ηi are the coefficients derived from the k linear constraints, and pi(t) are user selected functions that must be linearly independent from g(t). Recent research has applied this technique to embedding DE constraints using Equation (1), allowing for least-squares (LS) solutions of initial-value (IVP), boundary-value (BVP), and multi-value (MVP) problems on both linear [8] and nonlinear [9] ordinary differential equations (ODEs). In general, this approach has developed a fast, accurate, and robust unified framework to solve DEs.…”

Section: Background On the Theory Of Functional Connectionsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…This process transforms the DE into an unconstrained optimization problem where the free function is used to search for the solution of the DE. Prior studies [8,9,10,11], have defined this free function as a summation of basis functions; more specifically, orthogonal polynomials.This work was motivated by recent results that solve ordinary DEs using a least-squares support vector machine (LS-SVM) approach [12]. While this article focuses on the application of LS-SVMs to solve DEs, the study and use of LS-SVMs remains relevant in many areas.…”

mentioning

confidence: 99%

“…This article compares the combined method, referred to hereafter as CSVM for constrained LS-SVMs, to vanilla versions of TFC [8,9] and LS-SVM [12] over a variety of DEs. In all cases, the vanilla version of TFC outperforms both the LS-SVM and the CSVM methods in terms of accuracy and speed.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Analytically Embedding Differential Equation Constraints into Least Squares Support Vector Machines Using the Theory of Functional Connections

Leake

Johnston

Smith

et al. 2019

MAKE

Self Cite

View full text Add to dashboard Cite

Differential equations (DEs) are used as numerical models to describe physical phenomena throughout the field of engineering and science, including heat and fluid flow, structural bending, and systems dynamics. While there are many other techniques for finding approximate solutions to these equations, this paper looks to compare the application of the Theory of Functional Connections (TFC) with one based on least-squares support vector machines (LS-SVM). The TFC method uses a constrained expression, an expression that always satisfies the DE constraints, which transforms the process of solving a DE into solving an unconstrained optimization problem that is ultimately solved via least-squares (LS). In addition to individual analysis, the two methods are merged into a new methodology, called constrained SVMs (CSVM), by incorporating the LS-SVM method into the TFC framework to solve unconstrained problems. Numerical tests are conducted on four sample problems: One first order linear ordinary differential equation (ODE), one first order nonlinear ODE, one second order linear ODE, and one two-dimensional linear partial differential equation (PDE). Using the LS-SVM method as a benchmark, a speed comparison is made for all the problems by timing the training period, and an accuracy comparison is made using the maximum error and mean squared error on the training and test sets. In general, TFC is shown to be slightly faster (by an order of magnitude or less) and more accurate (by multiple orders of magnitude) than the LS-SVM and CSVM approaches. IntroductionDifferential equations (DE) and their solutions are important topics in science and engineering. The solutions drive the design of predictive systems models and optimization tools. Currently, these equations are solved by a variety of existing approaches with the most popular based on the Runge-Kutta family [1]. Other methods include those which leverage low-order Taylor expansions, namely Gauss-Jackson [2] and Chebyshev-Picard iteration [3,4,5], which have proven to be highly effective. More recently developed techniques are based on spectral collocation methods [6]. This approach discretizes the domain about collocation points, and the solution of the DE is expressed by a sum of "basis" functions with unknown coefficients that are approximated in order to satisfy the DE as closely as possible. Yet, in order to incorporate boundary conditions, one or more equations must be added to enforce the constraints.The Theory of Functional Connections (TFC) is a new technique that analytically derives a constrained expression which satisfies the problem's constraints exactly while maintaining a function that can be freely chosen [7]. This theory, initially called "Theory of Connections", has been renamed for two reasons. First, the "Theory of Connections" already identifies a specific theory in differential geometry, and second, what this theory is actually doing is "functional interpolation", as it provides all functions satisfying a set of constraints in terms of a function ...

show abstract

Section: Background On the Theory Of Functional Connectionsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Analytically Embedding Differential Equation Constraints into Least Squares Support Vector Machines Using the Theory of Functional Connections

Leake

Johnston

Smith

et al. 2019

MAKE

Self Cite

View full text Add to dashboard Cite

show abstract

“…The general equation to derive these interpolating expressions, named constrained expressions, follows as, y(x, g(x)) = g(x) + n k=1 η k s k (x) (1) where g(x) is the free function, η k are unknown coefficients to be solved by imposing the n constraint conditions, and s k (x) are "support functions," which are a set of n linearly independent functions. In prior work [30,31] as well as in this paper, the s k (x) support function set has been selected as the monomial set. The η k coefficients are computed by imposing the constraints using Eq.…”

Section: Introductionmentioning

confidence: 99%

Least-Squares Solutions of Eighth-Order Boundary Value Problems Using the Theory of Functional Connections

2020

Self Cite

View full text Add to dashboard Cite

This paper shows how to obtain highly accurate solutions of eighth-order boundary-value problems of linear and nonlinear ordinary differential equations. The presented method is based on the Theory of Functional Connections, and is solved in two steps. First, the Theory of Functional Connections analytically embeds the differential equation constraints into a candidate function (called a constrained expression) that contains a function that the user is free to choose. This expression always satisfies the constraints, no matter what the free function is. Second, the free-function is expanded as a linear combination of orthogonal basis functions with unknown coefficients. The constrained expression (and its derivatives) are then substituted into the eighth-order differential equation, transforming the problem into an unconstrained optimization problem where the coefficients in the linear combination of orthogonal basis functions are the optimization parameters. These parameters are then found by linear/nonlinear least-squares. The solution obtained from this method is a highly accurate analytical approximation of the true solution. Comparisons with alternative methods appearing in literature validate the proposed approach.

show abstract