The Lifted Newton Method and Its Application in Optimization

Abstract

We present a new "lifting" approach for the solution of nonlinear optimization problems (NLPs) that have objective and constraint functions with intermediate variables. Introducing these as additional degrees of freedom into the original problem, combined with adding suitable new constraints to ensure equivalence of the problems, we propose to solve this augmented system instead of the original system by a Newton-type method. This often offers advantages in terms of convergence rates and region of attraction. The main contribution of this article is an efficient algorithmic trick to generate the quantities needed for a Newton-type method on the augmented ("lifted") system with (a) almost no additional computational cost per iteration compared to a nonlifted Newton method, and (b) with negligible programming burden. We derive lifted schemes for Newton's method, as well as for constrained Gauss-Newton and adjoint based sequential quadratic programming (SQP) methods, and show equivalence of the new efficiently lifted approaches with "full-space" lifted Newton iterations. We establish conditions on the intermediate functions that imply faster local quadratic convergence for lifted versus nonlifted Newton iterations, a phenomenon often observed in practice but not yet explained theoretically. Finally, we compare numerically the behavior of the lifted approach with the nonlifted approach on several test problems, including a large scale example with 27 million intermediate variables. The algorithms and examples are available as open-source code in the C++ package LiftOpt.

Convergence Guarantees for Moving Horizon Estimation Based on the Real-Time Iteration Scheme

“…k } is calculated and the state estimatex k at time t = k is given bŷ x k := ξ (1) k . Subsequently, ξ (1) k is shifted by a map S k and used as the initialization of the iterative scheme at the next time-step t = k + 1:…”

Closing the Loop – Predictive Lifted Newton Trajectory Tracking Algorithm

“…Following the idea presented in [17], the motion planning problem (5) will be solved using a task priority version of the Lifted Newton Method [1]. …”

“…Constraints have been incorporated into the system through extending the system by extra state variables. As a result, the trajectory tracking problem has been made equivalent to an unconstrained motion planning problem formulated in the extended system, and solved with the task priority version of the Lifted Newton method [1,16]. In this context, the task priority Lifted Newton algorithm can be regarded as "multiple shooting" version of a ECSA Jacobian algorithm [31].…”

Optimization as an Analysis Tool for Human Complex Problem Solving

“…Our approach formulates the simulation task as equality constraints of the optimization problem and thus allows us to apply modern optimization techniques, including simultaneous strategies that solve simulation and optimization tasks at the same time. They have shown to be superior in many cases; compare, e.g., [8], [7], [2].…”