Midcourse guidance law with neural networks

Abstract

A dual neural network 'adaptive critic approach' is used in this study to generate midcourse guidance commands for a missile to reach a predicted impact point while maximizing its final velocity. The adaptive critic approach is based on approximate dynamic programming. The first network, called a 'critic', network, outputs the Lagrangian multipliers arising in an optimal control formulation while the second network, called an 'action' network, outputs the optimal guidance/control. While a typical adaptive critic structure consists of a single critic and a single controller, the midcourse guidance problem needs indexing in terms of the independent variable and therefore there is a cascade of critics and controllers each set for a different index. Every controller learns from the critic at the previous stage. Though the networks are trained off-line, the resulting control is in a feedback form. A midcourse guidance problem is the first testbed for this approach where the input is vector-valued. The numerical results for a number of scenarios show that the network performance is excellent. Corroboration for optimality is provided by comparisons of the numerical solutions using a shooting method for a number of scenarios. Numerical results demonstrate some attractive features of the adaptive critic approach and show that this formulation works very well in guiding the missile to its final conditions from an envelope of initial conditions. This application also demonstrates the use of adaptive critics as a tool to solve a class of 'free final time' problems in optimal control, which are usually very difficult.

Intelligent supply chain management using adaptive critic learning

“…For our context, U(t) is the operating cost, so we have (7) A secondary utility function J(t) is then formed (8) which embodies the desired control objective, discounted by through time. This is Bellman's equation, and a useful identity that follows from it is the Bellman Recursion (9) A promising collection of approximation techniques based on estimating the function using this identity has been proposed by Werbos [25]- [27].…”

“…Most previous applications of adaptive critic methodologies have been in the context of continuous dynamic systems. Recent examples include applications such as an autopilot design for agile missile control [8] and an autonomous steering and speed control system for an automobile [11]. This paper demonstrates the use of adaptive critic based approximate dynamic programming techniques to the simultaneous optimization of inventory control and transportation policies for a discrete event physical distribution system operating within a changing business environment.…”