Neural Network Output Optimization Using Interval Analysis

Abstract

Abstract-The problem of output optimization within a specified input space of neural networks (NNs) with fixed weights is discussed in this paper. The problem is (highly) nonlinear when nonlinear activation functions are used. This global optimization problem is encountered in the reinforcement learning (RL) community. Interval analysis is applied to guarantee that all solutions are found to any degree of accuracy with guaranteed bounds. The major drawbacks of interval analysis, i.e., dependency effect and high-computational load, are both present for the problem of NN output optimization. Taylor models (TMs) are introduced to reduce these drawbacks. They have excellent convergence properties for small intervals. However, the dependency effect still remains and is even made worse when evaluating large input do-

Guaranteed Globally Optimal continuous Reinforcement Learning

“…However, if Q is used, then during the training of the RL controller, it will often be necessary to determine the optimal action a = max a Q(s, a ) (5-2-1) for some state s. Thanks to the nonlinear nature of neural networks, the maximization in the above relation is very hard to do. It is possible to find the maximum (possibly using interval analysis, see (de Weerdt, Chu, & Mulder, 2009)) but this will computationally be very expensive, making the algorithm practically unfeasible. Thus, as value function V will be used.…”

“…In (de Weerdt et al, 2009), methods are explained with which the neural network output can be optimized using interval analysis. Given a neural network, this article discusses methods to maximize (or equivalently minimize) the NN output.…”

“…Whether the methods discussed in (de Weerdt et al, 2009) will be useful for the current project is still unsure. One might initially think they will not.…”

“…Then the way in which the policy learns should be altered. It is possible, using the methods discussed in (de Weerdt et al, 2009), to exactly find the optimal action a for a state s, given the value function V and the identifier I. This optimal action can subsequently be used to train the policy π, after which it will converge to the correct values as well.…”

Solving the linear interval tolerance problem for weight initialization of neural networks

“…Research in the area of neural networks has also benefitted from IA and a number of efforts utilizing concepts and methods from IA are reported in the literature. Examples are those by de Weerdt et al (2009) on the use of IA for optimizing the neural network output, Ishibuchi & Nii (1998) on the generalization ability of neural networks, Xu et al (2005) on robust stability criteria for interval neural networks, Li et al (2007) regarding training of neural networks, and others.…”