Stochastic searching on the line and its applications to parameter learning in nonlinear optimization

Abstract

We consider the problem of a learning mechanism (for example, a robot) locating a point on a line when it is interacting with an random environment which essentially informs it, possibly erroneously, which way it should move. In this paper we present a novel scheme by which the point can be learnt using some recently devised learning principles. The heart of the strategy involves discretizing the space and performing a controlled random walk on this space. The scheme is shown to be ε-optimal and to converge with probability 1. Although the problem is solved in its generality, its application in non-linear optimization has also been suggested. Typically, an optimization process involves working one's way toward the maximum (minimum) using the local information that is available. However, the crucial issue in these strategies is that of determining the parameter to be used in the optimization itself. If the parameter is too small the convergence is sluggish. On the other hand, if the parameter is too large, the system could erroneously converge or even oscillate. Our strategy can be used to determine the best parameter to be used in the optimization.

A Stochastic Search on the Line-Based Solution to Discretized Estimation

“…The principle with which we design the estimator, referred to to as the Stochastic Search on the Line Based Discretized Weak Estimator (SSLDE), is based on the solution to the Stochastic Search on the Line (SSL) problem, pioneered by Oommen in [7].…”

“…It is pertinent to mention that the SSL solution presented in [7] is not directly relevant to the problem currently studied. The reason for this is the following: The method proposed in [7] explicitly assumes the existence of an Oracle which informs the learning mechanism whether to go "right" or "left".…”

“…The reason for this is the following: The method proposed in [7] explicitly assumes the existence of an Oracle which informs the learning mechanism whether to go "right" or "left". Such an Oracle is non-existent in the estimation problem.…”

“…By virtue of discretization, our estimator realizes extremely fast adjustments of the running estimates by jumps, and it is thus able to robustly, and very quickly, track changes in the parameters of the distribution after a switch has occurred in the environment. The design principle of our strategy is based on a solution, pioneered by Oommen [7], for the Stochastic Search on the Line (SSL) problem. The SSL solution proposed in [7], assumes the existence of an Oracle which informs the LA whether to go "right" or "left".…”

“…The design principle of our strategy is based on a solution, pioneered by Oommen [7], for the Stochastic Search on the Line (SSL) problem. The SSL solution proposed in [7], assumes the existence of an Oracle which informs the LA whether to go "right" or "left". In our application domain, in order to achieve efficient estimation, we have to first infer (or rather simulate) such an Oracle.…”