Simultaneous Perturbation Stochastic Approximation with Norm‐Limited Update Vector

Abstract

This paper addresses the convergence of simultaneous perturbation stochastic approximation (SPSA) with a norm‐limited update vector. We first illustrate an unstable solution of the standard SPSA algorithm which motivates the consideration of a modified version, where the norm of the update vector is limited to a certain value. Next, a result on the almost‐sure convergence is presented by reducing the modified algorithm into the standard SPSA algorithm and restricting the probability distribution for the perturbation to a Bernoulli distribution. Finally, we apply the modified algorithm to a system identification problem to demonstrate its performance.

Broadcast control of multi-robot systems with norm-limited update vector

“…(iv) The detailed proof of fact (ii) is shown here. For the first condition, since 0 <âðkÞ < aðkÞ for all k 2 Z 0þ holds from equation ( 13) and C4, it follows a straightforward proof from Tanaka et al 19 which shows that our first condition of C4 0 holds. Now, we prove the second condition of C4 0 , that is,…”

“…Note that the general guidelines to design the gains aðkÞ and cðkÞ can be found in Sections 7.5.1 and 7.5.2 of Spall. 28 For the value of e, we referred to the example given in Tanaka et al 19 and adjusted this value according to the size of the deterministic movement taken by the robots that the user want, that is, the larger the value of e, the larger the deterministic movement of the robots. Figure 5 shows the snapshots of the initial positions of the robots x i ð0Þði ¼ 1; 2; .…”

“…In our modified broadcast control framework, a new local controller is derived so that it can limit the amount of the deterministic input in updating the robots' position. Note that the derivation is based on simultaneous perturbation stochastic approximation (SPSA) with norm-limited update vector algorithm adopted from Tanaka et al 19 The algorithm was adopted because of the practicality in solving various model-free optimization problems. [19][20][21] From this fact, we believe that the same idea can be applied to solve the instability problem in the standard broadcast control framework.…”

“…Note that the derivation is based on simultaneous perturbation stochastic approximation (SPSA) with norm-limited update vector algorithm adopted from Tanaka et al 19 The algorithm was adopted because of the practicality in solving various model-free optimization problems. [19][20][21] From this fact, we believe that the same idea can be applied to solve the instability problem in the standard broadcast control framework. On the other hand, we proved that the modified broadcast controller asymptotically had achieved the task with a probability of 1.…”

Experimental verification of active oscillation controller for vehicle drivetrain with backlash nonlinearity based on norm-limited SPSA

“…However, several researchers have indicated that the computation stability should be improved to apply SPSA for practical applications. 45 An effective approach is to explicitly limit the updating amount of design variable. 45 Therefore, the term a k g ^ k false( θ k false) is saturated as follows: In addition, by restricting Δ k ∈ double-struckR p to a Bernoulli distribution, equation (27) can be simply rewritten as follows: This improved version is referred to as SPSA with a norm-limited update vector (NLSPSA).…”