Optimization over discrete sets via SPSA

Gerencsér, László; Hill, S.D.; Vágó, ZS

doi:10.1109/cdc.1999.830893

Cited by 19 publications

(22 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In its basic form as outlined above, SPSA can only operate on unbounded continuous sets, and is thus unsuited for optimization on our bounded integer lattice P. A modified SPSA algorithm for such problems was first proposed and analyzed by Gerencsér et al [10,11]. While their method involved fixed gain step lengths and did not incorporate bounds, both points are easily integrated.…”

Section: An Integer Spsa Algorithmmentioning

confidence: 98%

On optimization algorithms for the reservoir oil well placement problem

et al. 2006

View full text Add to dashboard Cite

Determining optimal locations and operation parameters for wells in oil and gas reservoirs has a potentially high economic impact. Finding these optima depends on a complex combination of geological, petrophysical, flow regimen, and economical parameters that are hard to grasp intuitively. On the other hand, automatic approaches have in the past been hampered by the overwhelming computational cost of running thousands of potential cases using reservoir simulators, given that each of these runs can take on the order of hours. Therefore, the key issue to such automatic optimization is the development of algorithms that find good solutions with a minimum number of function evaluations. In this work, we compare and analyze the efficiency, effectiveness, and reliability of several optimization algorithms for the well placement problem. In particular, we consider the simultaneous perturbation stochastic approximation (SPSA), finite difference gradient (FDG), and very fast simulated annealing (VFSA) algorithms. None of these algorithms guarantees to find the optimal solution, but we show that both SPSA and VFSA are very efficient in finding nearly optimal solutions with a high probability. We illustrate this with a set of numerical experiments based on real data for single and multiple well placement problems.

show abstract

Section: An Integer Spsa Algorithmmentioning

confidence: 98%

On optimization algorithms for the reservoir oil well placement problem

et al. 2006

View full text Add to dashboard Cite

show abstract

“…, c , where , was introduced in [7]. In that algorithm, the difference estimate in (11) (12) provides an estimate of the minimum of the extension .…”

Section: The Spsa Methodsmentioning

confidence: 99%

“…An ordinal optimization method for finding the minimum was introduced in [6]. The method discussed here, which was introduced in [7], relies on simultaneous perturbation difference approximations.…”

Section: Introductionmentioning

confidence: 99%

Stochastic approximation on discrete sets using simultaneous difference approximations

Hill

Gerencsér

Vágó

2004

Proceedings of the 2004 American Control Conference

Self Cite

View full text Add to dashboard Cite

Abstract-A stochastic approximation method for optimizing a class of discrete functions is considered. The procedure is a version of the Simultaneous Perturbation Stochastic Approximation (SPSA) method that has been modified to obtain a stochastic optimization method for cost functions defined on a grid of points in Euclidean p-space having integer components. We discuss the algorithm and examine its convergence properties.

show abstract

“…For parameter optimisation in games when the parameters to be optimised are discrete-valued, a variant of SPSA due 4 The independence assumptions on the components of t make SPSA fundamentally different from the method of random directions stochastic approximation, RDSA, where the perturbation vector is sampled uniformly in the d-dimensional unit sphere and the two-sided differences are multiplied by ti , unlike in SPSA where they are multiplied by −1 ti . Springer to Gerencsér, Hill and Vágó (1999) could be used, though the analysis given by the authors there is limited to convex functions (just like for the previously cited work). Actually, in games defined over discrete structures, often the parameters are themselves continuous-valued, but the objective function is discontinuous, e.g., piecewise constant.…”

Section: Spsamentioning

confidence: 99%

Universal parameter optimisation in games based on SPSA

Kocsis

Szepesvári

2006

Mach Learn

View full text Add to dashboard Cite

Most game programs have a large number of parameters that are crucial for their performance. While tuning these parameters by hand is rather difficult, efficient and easy to use generic automatic parameter optimisation algorithms are known only for special problems such as the adjustment of the parameters of an evaluation function. The SPSA algorithm (Simultaneous Perturbation Stochastic Approximation) is a generic stochastic gradient method for optimising an objective function when an analytic expression of the gradient is not available, a frequent case in game programs. Further, SPSA in its canonical form is very easy to implement. As such, it is an attractive choice for parameter optimisation in game programs, both due to its generality and simplicity. The goal of this paper is twofold: (i) to introduce SPSA for the game programming community by putting it into a game-programming perspective, and (ii) to propose and discuss several methods that can be used to enhance the performance of SPSA. These methods include using common random numbers and antithetic variables, a combination of SPSA with RPROP, and the reuse of samples of previous performance evaluations. SPSA with the proposed enhancements was tested in some large-scale experiments on tuning the parameters of an opponent model, a policy and an evaluation function in our poker program, MCRAISE. Whilst SPSA with no enhancements failed to make progress using the allocated resources, SPSA with the enhancements proved to be competitive with other methods, including TD-learning; increasing the average payoff per game by as large as 0.19 times the size of the amount of the small bet. From the experimental study, we conclude that the use of an appropriately enhanced variant of SPSA for the optimisation of game program parameters is a viable approach, especially if no good alternative exist for the types of parameters considered.

show abstract

Optimization over discrete sets via SPSA

Cited by 19 publications

References 10 publications

On optimization algorithms for the reservoir oil well placement problem

On optimization algorithms for the reservoir oil well placement problem

Stochastic approximation on discrete sets using simultaneous difference approximations

Universal parameter optimisation in games based on SPSA

Contact Info

Product

Resources

About