Computing Budget Allocation for Efficient Ranking and Selection of Variances With Applicationto Target Tracking Algorithms

Abstract

This paper addresses the problem of ranking and selection for stochastic processes, such as target tracking algorithms, where variance is the performance metric. Comparison of different tracking algorithms or parameter sets within one algorithm relies on time-consuming and computationally demanding simulations. We present a method to minimize simulation time, yet to achieve a desirable confidence of the obtained results by applying ordinal optimization and computing budget allocation ideas and techniques, while taking into account statistical properties of the variance. The developed method is applied to a general tracking problem of sensors tracking targets using a sequential multi-sensor data fusion tracking algorithm. The optimization consists of finding the order of processing sensor information that results in the smallest variance of the position error. Results that we obtained with high confidence levels and in reduced simulation times confirm the findings from our previous research (where we considered only two sensors) that processing the best available sensor the last performs the best, on average. The presented method can be applied to any ranking and selection problem where variance is the performance metric.

An improved dynamic structure-based neural networks determination approaches to simulation optimization problems

“…In the simulation literature, ranking and selection procedures have often been recommended for solving the discrete simulation optimization problem [19,20]. Ranking and selection procedures can also be used in the following ways in relevant to simulation optimization [21,22]:…”

Efficient Dynamic Simulation Allocation in Ordinal Optimization

“…Chen et al [3] formalize this idea and develop a new approach called optimal computing budget allocation (OCBA), which can attain a speedup of an additional order of magnitude faster than base algorithms that can be proven to achieve exponential convergence for ordinal optimization. Similar budget allocation ideas have been extended to various applications (e.g., [4], [11], and [12]). Other simulation budget allocation schemes developed from a different perspective include the expected value of information procedure (VIP [5], [6]) and the indifference zone procedure (IZ [9]).…”

Stochastic systems simulation optimization

“…[24][25][26] -maximizes an approximation of the probability of correct selection P {CS}, where correct selection (CS) indicates choosing the alternative with minimum mean, leading to an efficient allocation algorithm that includes both means and variances. Extensions of the OCBA approach consider correlated sampling [27]; non-normal distributions [28,29]; using a regression model [30]; multiple objective functions [31][32][33]; using expected opportunity cost instead of the probability of correct selection [34,35]; minimizing variance instead of maximizing the probability of correct selection [36]; transient means ranking and selection [37,38]; selecting an optimal subset of topm solutions rather than the single best solution [39]; selecting the best design with consideration of stochastic constraints or design complexity [40,41].…”