Microassembly path planning using reinforcement learning for improving positioning accuracy of a 1 cm3 omni-directional mobile microrobot

Li, Jianghao; Li, Zhenbo; Chen, Jiapin

doi:10.1007/s10489-009-0191-x

Cited by 19 publications

(11 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We will justify this with the following experimental results that are based on the benchmark environments shown in Table 2. Table 3 shows the comparison between the t 0 calculated from (18) and the average number of iterations needed for the LA to converge, in practice. As the CPA and ACPA are well-established algorithms, we know that numerous experiments have been conducted to confirm their validity, and so we merely use the figures from [30] to record the practical results.…”

Section: Remarkmentioning

confidence: 99%

See 1 more Smart Citation

A formal proof of the ε-optimality of absorbing continuous pursuit algorithms using the theory of regular functions

et al. 2014

View full text Add to dashboard Cite

The most difficult part in the design and analysis of Learning Automata (LA) consists of the formal proofs of their convergence accuracies. The mathematical techniques used for the different families (Fixed Structure, Variable Structure, Discretized etc.) are quite distinct. Among the families of LA, Estimator Algorithms (EAs) are certainly the fastest, and within this family, the set of Pursuit algorithms have been considered to be the pioneering schemes. Informally, if the environment is stationary, their ε-optimality is defined as their ability to converge to the optimal action with an arbitrarily large probability, if the learning parameter is sufficiently small/large. The existing proofs of all the reported EAs follow the same fundamental principles, and to clarify this, in the interest of simplicity, we shall concentrate on the family of Pursuit algorithms. Recently, it has been reported Ryan and Omkar (J Appl Probab 49(3): [795][796][797][798][799][800][801][802][803][804][805] 2012) that the previous proofs for ε-optimality of all the reported EAs have a common flaw.The flaw lies in the condition which apparently supports the so-called "monotonicity" property of the probability of selecting the optimal action, which states that after some time instant t 0 , the reward probability estimates will be ordered correctly forever. The authors of the various proofs have rather offered a proof for the fact that the reward probability estimates are ordered correctly at a single point of time after t 0 , which, in turn, does not guarantee the ordering forever, rendering the previous proofs incorrect. While in Ryan and Omkar (J Appl Probab 49(3):795-805, 2012), a rectified proof was presented to prove the ε-optimality of the Continuous Pursuit Algorithm (CPA), which was the pioneering EA, in this paper, a new proof is provided for the Absorbing CPA (ACPA), i.e., an algorithm which follows the CPA paradigm but which artificially has absorbing states whenever any action probability is arbitrarily close to unity. Unlike the previous flawed proofs, instead of examining the monotonicity property of the action probabilities, it rather examines their submartingale property, and then, unlike the traditional approach, invokes the theory of Regular functions to prove that the probability of converging to the optimal action can be made arbitrarily close to unity. We believe that the proof is both unique and pioneering, and adds insights into the convergence of different EAs. It can also form the basis for formally demonstrating the ε-optimality of other Estimator algorithms which are artificially rendered absorbing.

show abstract

Section: Remarkmentioning

confidence: 99%

“…LA have also been used in natural language processing, string taxonomy [12], graph patitioning [13], map learning [14], service selection in stochastic environments [15], numerical optimization [16], web crawling [17], microassembly path planning [18], multiagent learning [19], and in batch sequencing and sizing in just-in-time manufacturing systems [20].…”

mentioning

confidence: 99%

A formal proof of the ε-optimality of absorbing continuous pursuit algorithms using the theory of regular functions

et al. 2014

View full text Add to dashboard Cite

show abstract

“…They have been used in game playing [4][5][6][7][8][9][10], parameter optimization [11,12], channel selection in cognitive radio networks [13], assigning capacities in prioritized networks [14], solving knapsack problems [15], optimizing the web polling problem [16,17], stochastically optimally allocating limited resources [15,18,19], service selection in stochastic environments [20], numerical optimization [21], web crawling [22], microassembly path planning [23], multiagent learning [24], and in batch sequencing and sizing in just-intime manufacturing systems [25]. An asynchronous actionreward learning has been used for nonstationary serial supply chain inventory control [26].…”

Section: Learning Automata: Concept and Their Applicationsmentioning

confidence: 99%

On incorporating the paradigms of discretization and Bayesian estimation to create a new family of pursuit learning automata

2013

View full text Add to dashboard Cite

There are currently two fundamental paradigms that have been used to enhance the convergence speed of Learning Automata (LA). The first involves the concept of utilizing the estimates of the reward probabilities, while the second involves discretizing the probability space in which the LA operates. This paper demonstrates how both of these can be simultaneously utilized, and in particular, by using the family of Bayesian estimates that have been proven to have distinct advantages over their maximum likelihood counterparts. The success of LA-based estimator algorithms over the classical, Linear Reward-Inaction (L RI )-like schemes, can be explained by their ability to pursue the actions with the highest reward probability estimates. Without access to reward probability estimates, it makes sense for schemes like the L RI to first make large exploring steps, and then to gradually turn exploration into exploitation by making progressively smaller learning steps. However, this behavior becomes counter-intuitive when pur- suing actions based on their estimated reward probabilities. Learning should then ideally proceed in progressively larger steps, as the reward probability estimates turn more accurate. This paper introduces a new estimator algorithm, the Discretized Bayesian Pursuit Algorithm (DBPA), that achieves this by incorporating both the above paradigms. The DBPA is implemented by linearly discretizing the action probability space of the Bayesian Pursuit Algorithm (BPA) (Zhang et al. in IEA-AIE 2011, Springer, New York, pp. 608-620, 2011. The key innovation of this paper is that the linear discrete updating rules mitigate the counterintuitive behavior of the corresponding linear continuous updating rules, by augmenting them with the reward probability estimates. Extensive experimental results show the superiority of DBPA over previous estimator algorithms. Indeed, the DBPA is probably the fastest reported LA to date. Apart from the rigorous experimental demonstration of the strength of the DBPA, the paper also briefly records the proofs of why the BPA and the DBPA are -optimal in stationary environments.

show abstract

“…The RL method is also frequently applied in various engineering tasks. It is used in nonstationary serial supply chain inventory control [18], adaptive control of nonlinear objects [43], adjusting robot behavior for autonomous navigation system [26] or path planning for improving positioning accuracy of a mobile microrobot [22]. There are also studies which propose the use of RL in non-technical domains, e.g.…”

Section: Introductionmentioning

confidence: 99%

Probabilistic neural network training procedure based on Q(0)-learning algorithm in medical data classification

Kusy

Zajdel

2014

Appl Intell

View full text Add to dashboard Cite

In this article, an iterative procedure is proposed for the training process of the probabilistic neural network (PNN). In each stage of this procedure, the Q(0)-learning algorithm is utilized for the adaptation of PNN smoothing parameter (σ). Four classes of PNN models are regarded in this study. In the case of the first, simplest model, the smoothing parameter takes the form of a scalar; for the second model, σ is a vector whose elements are computed with respect to the class index; the third considered model has the smoothing parameter vector for which all components are determined depending on each input attribute; finally, the last and the most complex of the analyzed networks, uses the matrix of smoothing parameters where each element is dependent on both class and input feature index. The main idea of the presented approach is based on the appropriate update of the smoothing parameter values according to the Q(0)-learning algorithm. The proposed procedure is verified on six repository data sets. The prediction ability of the algorithm is assessed by computing the test accuracy on 10 %, 20 %, 30 %, and 40 % of examples drawn randomly from each input data set. The results are compared with the test accuracy obtained by PNN trained using the conjugate gradient procedure, support vector machine algorithm, gene expression programming classifier, k-Means method, multilayer perceptron, radial basis function neural network and learning vector quantization neural network. It is shown M. Kusy () • R. Zajdel Faculty of Electrical and Computer Engineering,

show abstract

Microassembly path planning using reinforcement learning for improving positioning accuracy of a 1 cm3 omni-directional mobile microrobot

Cited by 19 publications

References 27 publications

A formal proof of the ε-optimality of absorbing continuous pursuit algorithms using the theory of regular functions

A formal proof of the ε-optimality of absorbing continuous pursuit algorithms using the theory of regular functions

On incorporating the paradigms of discretization and Bayesian estimation to create a new family of pursuit learning automata

Probabilistic neural network training procedure based on Q(0)-learning algorithm in medical data classification

Contact Info

Product

Resources

About