Abstract-In this paper, we present a quantitative, trajectory-based method for calibrating stochastic motion models of water-floating robots. Our calibration method is based on the Correlated Random Walk (CRW) model, and consists in minimizing the Kolmogorov-Smirnov (KS) distance between the step length and step angle distributions of real and simulated trajectories generated by the robots. First, we validate this method by calibrating a physics-based motion model of a single 3-cm-sized robot floating at a water/air interface under fluidic agitation. Second, we extend the focus of our work to multi-robot systems by performing a sensitivity analysis of our stochastic motion model in the context of SelfAssembly (SA). In particular, we compare in simulation the effect of perturbing the calibrated parameters on the predicted distributions of self-assembled structures. More generally, we show that the SA of water-floating robots is very sensitive to even small variations of the underlying physical parameters, thus requiring real-time tracking of its dynamics.

Abstract-Population-based learning techniques have been proven to be effective in dealing with noise in numerical benchmark functions and are thus promising tools for the high-dimensional optimization of controllers for multiple robots with limited sensing capabilities, which have inherently noisy performance evaluations. In this article, we apply a statistical technique called Optimal Computing Budget Allocation to improve the performance of Particle Swarm Optimization in the presence of noise for a multi-robot obstacle avoidance benchmark task. We present a new distributed PSO OCBA algorithm suitable for resource-constrained mobile robots due to its low requirements in terms of memory and limited local communication. Our results from simulation show that PSO OCBA outperforms other techniques for dealing with noise, achieving a more consistent progress and a better estimate of the ground-truth performance of candidate solutions. We then validate our simulations with real robot experiments where we compare the controller learned with our proposed algorithm to a potential field controller for obstacle avoidance in a cluttered environment. We show that they both achieve a high performance through different avoidance behaviors.

The ability to move in complex environments is a fundamental requirement for robots to be a part of our daily lives. While in simple environments it is usually straightforward for human designers to foresee the different conditions a robot will be exposed to, for more complex environments the human design of high-performing controllers becomes a challenging task, especially when the on-board resources of the robots are limited. In this article, we use a distributed implementation of Particle Swarm Optimization to design robotic controllers that are able to navigate around obstacles of different shape and size. We analyze how the behavior and performance of the controllers differ based on the environment where learning takes place, showing that different arenas lead to different avoidance behaviors. We also test the best controllers in environments not encountered during learning, both in simulation and with real robots, and show that no single learning environment is able to generate a behavior general and robust enough to succeed in all testing environments. IntroductionIn simple environments, it is usually straightforward for human designers to anticipate the different conditions a robot will be exposed to. Thus, robotic controllers can be designed manually by simplifying the number of parameters or inputs used. However, for more complex environments, the human design of high-performing controllers becomes a challenging task. This is especially true if the on-board resources of the robot are limited, as humans may not be aware of how to exploit limited sensing capabilities.Machine-learning techniques are an alternative to human design that can automatically synthesize robotic controllers in large search spaces, coping with discontinuities and nonlinearities, and find innovative solutions not foreseen by human designers. In particular, evaluative, on-board techniques can develop specific behaviors adapted to the environment where the robots are deployed.The purpose of this paper is twofold. First, to verify whether different behaviors arise as a function of the learning environment in the adaptation of multi-robot obstacle avoidance. Secondly, to test how the learned behaviors perform in environments not encountered during learning, that is, to evaluate how general are the solutions found in the learning process. The adaptation technique used is Particle Swarm Optimization (PSO) (Kennedy and Eberhart, 1995), which allows a distributed implementation in each robot, speeding up the adaptation process and adding robustness to failure of individual robots.The remainder of this article is organized as follows. Section Background introduces some related work on PSO, and on the influence of the environment in robotic adaptation. In the Hypotheses and Methods section we propose two hypotheses that motivate our research and describe the experimental methodology used to test them. Section Results and Discussion presents the experimental results obtained and discusses the validity of the proposed hypotheses. Final...

Abstract-The design of high-performing robotic controllers constitutes an example of expensive optimization in uncertain environments due to the often large parameter space and noisy performance metrics. There are several evaluative techniques that can be employed for on-line controller design. Adequate benchmarks help in the choice of the right algorithm in terms of final performance and evaluation time. In this paper, we use multi-robot obstacle avoidance as a benchmark to compare two different evaluative learning techniques: Particle Swarm Optimization and Q-learning. For Q-learning, we implement two different approaches: one with discrete states and discrete actions, and another one with discrete actions but a continuous state space. We show that continuous PSO has the highest fitness overall, and Q-learning with continuous states performs significantly better than Q-learning with discrete states. We also show that in the single robot case, PSO and Q-learning with discrete states require a similar amount of total learning time to converge, while the time required with Q-learning with continuous states is significantly larger. In the multi-robot case, both Q-learning approaches require a similar amount of time as in the single robot case, but the time required by PSO can be significantly reduced due to the distributed nature of the algorithm.

No abstract