Model-based real-time control of a magnetic manipulator system

Damsteeg, J.; Nageshrao, Subramanya; Babuška, Robert

doi:10.1109/cdc.2017.8264140

Cited by 5 publications

(7 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, magnetic manipulation is not constrained by the robot arm morphology, and it is less constrained by obstacles. A schematic of a magnetic manipulation setup [32] with two coils is shown in Figure 9.…”

Section: Magnetic Manipulationmentioning

confidence: 99%

Symbolic Regression Methods for Reinforcement Learning

Kubalı́k

Derner²,

Žegklitz

et al. 2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

Reinforcement learning algorithms can be used to optimally solve dynamic decision-making and control problems. With continuous-valued state and input variables, reinforcement learning algorithms must rely on function approximators to represent the value function and policy mappings. Commonly used numerical approximators, such as neural networks or basis function expansions, have two main drawbacks: they are black-box models offering no insight in the mappings learned, and they require significant trial and error tuning of their meta-parameters. In this paper, we propose a new approach to constructing smooth value functions in the form of analytic expressions by means of symbolic regression. We introduce three off-line methods for finding value functions based on a state transition model: symbolic value iteration, symbolic policy iteration, and a direct solution of the Bellman equation. The methods are illustrated on four nonlinear control problems: velocity control under friction, one-link and two-link pendulum swing-up, and magnetic manipulation. The results show that the value functions not only yield well-performing policies, but also are compact, mathematically tractable and easy to plug into other algorithms. This makes them potentially suitable for further analysis of the closed-loop system. A comparison with an alternative approach using neural networks shows that our method outperforms the neural network-based one.

show abstract

Section: Magnetic Manipulationmentioning

confidence: 99%

Symbolic Regression Methods for Reinforcement Learning

Kubalı́k

Derner²,

Žegklitz

et al. 2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…We use data measured on a real system and an empirical modelf (x) = −ic 1 x/(x 2 + c 2 ) 3 proposed in the literature [15] as the reference model. Parameters c 1 and c 2 were found empirically for the given system and this model was used to design well-performing nonlinear controllers in [1,8]. For this example, we define y = f (x).…”

Section: Methodsmentioning

confidence: 99%

“…As mentioned above, the coefficients β weighting the model's features in (8) are fitted using the least squares method. In particular, the coefficients are found such that the final model minimizes the sum of squared residuals over the training data set D. Thus, just the C t objective is considered at that moment, the C c is ignored.…”

Section: Alternative Way To Estimate Coefficients βmentioning

confidence: 99%

See 1 more Smart Citation

Symbolic regression driven by training data and prior knowledge

Kubalı́k

Derner

Babuška

2020

Proceedings of the 2020 Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

In symbolic regression, the search for analytic models is typically driven purely by the prediction error observed on the training data samples. However, when the data samples do not sufficiently cover the input space, the prediction error does not provide sufficient guidance toward desired models. Standard symbolic regression techniques then yield models that are partially incorrect, for instance, in terms of their steady-state characteristics or local behavior. If these properties were considered already during the search process, more accurate and relevant models could be produced. We propose a multi-objective symbolic regression approach that is driven by both the training data and the prior knowledge of the properties the desired model should manifest. The properties given in the form of formal constraints are internally represented by a set of discrete data samples on which candidate models are exactly checked. The proposed approach was experimentally evaluated on three test problems with results clearly demonstrating its capability to evolve realistic models that fit the training data well while complying with the prior knowledge of the desired model characteristics at the same time. It outperforms standard symbolic regression by several orders of magnitude in terms of the mean squared deviation from a reference model. CCS CONCEPTS • Computing methodologies → Genetic algorithms; Model development and analysis; • Theory of computation → Design and analysis of algorithms; Genetic programming; • Applied computing → Multi-criterion optimization and decision-making;

show abstract

“…e steel ball can be positioned by properly controlling the current and thereby the magnetic force of the coils. e magnetic force a coil exerts on the ball is highly dependent on the distance of the ball from the coil's centre, which introduces a significant nonmonotonic nonlinearity [22][23][24].…”

Section: Magnetic Manipulatormentioning

confidence: 99%

Control of Magnetic Manipulator Using Reinforcement Learning Based on Incrementally Adapted Local Linear Models

et al. 2021

View full text Add to dashboard Cite

Reinforcement learning (RL) agents can learn to control a nonlinear system without using a model of the system. However, having a model brings benefits, mainly in terms of a reduced number of unsuccessful trials before achieving acceptable control performance. Several modelling approaches have been used in the RL domain, such as neural networks, local linear regression, or Gaussian processes. In this article, we focus on techniques that have not been used much so far: symbolic regression (SR), based on genetic programming and local modelling. Using measured data, symbolic regression yields a nonlinear, continuous-time analytic model. We benchmark two state-of-the-art methods, SNGP (single-node genetic programming) and MGGP (multigene genetic programming), against a standard incremental local regression method called RFWR (receptive field weighted regression). We have introduced modifications to the RFWR algorithm to better suit the low-dimensional continuous-time systems we are mostly dealing with. The benchmark is a nonlinear, dynamic magnetic manipulation system. The results show that using the RL framework and a suitable approximation method, it is possible to design a stable controller of such a complex system without the necessity of any haphazard learning. While all of the approximation methods were successful, MGGP achieved the best results at the cost of higher computational complexity. Index Terms–AI-based methods, local linear regression, nonlinear systems, magnetic manipulation, model learning for control, optimal control, reinforcement learning, symbolic regression.

show abstract

Model-based real-time control of a magnetic manipulator system

Cited by 5 publications

References 13 publications

Symbolic Regression Methods for Reinforcement Learning

Symbolic Regression Methods for Reinforcement Learning

Symbolic regression driven by training data and prior knowledge

Control of Magnetic Manipulator Using Reinforcement Learning Based on Incrementally Adapted Local Linear Models

Contact Info

Product

Resources

About