An Application of Control- Tutored Reinforcement Learning to the Herding Problem

Lellis, Francesco De; Auletta, Fabrizia; Russo, Giovanni; DeLellis, Pietro; Bernardo, Mario di

doi:10.1109/cnna49188.2021.9610789

Cited by 3 publications

(2 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The computational aspect (both for learning and control) is a challenge in its own and different methods need to be rigorously benchmarked along this dimension. Approximation results exist that allow to reduce the computational burden and we highlight a perhaps less explored direction to integrate data-driven and model-based technologies so that they tutor each other [177,180,181]. Finally, we believe that the ultimate challenge will be to deploy the algorithms underpinned by the methods presented here in applications where reliable models are intrinsically probabilistic and/or hard/expensive to find.…”

Section: Concluding Discussionmentioning

confidence: 97%

Probabilistic design of optimal sequential decision-making algorithms in learning and control

Emiland¹,

Russo²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

This survey is focused on certain sequential decision-making problems that involve optimizing over probability functions. We discuss the relevance of these problems for learning and control. The survey is organized around a framework that combines a problem formulation and a set of resolution methods. The formulation consists of an infinite-dimensional optimization problem. The methods come from approaches to search optimal solutions in the space of probability functions. Through the lenses of this overarching framework we revisit popular learning and control algorithms, showing that these naturally arise from suitable variations on the formulation mixed with different resolution methods. A running example, for which we make the code available, complements the survey. Finally, a number of challenges arising from the survey are also outlined.

show abstract

Section: Concluding Discussionmentioning

confidence: 97%

Probabilistic design of optimal sequential decision-making algorithms in learning and control

Emiland¹,

Russo²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Following [26] and [27], we employ an RL solution to automatically identify an acceptable policy for a given initial condition x0 and to do so without the need of knowing the dynamics f . Namely, let r : X × X × U → R be a reward function, so that r (x ′ , x, u) is the reward obtained by the agent when taking action u in state x and arriving at the new state x ′ at the next time instant.…”

Section: Using Reinforcement Learning To Find Acceptable Control Poli...mentioning

confidence: 99%

Guaranteeing Control Requirements via Reward Shaping in Reinforcement Learning

De Lellis,

Coraggio,

Russo

et al. 2024

IEEE Trans. Contr. Syst. Technol.

View full text Add to dashboard Cite

In addressing control problems such as regulation and tracking through reinforcement learning (RL), it is often required to guarantee that the acquired policy meets essential performance and stability criteria such as a desired settling time and steady-state error before deployment. Motivated by this, we present a set of results and a systematic rewardshaping procedure that: 1) ensures the optimal policy generates trajectories that align with specified control requirements and 2) allows to assess whether any given policy satisfies them. We validate our approach through comprehensive numerical experiments conducted in two representative environments from OpenAI Gym: the Pendulum swing-up problem and the Lunar Lander. Utilizing both tabular and deep RL methods, our experiments consistently affirm the efficacy of our proposed framework, highlighting its effectiveness in ensuring policy adherence to the prescribed control requirements.

show abstract