On a probabilistic approach to synthesize control policies from example datasets

Gagliardi, Davide; Russo, Giovanni

doi:10.1016/j.automatica.2021.110121

Cited by 13 publications

(15 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Constraint (2e) can also capture bound constraints on U k of the form P(U k ∈ Ū) ≥ 1−γ k , where Ū ⊆ U and 0 ≤ γ k ≤ 1. See [9] for a detailed discussion, where it is also shown that these constraints are convex in the decision variables and hence can be solved without resorting to bounding approximations. The fulfillment of constraint (2f) instead guarantees that the probability that the state is outside some (e.g.…”

Section: Statement Of the Decision-making Problemmentioning

confidence: 99%

See 1 more Smart Citation

Probabilistic design of optimal sequential decision-making algorithms in learning and control

Emiland¹,

Russo²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

This survey is focused on certain sequential decision-making problems that involve optimizing over probability functions. We discuss the relevance of these problems for learning and control. The survey is organized around a framework that combines a problem formulation and a set of resolution methods. The formulation consists of an infinite-dimensional optimization problem. The methods come from approaches to search optimal solutions in the space of probability functions. Through the lenses of this overarching framework we revisit popular learning and control algorithms, showing that these naturally arise from suitable variations on the formulation mixed with different resolution methods. A running example, for which we make the code available, complements the survey. Finally, a number of challenges arising from the survey are also outlined.

show abstract

Section: Statement Of the Decision-making Problemmentioning

confidence: 99%

“…Finally, in e.g. [9,98] the reference pf is instead estimated from example data. Let Γ 0:T be the example dataset.…”

Section: Probabilistic Design Of Policies Through the Lenses Of Problemmentioning

confidence: 99%

Probabilistic design of optimal sequential decision-making algorithms in learning and control

Emiland¹,

Russo²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Remark 1: (1) is a probabilistic description, a black box type model [17] that can always be obtained from the data.…”

Section: Formulation Of the Control Problemmentioning

confidence: 99%

“…Inspired by [17], we tackle Problem 1 by splitting it in subproblems that can be solved recursively. Before introducing our main result we give two technical lemmas Lemma 2: let γ k : X → R be integrable under the measure given by π k|k−1 and consider min…”

Section: Tackling the Crowdsourcing Problemmentioning

confidence: 99%

On the crowdsourcing of behaviors for autonomous agents

Russo

2020

Preprint

Self Cite

View full text Add to dashboard Cite

This paper is concerned with the problem of designing, from data, agents that are able to craft their behavior from a set of contributors in order to fulfill some agent-specific task. This is not necessarily known to the contributors. After formalizing this crowdsourcing process as a control problem, we present a result to synthesize behaviors from the information made available by the contributors. The result is turned into an algorithm and the theoretical findings are complemented via an example.

show abstract

“…Results on data-driven control include [5]- [7], which take a behavioral systems perspective, [8], that presents an approach to compute minimumenergy controls for linear systems, [9]- [11] which introduce algorithms inspired from MPC, [12], where a data-driven design approach is devised for stabilizable Lipschitz systems. We also recall here [13] which, by leveraging an approach that can be traced back to [14], seeks to synthesize control policies from demonstration datasets for systems with actuation constraints and the recent [15], which discusses some of the key challenges of extracting control relevant information from large amounts of data. Recently, it has been shown that data-driven control can benefit from the use of properly patched data from multiple sources [16].…”

Section: Introductionmentioning

confidence: 99%

On the design of autonomous agents from multiple data sources

Garrabé,

Russo

2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper is concerned with the problem of designing agents able to dynamically select information from multiple data sources in order to tackle tasks that involve tracking a target behavior while optimizing a reward. We formulate this problem as a data-driven optimal control problem with integer decision variables and give an explicit expression for its solution. The solution determines how (and when) the data from the sources should be used by the agent. We also formalize a notion of agent's regret and, by relaxing the problem, give a regret upper bound. Simulations complement the results.

show abstract

On a probabilistic approach to synthesize control policies from example datasets

Cited by 13 publications

References 54 publications

Probabilistic design of optimal sequential decision-making algorithms in learning and control

Probabilistic design of optimal sequential decision-making algorithms in learning and control

On the crowdsourcing of behaviors for autonomous agents

On the design of autonomous agents from multiple data sources

Contact Info

Product

Resources

About