Learning Suboptimal Broadcasting Intervals in Multi-Agent Systems

Tolić, Domagoj; Palunko, Ivana

doi:10.1016/j.ifacol.2017.08.802

Cited by 4 publications

(25 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The sufficient conditions in [10, Section 4.1] typically entail the existence of a directed spanning tree (see [2, 22]), but one can envision less challenging/appealing MAS scenarios in which this directed spanning tree requirement is superfluous. In addition, since we are not interested exclusively in a single multi‐agent differential graphical game, [10, Section 4.1] imposes no requirements on graph edge weights found in [36]. Accordingly, subsets of our agents can effectively be involved in distinct multi‐agent differential graphical games.…”

Section: Methodsmentioning

confidence: 99%

“…Remark 5. The benefits of LSPI include implementation simplicity due to linear-in-parameter approximators, which in turn strips the algorithm down to a set of linear matrix equations (10). Accordingly, the principal numerical concern lies in the need for matrix inversion, which is a well-understood nuisance circumvented, among others, via the LU factorization [51].…”

Section: Learning Near-optimal Broadcasting Intervalsmentioning

confidence: 99%

“…Even though (18) can be solved iteratively (similarly to (17) since T h is a contraction), LSPI solves (18) at once by constructing the projected Bellman equation and employing the fact that it boils down to a system of linear equations (10). All details of the steps behind the transition from (6) and (18) to (10) are found in [32], [28,Chapter 3] and [24,Section 6.3].…”

Section: Near-optimality Boundsmentioning

confidence: 99%

“…In comparison with its precursor [10], the present article provides sufficient convergence and near-optimality guarantees and experimental verification of the online LSPI. In our framework, we first compute stabilizing upper bounds for intervals between two consecutive broadcasting instants of each individual agent (utilizing agents' approximate linear time invariant [LTI] dynamical models), thereby giving rise to asynchronous communication.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Learning near‐optimal broadcasting intervals in decentralized multi‐agent systems using online least‐square policy iteration

Palunko

Tolić

Prkacin

2021

IET Control Theory & Appl

Self Cite

View full text Add to dashboard Cite

Here, agents learn how often to exchange information with neighbours in cooperative multi-agent systems (MASs) such that their linear quadratic regulator (LQR)-like performance indices are minimized. The investigated LQR-like cost functions capture trade-offs between the energy consumption of each agent and MAS local control performance in the presence of exogenous disturbances, delayed and noisy data. Agent energy consumption is critical for prolonging the MAS mission and is composed of both control (e.g. acceleration, velocity) and communication efforts. Taking provably stabilizing upper bounds on broadcasting intervals as optimization constraints, an online off-policy model-free learning algorithm based on least square policy iteration (LSPI) to minimize the cost function of each agent is employed. Consequently, the obtained broadcasting intervals adapt to the most recent information (e.g. delayed and noisy agents' inputs and/or outputs) received from neighbours whilst provably stabilize the MAS. Chebyshev polynomials are utilized as the approximator in the LSPI whereas Kalman filtering handles sampled, corrupted, and delayed data. Subsequently, convergence and near-optimality of our LSPI scheme are inspected. The proposed methodology is verified experimentally using an inexpensive motion capture system and nano quadrotors.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Learning Near-optimal Broadcasting Intervalsmentioning

confidence: 99%

Section: Near-optimality Boundsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learning near‐optimal broadcasting intervals in decentralized multi‐agent systems using online least‐square policy iteration

Palunko

Tolić

Prkacin

2021

IET Control Theory & Appl

Self Cite

View full text Add to dashboard Cite

show abstract

“…These works typically consider offline algorithms with perfect model knowledge and state information so that closed-loop stability during learning is not a relevant concern. Somewhat reversed approaches, in which the authors start off with robust stability (e.g., Lp-stability) and employ RL towards (sub)optimality, are also of interest ( Anderson et al, 2007;Friedrich & Buss, 2017;Kretchmar et al, 2001;Toli ć & Palunko, 2017 ). Such approaches follow the control philosophy of trading off optimality for stability.…”

Section: Stability Considerations For Reinforcement Learningmentioning

confidence: 99%

Reinforcement learning for control: Performance, stability, and deep approximators

Buşoniu

Bruin

Tolić

et al. 2018

Annual Reviews in Control

Self Cite

345

141

View full text Add to dashboard Cite

Reinforcement learning (RL) offers powerful algorithms to search for optimal controllers of systems with nonlinear, possibly stochastic dynamics that are unknown or highly uncertain. This review mainly covers artificial-intelligence approaches to RL, from the viewpoint of the control engineer. We explain how approximate representations of the solution make RL feasible for problems with continuous states and control actions. Stability is a central concern in control, and we argue that while the control-theoretic RL subfield called adaptive dynamic programming is dedicated to it, stability of RL largely remains an open question. We also cover in detail the case where deep neural networks are used for approximation, leading to the field of deep RL, which has shown great success in recent years. With the control practitioner in mind, we outline opportunities and pitfalls of deep RL; and we close the survey with an outlook that -among other things -points out some avenues for bridging the gap between control and artificial-intelligence RL techniques.

show abstract

Least-squares policy iteration algorithms for robotics: Online, continuous, and automatic

Friedrich

Schreibauer

Buss

2019

Engineering Applications of Artificial Intelligence

View full text Add to dashboard Cite

Learning Suboptimal Broadcasting Intervals in Multi-Agent Systems

Cited by 4 publications

References 7 publications

Learning near‐optimal broadcasting intervals in decentralized multi‐agent systems using online least‐square policy iteration

Learning near‐optimal broadcasting intervals in decentralized multi‐agent systems using online least‐square policy iteration

Reinforcement learning for control: Performance, stability, and deep approximators

Least-squares policy iteration algorithms for robotics: Online, continuous, and automatic

Contact Info

Product

Resources

About