Deep reinforcement learning for scheduling in large-scale networked control systems

Redder, Adrian; Ramaswamy, Arunselvan; Quevedo, Daniel E.

doi:10.1016/j.ifacol.2019.12.177

Cited by 20 publications

(17 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then using Bernstein's inequality (32), we have that the probability of correct answer in (35) is larger than 1−δ if…”

Section: Sharper Sample Complexity Results For Low-variance Channelsmentioning

confidence: 99%

“…In contrast our work is focused on collecting data and learning unknown channel models instead of system dynamics. In the context of networked control systems very recent works from the last two years are proposing data-based approaches including deep learning for allocating resources and scheduling [12,15,25,32,40] as well as for controller design [6,35]. A related but broader topic is also the identification of switched systems [29] where the matrix dynamics are also unknown.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Sample Complexity of Networked Control Systems Over Unknown Channels

Gatsis

Pappas

2018

2018 IEEE Conference on Decision and Control (CDC)

View full text Add to dashboard Cite

Recent control trends are increasingly relying on communication networks and wireless channels to close the loop for Internetof-Things applications. Traditionally these approaches are model-based, i.e., assuming a network or channel model they are focused on stability analysis and appropriate controller designs. However the availability of such wireless channel modeling is fundamentally challenging in practice as channels are typically unknown a priori and only available through data samples. In this work we aim to develop algorithms that rely on channel sample data to determine the stability and performance of networked control tasks. In this regard our work is the first to characterize the amount of channel modeling that is required to answer such a question. Specifically we examine how many channel data samples are required in order to answer with high confidence whether a given networked control system is stable or not. This analysis is based on the notion of sample complexity from the learning literature and is facilitated by concentration inequalities. Moreover we establish a direct relation between the sample complexity and the networked system stability margin, i.e., the underlying packet success rate of the channel and the spectral radius of the dynamics of the control system. This illustrates that it becomes impractical to verify stability under a large range of plant and channel configurations. We validate our theoretical results in numerical simulations.

show abstract

“…Then using Bernstein's inequality (32), we have that the probability of correct answer in (35) is larger than 1−δ if…”

Section: Sharper Sample Complexity Results For Low-variance Channelsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Sample Complexity of Networked Control Systems Over Unknown Channels

Gatsis

Pappas

2018

2018 IEEE Conference on Decision and Control (CDC)

View full text Add to dashboard Cite

show abstract

“…The network model also needs to be learned in real time to achieve the best performance for ETC. A preliminary attempt has been carried out in [53], in which DRL is used to learn the communication network dynamics instead of the model of the plant as shown in Fig. 6.…”

Section: Joint Learning Of System and Network Modelsmentioning

confidence: 99%

Machine Learning in Event-Triggered Control: Recent Advances and Open Issues

Sedghi¹,

Ijaz²,

Noor-A-Rahim³

et al. 2020

Preprint

View full text Add to dashboard Cite

Network Control Systems (NCSs) have attracted much interest over the past decade as part of a move towards more decentralised control applications and the rise of cyberphysical system applications. Many practical NCSs face the challenges of limited communication bandwidth resources, reliability and lack of knowledge of network dynamics, particularly when wireless networks are involved. Machine learning (ML) combined with event-triggered control (ETC) has the potential to ease some of these challenges. For example, ML can be used to overcome the problem of a lack of network models by learning system behaviour or adapt to dynamically changing models by continually learning model dynamics. ETC can help to conserve bandwidth resources by communicating only when needed or when resources are available. Here, we present a review of the literature on work that combines ML and ETC. The literature on supervised, semi-supervised, unsupervised and reinforcement learning based approaches such as deep reinforcement learning and statistical learning in combination with ETC is explored. Furthermore, the difference between the application of these learning algorithms on model-based and model-free systems are discussed. Following the analysis of the literature, we highlight open research questions and challenges related to ML-based ETC and propose approaches to possible solutions to these challenges.

show abstract

“…In [16], [17], the fusion center feeds back estimation quality that is coupled with local sensor quality to jointly optimize cross-layer performance. Combined with reinforcement learning, several works designed optimal strategies for more realistic systems with feedback [21]- [23].…”

Section: A Related Literaturementioning

confidence: 99%

Distributed remote estimation over the collision channel with and without local communication

Zhang

Vasconcelos

Cui

et al. 2020

Preprint

View full text Add to dashboard Cite

The emergence of the Internet-of-Things and cyber-physical systems necessitates the coordination of access to limited communication resources in an autonomous and distributed fashion. Herein, the optimal design of a wireless sensing system with n sensors communicating with a fusion center via a collision channel of limited capacity k (k < n) is considered. In particular, it is shown that the problem of minimizing the mean-squared error subject to a threshold-based strategy at the transmitters is quasi-convex. As such, low complexity, numerical optimization methods can be applied. When coordination among sensors is not possible, the performance of the optimal threshold strategy is close to that of a centralized lower bound. The loss due to decentralization is thoroughly characterized.Local communication among sensors (using a sparsely connected graph), enables the on-line learning of unknown parameters of the statistical model. These learned parameters are employed to compute the desired thresholds locally and autonomously. Consensus-based strategies are investigated and analyzed for parameter estimation. One strategy approaches the performance of the decentralized approach with fast convergence and a second strategy approaches the performance of the centralized approach, albeit with slower convergence. A hybrid scheme that combines the best of both approaches is proposed offering a fast convergence and excellent convergent performance.

show abstract

Deep reinforcement learning for scheduling in large-scale networked control systems

Cited by 20 publications

References 20 publications

Sample Complexity of Networked Control Systems Over Unknown Channels

Sample Complexity of Networked Control Systems Over Unknown Channels

Machine Learning in Event-Triggered Control: Recent Advances and Open Issues

Distributed remote estimation over the collision channel with and without local communication

Contact Info

Product

Resources

About