DeepCAS: A Deep Reinforcement Learning Algorithm for Control-Aware Scheduling

Abstract

We consider networked control systems consisting of multiple independent controlled subsystems, operating over a shared communication network. Such systems are ubiquitous in cyber-physical systems, Internet of Things, and large-scale industrial systems. In many large-scale settings, the size of the communication network is smaller than the size of the system. In consequence, scheduling issues arise. The main contribution of this paper is to develop a deep reinforcement learning-based control-aware scheduling (DEEPCAS) algorithm to tackle these issues. We use the following (optimal) design strategy: First, we synthesize an optimal controller for each subsystem; next, we design a learning algorithm that adapts to the chosen subsystems (plants) and controllers. As a consequence of this adaptation, our algorithm finds a schedule that minimizes the control loss. We present empirical results to show that DEEPCAS finds schedules with better performance than periodic ones.

Deep reinforcement learning for wireless sensor scheduling in cyber–physical systems

“…Remark III.4. A scheduling problem for control of multiple processes using a deep reinforcement learning approach has also been recently studied by us in [29], but without consideration of packet drops. The setup in [29] also requires extra overhead in the transmission of error information (between the state estimates at the sensor and controller) from the sensors to the scheduler at every time step, which could be considerable.…”

Predictive Triggering for Distributed Control of Resource Constrained Multi-agent Systems

“…Sending a probability instead of state-information allows us to consider a low dimensional and system independent priority measure, resulting in reduced bandwidth consumption, as we will show in simulation studies and real experiments. Recently, Demirel et al (2018) proposed DeepCAS, a deep reinforcement learning (RL) algorithm for the control-aware scheduling of NCS. However powerful RL may be, difficulties arise in training and when implementation on embedded hardware is required.…”

Control-Aware Scheduling for Low Latency Wireless Systems with Deep Learning

“…Most DNN training techniques are not immediately usable here as the latency requirements of the system pose constraints in the optimization problem [16]. Recent advancements apply techniques from both reinforcement learning and deep learning for control-aware scheduling in simple systems [12]- [14] and traditional wireless systems with latency constraints [17], [18]. Learning-based scheduling policies are well suited for URLLC and control as the computational complexity at each scheduling round is very low and can furthermore be implemented model-free when system dynamics and communication models are unknown.…”