This paper studies how to schedule wireless transmissions from sensors to estimate the states of multiple remote, dynamic processes. Sensors make observations of each of the processes. Information from the different sensors have to be transmitted to a central gateway over a wireless network for monitoring purposes, where typically fewer wireless channels are available than there are processes to be monitored. Such estimation problems routinely occur in large-scale Cyber-Physical Systems, especially when the dynamic systems (processes) involved are geographically separated. For effective estimation at the gateway, the sensors need to be scheduled appropriately, i.e., at each time instant to decide which sensors have network access and which ones do not. To solve this scheduling problem, we formulate an associated Markov decision process (MDP). Further, we solve this MDP using a Deep Q-Network, a deep reinforcement learning algorithm that is at once scalable and model-free. We compare our scheduling algorithm to popular scheduling algorithms such as round-robin and reduced-waitingtime, among others. Our algorithm is shown to significantly outperform these algorithms for randomly generated example scenarios.arXiv:1809.05149v1 [cs.SY]

We consider networked control systems consisting of multiple independent controlled subsystems, operating over a shared communication network. Such systems are ubiquitous in cyber-physical systems, Internet of Things, and large-scale industrial systems. In many large-scale settings, the size of the communication network is smaller than the size of the system. In consequence, scheduling issues arise. The main contribution of this paper is to develop a deep reinforcement learning-based control-aware scheduling (DEEPCAS) algorithm to tackle these issues. We use the following (optimal) design strategy: First, we synthesize an optimal controller for each subsystem; next, we design a learning algorithm that adapts to the chosen subsystems (plants) and controllers. As a consequence of this adaptation, our algorithm finds a schedule that minimizes the control loss. We present empirical results to show that DEEPCAS finds schedules with better performance than periodic ones.

The rainbow connection number, rc(G), of a connected graph G is the minimum number of colours needed to colour its edges, so that every pair of its vertices is connected by at least one path in which no two edges are coloured the same. In this note we show that for every bridgeless graph G with radius r, rc(G) ≤ r(r + 2). We demonstrate that this bound is the best possible for rc(G) as a function of r, not just for bridgeless graphs, but also for graphs of any stronger connectivity. It may be noted that for a general 1-connected graph G, rc(G) can be arbitrarily larger than its radius (K 1,n for instance). We further show that for every bridgeless graph G with radius r and chordality (size of a largest induced cycle) k, rc(G) ≤ rk. Hitherto, the only reported upper bound on the rainbow connection number of bridgeless graphs is 4n/5 − 1, where n is order of the graph [1].It is known that computing rc(G) is NP-Hard [2]. Here, we present a (r + 3)-factor approximation algorithm which runs in O(nm) time and a (d + 3)-factor approximation algorithm which runs in O(dm) time to rainbow colour any connected graph G on n vertices, with m edges, diameter d and radius r.

In this paper the stability theorem of Borkar and Meyn is extended to include the case when the mean field is a set-valued map. Two different sets of sufficient conditions are presented that guarantee the 'stability and convergence' of stochastic recursive inclusions. Our work builds on the works of Benaïm, Hofbauer and Sorin as well as Borkar and Meyn. As a corollary to one of the main theorems, a natural generalization of the Borkar and Meyn Theorem follows. In addition, the original theorem of Borkar and Meyn is shown to hold under slightly relaxed assumptions. As an application to one of the main theorems we discuss a solution to the 'approximate drift problem'. Finally, we analyze the stochastic gradient algorithm with "constant error gradient estimators" as yet another application of our main result.

Rainbow connection number, rc(G), of a connected graph G is the minimum number of colors needed to color its edges so that every pair of vertices is connected by at least one path in which no two edges are colored the same (Note that the coloring need not be proper). In this paper we study the rainbow connection number with respect to three important graph product operations (namely cartesian product, lexicographic product and strong product) and the operation of taking the power of a graph. In this direction, we show that if G is a graph obtained by applying any of the operations mentioned above on non-trivial graphs, then rc(G) ≤ 2r(G) + c, where r(G) denotes the radius of G and c ∈ {0, 1, 2}. In general the rainbow connection number of a bridgeless graph can be as high as the square of its radius [1]. This is an attempt to identify some graph classes which have rainbow connection number very close to the obvious lower bound of diameter (and thus the radius). The bounds reported are tight upto additive constants. The proofs are constructive and hence yield polynomial time (2 + 2 r(G) )-factor approximation algorithms.