Transmission scheduling for multi-process multi-sensor remote estimation via approximate dynamic programming

Forootani, Ali; Iervolino, Raffaele; Tipaldi, Massimo; Dey, Subhrakanti

doi:10.1016/j.automatica.2021.110061

Cited by 22 publications

(10 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since our work focuses on the remote state estimation, each local estimator is assumed to be stable and operate in the steady state, i.e., the estimation error covariance of the local KF is a constant P s n (t) Pn , ∀t ∈ N, where N is the set of positive integers [4], [5], [8], [11].…”

Section: A Local State Estimationmentioning

confidence: 99%

“…However, existing works on dynamic resource allocation in NOMA-based remote estimation only considered the simple multi-sensor-single-channel setting, and focused on either power allocation [7]- [9] or channel assignment problems [10], rather than joint design ones. Moreover, the developed policy optimization methods for resource allocation can only handle very small-scale systems, e.g., a five-sensor-single-channel setting [8], due to the curse of dimensionality in policy optimization. We aim to address these limitations of NOMAbased remote estimation.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Reinforcement Learning for Radio Resource Allocation in NOMA-based Remote State Estimation

Pang¹,

Liu²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

Remote state estimation, where many sensors send their measurements of distributed dynamic plants to a remote estimator over shared wireless resources, is essential for missioncritical applications of Industry 4.0. Most of the existing works on remote state estimation assumed orthogonal multiple access and the proposed dynamic radio resource allocation algorithms can only work for very small-scale settings. In this work, we consider a remote estimation system with non-orthogonal multiple access. We formulate a novel dynamic resource allocation problem for achieving the minimum overall long-term average estimation mean-square error. Both the estimation quality state and the channel quality state are taken into account for decision making at each time. The problem has a large hybrid discrete and continuous action space for joint channel assignment and power allocation. We propose a novel action-space compression method and develop an advanced deep reinforcement learning algorithm to solve the problem. Numerical results show that our algorithm solves the resource allocation problem effectively, presents much better scalability than the literature, and provides significant performance gain compared to some benchmarks.

show abstract

Section: A Local State Estimationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Deep Reinforcement Learning for Radio Resource Allocation in NOMA-based Remote State Estimation

Pang¹,

Liu²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Classical dynamic programming algorithms were discussed to solve the long-term average MSE minimization problem. The follow-up work [19] adopted approximate dynamic programming for reducing the computation complexity. Li et al [20] formulated the multi-sensor transmission power control problem into an MDP and a Markov game, and developed Q-learning-based solutions.…”

Section: A Related Workmentioning

confidence: 99%

“…2) Power allocation only or channel assignment only policy. The existing radio resource allocations problems in remote estimation systems either considered sensor transmit power allocation [18], [20] or channel assignment [11]- [15], [17], [19]. However, for NOMA-based systems, it is ideal to jointly optimize both channel assignment and power control policies.…”

Section: B Limitations and Challengesmentioning

confidence: 99%

“…3) Resource allocation without utilizing channel state information. Existing algorithms have used instantaneous estimation quality indicators rather than channel state information (CSI) for generating resource allocation action in each time step [17]- [19]. Considering fading channel states in practice, it is necessary to utilize CSI for resource allocation.…”

Section: B Limitations and Challengesmentioning

confidence: 99%

See 1 more Smart Citation

DRL-based Resource Allocation in Remote State Estimation

Pang¹,

Liu²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

Remote state estimation, where sensors send their measurements of distributed dynamic plants to a remote estimator over shared wireless resources, is essential for mission-critical applications of Industry 4.0. Existing algorithms on dynamic radio resource allocation for remote estimation systems assumed oversimplified wireless communications models and can only work for small-scale settings. In this work, we consider remote estimation systems with practical wireless models over the orthogonal multipleaccess and non-orthogonal multiple-access schemes. We derive necessary and sufficient conditions under which remote estimation systems can be stabilized. The conditions are described in terms of the transmission power budget, channel statistics, and plants' parameters. For each multiple-access scheme, we formulate a novel dynamic resource allocation problem as a decision-making problem for achieving the minimum overall long-term average estimation mean-square error. Both the estimation quality and the channel quality states are taken into account for decision making. We systematically investigated the problems under different multiple-access schemes with large discrete, hybrid discrete-and-continuous, and continuous action spaces, respectively. We propose novel action-space compression methods and develop advanced deep reinforcement learning algorithms to solve the problems. Numerical results show that our algorithms solve the resource allocation problems effectively and provide much better scalability than the literature.

show abstract