Deep Q-Learning for Low-Latency Tactile Applications: Microgrid Communications

Elsayed, Medhat; Erol‐Kantarci, Melike

doi:10.1109/smartgridcomm.2018.8587476

Cited by 19 publications

(17 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, we are witnessing the growth of scaled-up research towards adopting the concept of small-cell networks to improve the coverage, latency, and throughput of cellular networks, making this a promising solution for Tactile Internet applications with stringent QoS requirements [122]. In the context of networked microgrids, where a small end-to-end delay of control messages is needed, Elsayed et al [116] study the joint problem of resource allocation and user association for uplink transmission. The authors consider the coexistence of both critical user devices (e.g., microgrid controllers) and non-critical users (e.g., conventional user equipment) over a small-cell wireless network.…”

Section: Table IIImentioning

confidence: 99%

“…The uplink radio resource allocation algorithms for the Tactile Internet can be either grant-based [114]- [116] or grant-free [117]- [121] algorithms. In grant-based schemes, the UE is required to enter a scheduling and grant procedure for uplink transmission.…”

Section: A Uplink Transmissionmentioning

confidence: 99%

“…Unlike [114], [115] and [116], Ma et al [117] propose a grant-free multiple access scheme, in which tactile users do not need to enter a scheduling and grant procedure for uplink transmission to improve the uplink delay. The proposed scheme also adopts frequency diversity, where the tactile users transmit duplicated data via multiple subcarriers to improve the reliability.…”

Section: A Uplink Transmissionmentioning

confidence: 99%

See 2 more Smart Citations

A Comprehensive Survey of the Tactile Internet: State-of-the-Art and Research Directions

Promwongsa

Ebrahimzadeh

Naboulsi

et al. 2021

IEEE Commun. Surv. Tutorials

101

View full text Add to dashboard Cite

The Internet has made several giant leaps over the years, from a fixed to a mobile Internet, then to the Internet of Things, and now to a Tactile Internet. The Tactile Internet goes far beyond data, audio and video delivery over fixed and mobile networks, and even beyond allowing communication and collaboration among things. It is expected to enable haptic communications and allow skill set delivery over networks. Some examples of potential applications are telesurgery, vehicle fleets, augmented reality and industrial process automation. Several papers already cover many of the Tactile Internet-related concepts and technologies, such as haptic codecs, applications, and supporting technologies. However, none of them offers a comprehensive survey of the Tactile Internet, including its architectures and algorithms. Furthermore, none of them provides a systematic and critical review of the existing solutions. To address these lacunae, we provide a comprehensive survey of the architectures and algorithms proposed to date for the Tactile Internet. In addition, we critically review them using a well-defined set of requirements and discuss some of the lessons learned as well as the most promising research directions.

show abstract

Section: Table IIImentioning

confidence: 99%

Section: A Uplink Transmissionmentioning

confidence: 99%

Section: A Uplink Transmissionmentioning

confidence: 99%

See 1 more Smart Citation

A Comprehensive Survey of the Tactile Internet: State-of-the-Art and Research Directions

Promwongsa

Ebrahimzadeh

Naboulsi

et al. 2021

IEEE Commun. Surv. Tutorials

101

View full text Add to dashboard Cite

show abstract

“…[6] exploits machine learning to solve the Branch and Bound resource algorithm quicker, which MILP-solvers use to find solutions for MILP problems. In [12] and [13], resource block allocation has been considered for missioncritical services and micro-grid communication, where the goal is to minimize delay. The base stations are agents that interact with the environment by selecting an action given their state and receiving a reward plus their new state.…”

Section: Related Workmentioning

confidence: 99%

A Recurrent Neural Network Based Approach for Coordinating Radio and Computing Resources Allocation in Cloud-RAN

Sharara

Hoteit

2021

2021 IEEE 22nd International Conference on High Performance Switching and Routing (HPSR)

View full text Add to dashboard Cite

Cloud Radio Access Network (Cloud-RAN) is a novel architecture that aims at centralizing the baseband processing of base stations. This architecture opens paths for joint, flexible, and optimal management of radio and computing resources. To increase the benefit from this architecture, efficient resource management algorithms need to be devised. In this paper, we consider a coordinated allocation of radio and computing resources to mobile users. Optimal resource allocation that respects the Hybrid-Automatic-Repeat-Request deadline may require formulating high-complexity and resourceheavy algorithms. We consider two Integer Linear Programming problems (ILP) that implement a coordinated allocation of radio and computing resources with the objectives of maximizing throughput and maximizing users' satisfaction, respectively. Since solving these highly-complex problems requires a high execution time, we investigate low-complexity alternatives based on machine learning models; more precisely on Recurrent Neural Networks (RNN). These RNN models aim to depict the performance of the ILP problems with a much lower execution time. Our simulation results demonstrate the great ability of RNN models to perform very closely to the ILP problems while being able to reduce the execution time by up to 99.65%.

show abstract

“…DQN has huge potential for controlling a number of services in a smart city as it can continuously update to new data. In the last two years it has already been applied to routing for crowd management [56], energy efficient data collection [57], low latency microgrid communication [58] and resource allocation at the edge [59].…”

Section: ) Deep-q-network (Dqn)mentioning

confidence: 99%

Urban Intelligence With Deep Edges

White

Clarke

2020

IEEE Access

View full text Add to dashboard Cite

With the increased accuracy available from state of the art deep learning models and new embedded devices at the edge of the network capable of running and updating these models there is potential for urban intelligence at the edge of the network. The physical proximity of these edge devices will allow for intelligent reasoning one hop away from data generation. This will allow a range of modern urban reasoning applications that require reduced latency and jitter such as remote surgery, vehicle collision detection and augmented reality. The traffic flow from IoT devices to the cloud will also be reduced as with the increased accuracy from deep learning models only a subset of the data will need to be reported after a first pass analysis. However, the training time of deep learning models can be long, taking weeks on multiple desktop GPUs for large datasets. In this paper we show how transfer learning can be used to update the last layers of pre-trained models at the edge of the network, dramatically reducing the training time and allowing the model to perform new tasks without data ever having to be sent to the cloud. This will also improve the users' privacy, which is a key requirement for urban intelligence applications with the introduction of GDPR. We compare our approach to alternative IoT urban intelligence architectures such as cloud-based architectures and deep learning algorithms trained only on local data. INDEX TERMS Edge computing, transfer learning, deep learning, urban intelligence, IoT, QoS. I. INTRODUCTION Urban intelligence can provide insight into a number of the major problems being faced by our cities, such as air pollution, increased mobile communication network demand, traffic congestion and water floods, etc., through the use of data from IoT sensors and intelligent analytics on the data. This can allow for the efficient running of a smart city, making the most of the resources available. Problems in smart cities such as increased network demand and crowd congestion are set to get increasingly difficult with the UN predicting 60% of people globally will live in cities with at least half a million inhabitants by 2030 [1]. This will exacerbate the current problems in our cities, so it will be critical to mange these problems with intelligent urban reasoning algorithms and a suitable deployment architecture. IoT devices will enable access to a range of devices and information that was not available to urban authorities before. This additional information from air pollution sensors, CCTV cameras, vehicles and so on will allow the development of more advanced urban reasoning applications [2]. The associate editor coordinating the review of this manuscript and approving it for publication was Hao Ji.

show abstract

Deep Q-Learning for Low-Latency Tactile Applications: Microgrid Communications

Cited by 19 publications

References 15 publications

A Comprehensive Survey of the Tactile Internet: State-of-the-Art and Research Directions

A Comprehensive Survey of the Tactile Internet: State-of-the-Art and Research Directions

A Recurrent Neural Network Based Approach for Coordinating Radio and Computing Resources Allocation in Cloud-RAN

Urban Intelligence With Deep Edges

Contact Info

Product

Resources

About