Reinforcement Learning-Based Control and Networking Co-Design for Industrial Internet of Things

Xu, Hansong; Liu, Xing; Yu, Wei; Griffith, David; Golmie, Nada

doi:10.1109/jsac.2020.2980909

Cited by 57 publications

(18 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This method automatically assigned sampling rates and backoff delays to the control and network subsystems in the industrial Internet of things system. In order to improve the system performance of highly coupled industrial IoT, according to the characteristics of industrial systems, Reference [25] leveraged reinforcement learning technology to automatically configure control and network systems in dynamic industrial environments. Three new strategies are designed to accelerate the convergence of reinforcement learning.…”

Section: Scheduling Technologies In Iiotmentioning

confidence: 99%

Deep Reinforcement Learning for Scheduling in an Edge Computing‐Based Industrial Internet of Things

Zhang

Jiaqi

et al. 2021

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

The demand for improving productivity in manufacturing systems makes the industrial Internet of things (IIoT) an important research area spawned by the Internet of things (IoT). In IIoT systems, there is an increasing demand for different types of industrial equipment to exchange stream data with different delays. Communications between massive heterogeneous industrial devices and clouds will cause high latency and require high network bandwidth. The introduction of edge computing in the IIoT can address unacceptable processing latency and reduce the heavy link burden. However, the limited resources in edge computing servers are one of the difficulties in formulating communication scheduling and resource allocation strategies. In this article, we use deep reinforcement learning (DRL) to solve the scheduling problem in edge computing to improve the quality of services provided to users in IIoT applications. First, we propose a hierarchical scheduling model considering the central-edge computing heterogeneous architecture. Then, according to the model characteristics, a deep intelligent scheduling algorithm (DISA) based on a double deep Q network (DDQN) framework is proposed to make scheduling decisions for communication. We compare DISA with other baseline solutions using various performance metrics. Simulation results show that the proposed algorithm is more effective than other baseline algorithms.

show abstract

Section: Scheduling Technologies In Iiotmentioning

confidence: 99%

Deep Reinforcement Learning for Scheduling in an Edge Computing‐Based Industrial Internet of Things

Zhang

Jiaqi

et al. 2021

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

show abstract

“…Authors in [99] leveraged RL to assign actions to networking and control systems in a combined manner under a dynamic IIoT environment. More specifically, based on the data forwarded from sensing nodes about the system, an extended Kalman filter estimates a system's state which is forwarded to an RL based agent which decides commands for the networking and control.…”

Section: B Techniques Using Edge-cloud Architecturementioning

confidence: 99%

Edge Intelligence for Data Handling and Predictive Maintenance in IIOT

Hafeez

McArdle

2021

IEEE Access

View full text Add to dashboard Cite

The use of IoT has become pervasive and IoT devices are common in many domains. Industrial IoT (IIoT) utilises IoT devices and sensors to monitor machines and environments to ensure optimal performance of equipment and processes. Predictive Maintenance (PM) which monitors the health of machines to determine the probable failure of components is one IIoT technique which is receiving attention lately. To achieve effective PM, massive amounts of data are collected, processed and ultimately analysed by Machine Learning (ML) algorithms. Traditionally IoT sensors transmit their data readings to the cloud for processing and modelling. Handling and transmitting massive amounts of data between IoT devices and infrastructure has a cost. Edge Computing (EC) in which both sensors and intermediate nodes can process data provides opportunities to reduce data transmission costs and increase processing speed. This article examines IIoT for PM and discusses how and where data can be processed and analysed. Initially, this article presents sampling and data reduction techniques. These techniques allow for a reduction in the amount of data transmitted to the cloud for processing but there are potential accuracy trade-offs when ML algorithms utilise reduced datasets. An alternative approach is to move ML algorithms closer to the data to reduce data transmission. There are three main techniques that utilise the EC paradigm to perform ML and data processing on intermediary nodes. These techniques are categorized according to where data processing occurs: Device and Edge, Edge and Cloud and Device and Cloud (Federated Learning). In addition to exploring traditional approaches, these three state-of-the-art techniques are examined in this article and their benefits and weaknesses are presented. A novel architecture to demonstrate how EC can be utilized both for data reduction and PM in IIoT is also proposed.

show abstract

“…Machine learning models trained on sample data are used to make predictions and decisions on entirely new input, and are used in a wide variety of applications, including spam filtering, computer vision, anomaly detection, and malware analysis, and are especially useful in areas where it is very difficult, very costly, or infeasible to develop a conventional algorithm to perform the complex task [41]. Machine Learning can be applied to a number of areas of the IoT search framework, and could play a critical role in optimizing performance [41], [70], [71]. For example, machine learning could be applied to predict the density of queries such that the system could dynamically adjust the time interval, over which queries are aggregated to avoid large query latency.…”

Section: B Intelligencementioning

confidence: 99%

Towards Efficient and Intelligent Internet of Things Search Engine

et al. 2021

Self Cite

View full text Add to dashboard Cite

The Internet of Things (IoT) has created a novel ecosystem for sensing and actuation throughout our world, enabling intelligently controlled autonomous systems to conserve energy, water crops, manage factories, and provide situation awareness on an unprecedented scale. As IoT progresses, the interest in IoT search engines, that is, search engines to find IoT devices and retrieve IoT data, has grown. While basic examples of IoT search engines exist, considerable challenges prevent the full realization of an efficient and intelligent IoT search engine that provides universal data service, scalable data communication and retrieval, and efficient querying of massively distributed heterogeneous devices and data. In this article, we first propose a generic framework for the IoT search engine, and then present a naming service for the IoT system, an essential component for an effective IoT search engine. We also outline some research challenges and possible solutions for building efficiency and intelligence in the IoT search engine. Further, we present a case study and seek to address a particular aspect of the query process for IoT search, namely efficient and timely query processing. Given the now obvious advances in machine learning, the potential for deep learning-based prediction to improve resource use, and thus query retrieval, is clear. In detail, we utilize Long-Short-Term Memory (LSTM) neural network architecture to predict aggregated query volumes to be preemptively applied and stored for immediate response. Combining several realistic IoT datasets, we explore the efficacy of simultaneously predicting multiple targets for predictive query retrieval.INDEX TERMS Machine learning, Internet of Things, search engine, edge intelligence, applications.

show abstract

Reinforcement Learning-Based Control and Networking Co-Design for Industrial Internet of Things

Cited by 57 publications

References 34 publications

Deep Reinforcement Learning for Scheduling in an Edge Computing‐Based Industrial Internet of Things

Deep Reinforcement Learning for Scheduling in an Edge Computing‐Based Industrial Internet of Things

Edge Intelligence for Data Handling and Predictive Maintenance in IIOT

Towards Efficient and Intelligent Internet of Things Search Engine

Contact Info

Product

Resources

About