Reinforcement Learning for Cyber-Physical Systems

Liu, Xing; Xu, Hansong; Liao, Weixian; Yu, Wei

doi:10.1109/icii.2019.00063

Cited by 20 publications

(4 citation statements)

References 77 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reinforcement learning (RL) is a branch of the dynamic ML paradigm where software agents learn to make decisions in an environment by receiving feedback as rewards [43]. Through a cycle of observations, actions, and learning, the agents gradually improve their decision-making abilities to achieve long-term goals.…”

Section: Artificial Intelligence and Machine Learning (Ai/ml)mentioning

confidence: 99%

“…This proactive approach ensures network resilience and reduces downtime. Despite RL's promising capabilities for optimizing O-RAN and the emerging O-RAN-based IoT systems, challenges related to training data, model robustness, and real-time adaptation, among others, remain open for the research community to solve [43,44].…”

Section: Artificial Intelligence and Machine Learning (Ai/ml)mentioning

confidence: 99%

See 1 more Smart Citation

Open Radio Access Networks for Smart IoT Systems: State of Art and Future Directions

Musa,

Hussaini,

Qian

et al. 2023

Future Internet

Self Cite

View full text Add to dashboard Cite

The Internet of Things (IoT) constitutes a vast network comprising various components such as physical devices, vehicles, buildings, and other items equipped with sensors, actuators, and software. These components are interconnected, facilitating the collection and exchange of copious data across networked communications. IoT empowers extensive monitoring and control over a myriad of objects, enabling them to gather and disseminate data that bolster applications, thereby enhancing the system’s capacity for informed decision making, environmental surveillance, and autonomous inter-object interaction, all without the need for direct human involvement. These systems have achieved seamless connectivity requirements using the next-generation wireless network infrastructures (5G, 6G, etc.), while their diverse reliability and quality of service (QoS) requirements across various domains require more efficient solutions. Open RAN (O-RAN), i.e., open radio open access network (RAN), promotes flexibility and intelligence in the next-generation RAN. This article reviews the applications of O-RAN in supporting the next-generation smart world IoT systems by conducting a thorough survey. We propose a generic problem space, which consists of (i) IoT Systems: transportation, industry, healthcare, and energy; (ii) targets: reliable communication, real-time analytics, fault tolerance, interoperability, and integration; and (iii) artificial intelligence and machine learning (AI/ML): reinforcement learning (RL), deep neural networks (DNNs), etc. Furthermore, we outline future research directions concerning robust and scalable solutions, interoperability and standardization, privacy, and security. We present a taxonomy to unveil the security threats to emerge from the O-RAN-assisted IoT systems and the feasible directions to move this research forward.

show abstract

Section: Artificial Intelligence and Machine Learning (Ai/ml)mentioning

confidence: 99%

Section: Artificial Intelligence and Machine Learning (Ai/ml)mentioning

confidence: 99%

Open Radio Access Networks for Smart IoT Systems: State of Art and Future Directions

Musa,

Hussaini,

Qian

et al. 2023

Future Internet

Self Cite

View full text Add to dashboard Cite

show abstract

“…In CPS, vast amounts of data will be generated, representing space and time while operating themselves. Such data is generally denoted as CPS spatial-temporal data [9][10][11].…”

Section: Introductionmentioning

confidence: 99%

Deep Neural Networks for Spatial-Temporal Cyber-Physical Systems: A Survey

et al. 2023

Self Cite

View full text Add to dashboard Cite

Cyber-physical systems (CPS) refer to systems that integrate communication, control, and computational elements into physical processes to facilitate the control of physical systems and effective monitoring. The systems are designed to interact with the physical world, monitor and control the physical processes while in operation, and generate data. Deep Neural Networks (DNN) comprise multiple layers of interconnected neurons that process input data to produce predictions. Spatial-temporal data represents the physical world and its evolution over time and space. The generated spatial-temporal data is used to make decisions and control the behavior of CPS. This paper systematically reviews the applications of DNNs, namely convolutional, recurrent, and graphs, in handling spatial-temporal data in CPS. An extensive literature survey is conducted to determine the areas in which DNNs have successfully captured spatial-temporal data in CPS and the emerging areas that require attention. The research proposes a three-dimensional framework that considers: CPS (transportation, manufacturing, and others), Target (spatial-temporal data processing, anomaly detection, predictive maintenance, resource allocation, real-time decisions, and multi-modal data fusion), and DNN schemes (CNNs, RNNs, and GNNs). Finally, research areas that need further investigation are identified, such as performance and security. Addressing data quality, strict performance assurance, reliability, safety, and security resilience challenges are the areas that are required for further research.

show abstract

“…RL, a branch of ML in which an agent learns to make sequential decisions through interactions with an environment, is considered in [44], [53], [54], [67], [77]. The research focused on the context of planning and control in the business plan and logistics industrial systems.…”

mentioning

confidence: 99%

Artificial intelligence and internet of things in manufacturing decision processes

Wijaya,

Hermanto Rudy,

Debora

et al. 2024

IJ-AI

View full text Add to dashboard Cite

This paper explores the influence of the internet of things (IoT) and artificial intelligence (AI) on the decision-making processes of modern manufacturing systems. With the proliferation of IoT devices and the development of AI technologies, manufacturing companies increasingly leverage these technologies to improve their decision-making abilities. This study aims to investigate the potential benefits, difficulties, and ramifications of integrating IoT and AI in manufacturing systems. The review employs the preferred reporting items for systematic reviews and meta-analyses (PRISMA) method with a systematic mapping process with four research questions. A total of 1282 articles were collected between 2017 and 2023, reviewed in accordance with the inclusion and exclusion criteria, and 66 articles were chosen. The research on IoT and AI technologies influentially affects other research in the production control layer manufacturing area based on the top-ten cited articles. In contrast, the research in this area focused on the operations management layer, specifically manufacturing analytics processes. This paper’s findings contribute to a greater understanding of the impact of IoT and AI on decision-making in modern multi-domain manufacturing systems and provide direction for future research in this field.

show abstract

Reinforcement Learning for Cyber-Physical Systems

Cited by 20 publications

References 77 publications

Open Radio Access Networks for Smart IoT Systems: State of Art and Future Directions

Open Radio Access Networks for Smart IoT Systems: State of Art and Future Directions

Deep Neural Networks for Spatial-Temporal Cyber-Physical Systems: A Survey

Artificial intelligence and internet of things in manufacturing decision processes

Contact Info

Product

Resources

About