Application of Reinforcement Learning in Production Planning and Control of Cyber Physical Production Systems

Kuhnle, Andreas; Lanza, Gisela

doi:10.1007/978-3-662-58485-9_14

Cited by 10 publications

(5 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This study demonstrates that it is possible to improve system performance by predicting its future behavior. Therefore, as future work, we intend to use machine learning (ML) in the decision-making process [18] to improve the way we predict environmental behavior and check the mathematical expectation of each incident. This helps the system make a decision, based on whether it will gain a profit or a loss by choosing a specific route.…”

Section: Discussion and Future Workmentioning

confidence: 99%

Verification and Optimization of Cyber-Physical Systems: Preprint for FedCSIS

Soltani¹,

Kang²,

Mena³

2021

Preprint

View full text Add to dashboard Cite

Optimizing CPS behavior in terms of energy consumption can have a significant impact on system reliability. The environment influences the system's behavior, and neglecting the environmental behavior has an indirect negative impact on optimizing the system's behavior. In this work, to increase the system's flexibility, the behavior of the environment is modeled dynamically to apply the disorderliness of its behavior. The resulting models are formally verified. By examining the past environmental behavior and predicting its future behavior, energy optimization is done more dynamically. The verification results acquired using a UPPAAL-SMC show that the optimization of system behavior by predicting the environmental behavior has been successful. Our approach is demonstrated using a case study within an I4 setting.

show abstract

Section: Discussion and Future Workmentioning

confidence: 99%

Verification and Optimization of Cyber-Physical Systems: Preprint for FedCSIS

Soltani¹,

Kang²,

Mena³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Cunha et al (2020) present a review paper on the use of evolutionary algorithms and deep reinforcement learning to solve job shop scheduling, as they believe that the use of deep reinforcement learning could revolutionize scheduling. Also Kuhnle and Lanza (2019) discuss possible applications of reinforcement learning in the area of production planning and control. The authors note that the complexity in production has increased significantly due to increased product diversity, lower quantities and higher quality requirements.…”

Section: Reinforcement Learning Applications In Production Controlmentioning

confidence: 99%

Modelling and condition-based control of a flexible and hybrid disassembly system with manual and autonomous workstations using reinforcement learning

et al. 2022

Self Cite

View full text Add to dashboard Cite

Remanufacturing includes disassembly and reassembly of used products to save natural resources and reduce emissions. While assembly is widely understood in the field of operations management, disassembly is a rather new problem in production planning and control. The latter faces the challenge of high uncertainty of type, quantity and quality conditions of returned products, leading to high volatility in remanufacturing production systems. Traditionally, disassembly is a manual labor-intensive production step that, thanks to advances in robotics and artificial intelligence, starts to be automated with autonomous workstations. Due to the diverging material flow, the application of production systems with loosely linked stations is particularly suitable and, owing to the risk of condition induced operational failures, the rise of hybrid disassembly systems that combine manual and autonomous workstations can be expected. In contrast to traditional workstations, autonomous workstations can expand their capabilities but suffer from unknown failure rates. For such adverse conditions a condition-based control for hybrid disassembly systems, based on reinforcement learning, alongside a comprehensive modeling approach is presented in this work. The method is applied to a real-world production system. By comparison with a heuristic control approach, the potential of the RL approach can be proven simulatively using two different test cases.

show abstract

“…This is achieved by exploiting its learnt experience and exploring new strategies (Sutton and Barto 2018). Kuhnle and Lanza (2019) applied RL in production planning and control of a Cyber Physical System (CPS) in which physical resources are monitored and controlled through computer-based algorithms. They addressed the decisions that are related to both order dispatching and maintenance management.…”

Section: Reinforcement Learning With Simulation Modelsmentioning

confidence: 99%

Simulation and deep reinforcement learning for adaptive dispatching in semiconductor manufacturing systems

et al. 2021

View full text Add to dashboard Cite

Fabrication areas in semiconductor industry are considered one of the most complex production systems. This complexity is caused by the high-mix of products and end-user market-based demands in that industry. Its dynamic and challenging processing requirements affect the handling capabilities of traditional production management paradigms. In this paper, we propose an application for dispatching and resources allocation through reinforcement learning. The application is based on a discrete-event simulation model for a case study of a real semiconductor manufacturing system. The model is built using both data-driven and agent-based approaches. The model simulates the various processing aspects that are present normally in these complex systems. The model's agents are responsible for dispatching tasks and allocation of the different system's resources. They employ Deep-Q-Network reinforcement learning. They learn simultaneously through the model execution. An independent Deep-Q-Network is trained for each agent. The model provides the training environment for the agents in which their decisions are applied and assessed for their adequacy. Our formulation of the environment's state and the reward function for the learning algorithms creates cooperative decision-making policies for the agents. This results in improving the global performance of the whole system, and the performance of each agent's resources. Our approach is compared to heuristics-based strategies that are applied in our case study. It achieved better production performance than the currently applied strategy.

show abstract

Application of Reinforcement Learning in Production Planning and Control of Cyber Physical Production Systems

Cited by 10 publications

References 9 publications

Verification and Optimization of Cyber-Physical Systems: Preprint for FedCSIS

Verification and Optimization of Cyber-Physical Systems: Preprint for FedCSIS

Modelling and condition-based control of a flexible and hybrid disassembly system with manual and autonomous workstations using reinforcement learning

Simulation and deep reinforcement learning for adaptive dispatching in semiconductor manufacturing systems

Contact Info

Product

Resources

About