Optimization of global production scheduling with deep reinforcement learning

Waschneck, Bernd; Reichstaller, André; Belzner, Lenz; Altenmüller, Thomas; Bauernhansl, Thomas; Knapp, Alexander; Kyek, Andreas

doi:10.1016/j.procir.2018.03.212

Cited by 281 publications

(82 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Constant reward type was used for training. Constant [36] and variable [10] reward types have been utilised in the earlier RL studies. The maximum level of training steps (4000), and other parameters (Table 4) were fixed based on the initial testing.…”

Section: Discussionmentioning

confidence: 99%

“…RL can be modelled as a Markov Decision Process consisting of states, actions, transitions, and rewards (and its discount factor). RL has been utilised in many real world applications such as learning to play Go (AlphaGo), Atari's 2600 video games, production scheduling [36], and video transcoding [10]. Random Forests have traditionally been successful for solving classification problems (examples in [37]).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Online architecture for predicting live video transcoding resources

2019

View full text Add to dashboard Cite

End users stream video increasingly from live broadcasters (via YouTube Live, Twitch etc.). Adaptive live video streaming is realised by transcoding different representations of the original video content. Management of transcoding resources creates costs for the service provider, because transcoding is a CPU-intensive task. Additionally, the content must be transcoded within real time with the transcoding resources in order to provide satisfying Quality of Service. The contribution of this paper is validation of an online architecture for enabling live video transcoding with Docker in a Kubernetes-based cloud environment. Particularly, online cloud resource allocation has been focused on by executing experiments in several configurations. The results indicate that Random Forest regressor provided the best overall performance in terms of precision regarding transcoding speed and CPU consumption on resources, and the amount of realised transcoding tasks. Reinforcement Learning provided lower performance, and required more effort in terms of training.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Online architecture for predicting live video transcoding resources

2019

View full text Add to dashboard Cite

show abstract

“…DQN proposed by Google DeepMind is a kind of well‐known deep reinforcement learning. In this work, the motivation to utilize DQN architecture is threefold.…”

Section: Problem Formulatementioning

confidence: 99%

Task scheduling based on deep reinforcement learning in a cloud manufacturing environment

Dong

Xue

Xiao

et al. 2020

Concurrency and Computation

View full text Add to dashboard Cite

Cloud manufacturing promotes the transformation of intelligence for the traditional manufacturing mode. In a cloud manufacturing environment, the task scheduling plays an important role.However, as the number of problem instances increases, the solution quality and computation time always go against. Existing task scheduling algorithms can get local optimal solutions with the high computational cost, especially for large problem instances. To tackle this problem, a task scheduling algorithm based on a deep reinforcement learning architecture (RLTS) is proposed to dynamically schedule tasks with precedence relationship to cloud servers to minimize the task execution time. Meanwhile, the Deep-Q-Network, as a kind of deep reinforcement learning algorithms, is employed to consider the problem of complexity and high dimension.In the simulation, the performance of the proposed algorithm is compared with other four heuristic algorithms. The experimental results show that RLTS can be effective to solve the task scheduling in a cloud manufacturing environment. KEYWORDS cloud manufacturing, Deep-Q-Network, deep reinforcement learning, task scheduling INTRODUCTIONThe traditional manufacturing is transforming into the intelligent manufacturing due to the development of information and technology.Cloud manufacturing, 1 as an advanced manufacturing paradigm, is a manifestation of the concept of ''Manufacturing as a Service''. The cloud manufacturing platform aggregates a collection of distributed servers to execute the demand of users. However, it still faces many challenges in some aspects, such as security, sustainability, reliability, and optimized management. Task scheduling optimization between the server resources and the user's demand tasks is the critical factors restricting the development of cloud manufacturing. [2][3][4] In a large-scale distributed cloud manufacturing environment, unreasonable task scheduling will cause the problems of reducing cloud server resource utilization, degrading system performance, and increasing operating costs. Therefore, how to schedule tasks reasonably and effectively has always been the focus in both industrial and academic communities.The task scheduling in this work is simply described. The user submits an application consisting of tasks with the precedence constraint to the cloud platform. When it receives the application, servers in the platform are allocated according to the demand of each task to finish the user's request. It is critical for the platform to efficiently leverage its servers to finish the timely and cost effective delivery to users. SchedulingConcurrency Computat Pract Exper. 2020;32:e5654. wileyonlinelibrary.com/journal/cpe

show abstract

“…In terms of system utilization, the RL-agent outperforms FIFO by around 10%. Furthermore, Waschneck et al (2018) propose a combination of supervised learning and deep RL for job-shop scheduling in a semiconductor production [19]. They compare the performance of their approach with an event handler, which operates based on expert knowledge.…”

Section: Related Workmentioning

confidence: 99%

Evolving Neural Networks to Solve a Two-Stage Hybrid Flow Shop Scheduling Problem with Family Setup Times

Lang¹,

Reggelin²,

Behrendt³

et al. 2020

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

We present a novel strategy to solve a two-stage hybrid flow shop scheduling problem with family setup times. The problem is derived from an industrial case. Our strategy involves the application of NeuroEvolution of Augmenting Topologies -a genetic algorithm, which generates arbitrary neural networks being able to estimate job sequences. The algorithm is coupled with a discrete-event simulation model, which evaluates different network configurations and provides training signals. We compare the performance and computational efficiency of the proposed concept with other solution approaches. Our investigations indicate that NeuroEvolution of Augmenting Topologies can possibly compete with state-of-the-art approaches in terms of solution quality and outperform them in terms of computational efficiency.

show abstract

Optimization of global production scheduling with deep reinforcement learning

Cited by 281 publications

References 12 publications

Online architecture for predicting live video transcoding resources

Online architecture for predicting live video transcoding resources

Task scheduling based on deep reinforcement learning in a cloud manufacturing environment

Evolving Neural Networks to Solve a Two-Stage Hybrid Flow Shop Scheduling Problem with Family Setup Times

Contact Info

Product

Resources

About