Reinforcement learning for joint pricing, lead-time and scheduling decisions in make-to-order systems

Li, Xueping; Wang, Jiao; Sawhney, Rapinder

doi:10.1016/j.ejor.2012.03.020

Cited by 32 publications

(6 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…According to Ng and Russell (2000), IRL may be useful when an agent is learning a "skilled behaviour," such as the planning optimal scheduling process, and for which the reward function being optimized is determined by "a natural system", such as a production system. Li et al (2012) propose the use of Q-learning algorithm-based reinforcement learning for joint pricing and lead time decisions in a make-to-order system, where the decision problem is modelled as a semi-Markov decision problem. Tuncel et al (2014) propose a Monte Carlo reinforcement learning algorithm for line balancing in disassembly operations under uncertain demand.…”

Section: Choosing An Appropriate Machine Learning Algorithmmentioning

confidence: 99%

Designing and developing smart production planning and control systems in the industry 4.0 era: a methodology and case study

et al. 2021

View full text Add to dashboard Cite

In furtherance of emerging research within smart production planning and control (PPC), this paper prescribes a methodology for the design and development of a smart PPC system. A smart PPC system uses emerging technologies such as the internet of things, big-data analytics tools and machine learning running on the cloud or on edge devices to enhance performance of PPC processes. It achieves this by using a wider range of data sources from the production system, capturing and utilizing the experience of production planners, using analytics and machine learning to harness insights from the data and allowing dynamic and near real-time action to the continuously changing production system. The proposed methodology is illustrated with a case study in a sweets and snacks manufacturing company, to highlight the key considerations and challenges production managers might face during its application. The case further demonstrates considerations for scalability and flexibility via a loosely coupled, service-oriented architecture and the selection of fitting algorithms respectively to address a business requirement for a short-term, multi-criteria and event-driven production planning and control solution. Finally, the paper further discusses the challenges of PPC in smart manufacturing and the importance of fitting smart technologies to planning environment characteristics.

show abstract

Section: Choosing An Appropriate Machine Learning Algorithmmentioning

confidence: 99%

Designing and developing smart production planning and control systems in the industry 4.0 era: a methodology and case study

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Although the concept of RL is not widely applied in manufacturing, it is still not entirely new. RL techniques have been used in problems, such as scheduling [24], [25], production goal regulation [26], and the concept of biological manufacturing systems (BMS) [27]. Although, these approaches are not directly applicable to the ramp-up problem, they provide insight on how to model manufacturing problems for ML.…”

Section: Learning Approaches For Ramp-upmentioning

confidence: 99%

An MDP Model-Based Reinforcement Learning Approach for Production Station Ramp-Up Optimization: Q-Learning Analysis

Doltsinis

Ferreira

Lohse

2014

IEEE Trans. Syst. Man Cybern, Syst.

View full text Add to dashboard Cite

Ramp-up is a significant bottleneck for the introduction of new or adapted manufacturing systems. The effort and time required to ramp-up a system is largely dependent on the effectiveness of the human decision making process to select the most promising sequence of actions to improve the system to the required level of performance. Although existing work has identified significant factors influencing the effectiveness of rampup, little has been done to support the decision making during the process. This paper approaches ramp-up as a sequential adjustment and tuning process that aims to get a manufacturing system to a desirable performance in the fastest possible time. Production stations and machines are the key resources in a manufacturing system. They are often functionally decoupled and can be treated in the first instance as independent rampup problems. Hence, this paper focuses on developing a Markov decision process (MDP) model to formalize ramp-up of production stations and enable their formal analysis. The aim is to capture the cause-and-effect relationships between an operator's adaptation or adjustment of a station and the station's response to improve the effectiveness of the process. Reinforcement learning has been identified as a promising approach to learn from rampup experience and discover more successful decision-making policies. Batch learning in particular can perform well with little data. This paper investigates the application of a Q-batch learning algorithm combined with an MDP model of the ramp-up process. The approach has been applied to a highly automated production station where several ramp-up processes are carried out. The convergence of the Q-learning algorithm has been analyzed along with the variation of its parameters. Finally, the learned policy has been applied and compared against previous ramp-up cases.

show abstract

“…The idea of Q-learning based on value iteration has been widely used to solve scheduling problems as a type of reinforcement learning. Li et al 34 modeled issues regarding the order decision making and scheduling in an order system and proposed an algorithm based on Q-learning. Xanthopoulos et al 35 proposed a reinforcement learning–based approach for a single scheduling problem with stochastic arrivals and processing time.…”

Section: Introductionmentioning

confidence: 99%

A scheduling procedure for a flow shop–like knowledgeable manufacturing cell with self-evolutionary features

Yan

2016

Proceedings of the Institution of Mechanical Engineers, Part B:

View full text Add to dashboard Cite

As a component of knowledgeable manufacturing systems, the structure of flow shop–like knowledgeable manufacturing cells is similar to that of a flow shop, thus representing an NP-hard issue. Here, we propose a self-evolutionary algorithm that exhibits learning ability and is composed of learning and scheduling modules. Unlike traditional scheduling algorithms, whose performances remain unchanged when the procedure is coded, the performance of the algorithm proposed in this study gradually improves as the learning process continues. The self-evolutionary ability is realized through the training of a hybrid kernel support vector machine. The hybrid kernel support vector machine was designed to approximate the value of the Q-function to select the appropriate action for the scheduling module and thus to obtain the optimal solution. An iterative process of value based on the Q-learning was adopted to train the hybrid kernel support vector machine to gradually enhance the algorithm’s efficiency and accuracy. The extracted state features of the flow shop–like knowledgeable manufacturing cells serve as inputs to hybrid kernel support vector machine for easy generalization of the learning results. The action exerted on a feasible solution is also defined as the input of the hybrid kernel support vector machine. The computational results show that the performance of the proposed procedure improves as the learning process progresses. Data from the computation and comparisons with other algorithms verify the validity and efficiency of the proposed algorithm.

show abstract

Reinforcement learning for joint pricing, lead-time and scheduling decisions in make-to-order systems

Cited by 32 publications

References 36 publications

Designing and developing smart production planning and control systems in the industry 4.0 era: a methodology and case study

Designing and developing smart production planning and control systems in the industry 4.0 era: a methodology and case study

An MDP Model-Based Reinforcement Learning Approach for Production Station Ramp-Up Optimization: Q-Learning Analysis

A scheduling procedure for a flow shop–like knowledgeable manufacturing cell with self-evolutionary features

Contact Info

Product

Resources

About