Reinforcement learning for combined production-maintenance and quality control of a manufacturing system with deterioration failures

Paraschos, Panagiotis D.; Koulinas, Georgios K.; Koulouriotis, Dimitrios E.

doi:10.1016/j.jmsy.2020.07.004

Cited by 77 publications

(27 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have reported the continuous formulation of QWK (for the results, the values reported are generally those from discrete QWK, while the continuous version is used only for the state-of-the-art experimental comparison in the training process [37]) according to Eq. (11).…”

Section: Ordinal Metricsmentioning

confidence: 99%

“…The application of ML and DL techniques offers great opportunities to automatize the overall QC process [11]. Indeed, these methodologies for QC have been employed in several industrial areas, but state-of-the-art is mainly oriented to present ad hoc rather than vanilla ML solutions capable of dealing with challenges of this domain, namely the intrinsic variability of the annotation procedure and the difficulty to generalize across different sets.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A novel deep ordinal classification approach for aesthetic quality control classification

Rosati

Romeo

Vargas

et al. 2022

Neural Comput & Applic

View full text Add to dashboard Cite

Nowadays, decision support systems (DSSs) are widely used in several application domains, from industrial to healthcare and medicine fields. Concerning the industrial scenario, we propose a DSS oriented to the aesthetic quality control (AQC) task, which has quickly established itself as one of the most crucial challenges of Industry 4.0. Taking into account the increasing amount of data in this domain, the application of machine learning (ML) and deep learning (DL) techniques offers great opportunities to automatize the overall AQC process. State-of-the-art is mainly oriented to approach this problem with a nominal DL classification method which does not exploit the ordinal structure of the AQC task, thus not penalizing the error among distant AQC classes (which is a relevant aspect for the real use case). The paper introduces a DL ordinal methodology for the AQC classification. Differently from other deep ordinal methods, we combined the standard categorical cross-entropy with the cumulative link model and we imposed the ordinal constraint via the thresholds and slope parameters. Experimental results were performed for solving an AQC task on a novel image dataset originated from a specific company’s demand (i.e., aesthetic assessment of wooden stocks). We demonstrated how the proposed methodology is able to reduce misclassification errors (up to 0.937 quadratic weight kappa loss) among distant classes while overcoming other state-of-the-art deep ordinal models and reducing the bias factor related to the item geometry. The proposed DL approach was integrated as the main core of a DSS supported by Internet of Things (IoT) architecture that can support the human operator by reducing up to 90% the time needed for the qualitative analysis carried out manually in this specific domain.

show abstract

Section: Ordinal Metricsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A novel deep ordinal classification approach for aesthetic quality control classification

Rosati

Romeo

Vargas

et al. 2022

Neural Comput & Applic

View full text Add to dashboard Cite

show abstract

“…To minimize the makespan for an MCP scheduling problem, the authors of [25] propose a reinforcement learning (RL) algorithm to setup a change scheduling method. [26] discusses a Reinforcement Learning method to find the optimal trade-off between conflicting performance metrics for the optimization of the total expected profit of the system.…”

Section: B Deep Reinforcement Learning Applications In Decision Makin...mentioning

confidence: 99%

Online Multimodal Transportation Planning using Deep Reinforcement Learning

Farahani¹,

Genga²,

Dijkman³

2021

Preprint

View full text Add to dashboard Cite

In this paper we propose a Deep Reinforcement Learning approach to solve a multimodal transportation planning problem, in which containers must be assigned to a truck or to trains that will transport them to their destination. While traditional planning methods work "offline" (i.e., they take decisions for a batch of containers before the transportation starts), the proposed approach is "online", in that it can take decisions for individual containers, while transportation is being executed. Planning transportation online helps to effectively respond to unforeseen events that may affect the original transportation plan, thus supporting companies in lowering transportation costs. We implemented different container selection heuristics within the proposed Deep Reinforcement Learning algorithm and we evaluated its performance for each heuristic using data that simulate a realistic scenario, designed on the basis of a real case study at a logistics company. The experimental results revealed that the proposed method was able to learn effective patterns of container assignment. It outperformed tested competitors in terms of total transportation costs and utilization of train capacity by 20.48% to 55.32% for the cost and by 7.51% to 20.54% for the capacity. Furthermore, it obtained results within 2.7% for the cost and 0.72% for the capacity of the optimal solution generated by an Integer Linear Programming solver in an offline setting.

show abstract

“…There have been some research studies that propose distributed dynamic maintenance scheduling where the agents (sub-components) decide about their optimal maintenance individually, such as [2], [30], and [23].…”

Section: Introductionmentioning

confidence: 99%

“…The authors consider finite discrete values for the degradation state and solve the problem using Q-learning. The authors of [23] propose a dynamic maintenance and production scheduling using a RL approach by considering one agent with discrete action space. These studies do not consider coupling constraints among the agents that ensure that the total demand is fulfilled.…”

Section: Introductionmentioning

confidence: 99%

Distributed joint dynamic maintenance and production scheduling in manufacturing systems: Framework based on model predictive control and Benders decomposition

Rokhforoz¹,

Fink²

2020

Preprint

View full text Add to dashboard Cite

Scheduling the maintenance based on the condition, respectively the degradation level of the system leads to improved system's reliability while minimizing the maintenance cost. Since the degradation level changes dynamically during the system's operation, we face a dynamic maintenance scheduling problem.In this paper, we address the dynamic maintenance scheduling of manufacturing systems based on their degradation level. The manufacturing system consists of several units with a defined capacity and an individual dynamic degradation model, seeking to optimize their reward. The units sell their production capacity, while maintaining the systems based on the degradation state to prevent failures. The manufacturing units are jointly responsible for fulfilling the demand of the system. This induces a coupling constraint among the agents. Hence, we face a large-scale mixed-integer dynamic maintenance scheduling problem. In order to handle the dynamic model of the system and large-scale optimization, we propose a distributed algorithm using model predictive control (MPC) and Benders decomposition method. In the proposed algorithm, first, the master problem obtains the maintenance scheduling for all the agents, and then based on this data, the agents obtain their optimal production using the distributed MPC method which employs the dual decomposition approach to tackle the coupling constraints among the agents. The effectiveness of the proposed method is investigated on a case study.

show abstract

Reinforcement learning for combined production-maintenance and quality control of a manufacturing system with deterioration failures

Cited by 77 publications

References 45 publications

A novel deep ordinal classification approach for aesthetic quality control classification

A novel deep ordinal classification approach for aesthetic quality control classification

Online Multimodal Transportation Planning using Deep Reinforcement Learning

Distributed joint dynamic maintenance and production scheduling in manufacturing systems: Framework based on model predictive control and Benders decomposition

Contact Info

Product

Resources

About