Dynamic Dispatching for Large-Scale Heterogeneous Fleet via Multi-agent Deep Reinforcement Learning

Zhang, Chi; Odonkor, Philip; Zheng, Shuai; Khorasgani, Hamed; Serita, Susumu; Gupta, Chetan; Wang, Haiyan

doi:10.1109/bigdata50022.2020.9378191

Cited by 21 publications

(13 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As this process is repeated, the agent gradually learns to seek out positive actions and avoid negative ones and eventually determines an optimal (maximum‐reward) path to its goal. [ 18–20 ] Unlike supervised learning algorithms, reinforcement learning does not need correct answers or targets to solve a given problem, it only needs information on whether an answer from the ML agent is in the correct direction. Reinforcement learning has useful applications in robotics, as well as optimization.…”

Section: Ai Paradigms Techniques and Workflowsmentioning

confidence: 99%

“…Recurrent Neural Networks have been used to optimize delivery and dispatch services by autonomous guided vehicles while avoiding conflicts with workers or other vehicles. [ 54 ] RL has also been used to optimize dispatch within a facility [ 20 ] (such as a factory floor or warehouse), and for job‐shop scheduling—where one product requiring several tasks which must be completed on separate machines while ensuring an optimal layout of equipment. [ 55 ] Other use case applications for RL include improving the ability of robots to identify and pick out an object from specific bins, [ 56 ] select the best paths to minimize unnecessary stops, and avoid obstacles and interference with human operators.…”

Section: Ai/ml Applications In Manufacturingmentioning

confidence: 99%

See 1 more Smart Citation

A review of artificial intelligence applications in manufacturing operations

Plathottam,

Rzonca,

Lakhnori

et al. 2023

J Adv Manuf & Process

View full text Add to dashboard Cite

Artificial intelligence (AI) and machine learning (ML) can improve manufacturing efficiency, productivity, and sustainability. However, using AI in manufacturing also presents several challenges, including issues with data acquisition and management, human resources, infrastructure, as well as security risks, trust, and implementation challenges. For example, getting the data needed to train AI models can be difficult for rare events or costly for large datasets that need labeling. AI models can also pose security risks when integrated into industrial control systems. In addition, some industry players may be hesitant to use AI due to a lack of trust or understanding of how it works. Despite these challenges, AI has the potential to be extremely helpful in manufacturing, particularly in applications such as predictive maintenance, quality assurance, and process optimization. It is important to consider the specific needs and capabilities of each manufacturing scenario when deciding whether and how to use AI in manufacturing. This review identifies current developments, challenges, and future directions in AI/ML relevant to manufacturing, with the goal of improving understanding of AI/ML technologies available for solving manufacturing problems, providing decision‐support for prioritizing and selecting appropriate AI/ML technologies, and identifying areas where further research can yield transformational returns for the industry. Early experience suggests that AI/ML can have significant cost and efficiency benefits in manufacturing, especially when combined with the ability to capture enormous amounts of data from manufacturing systems.

show abstract

Section: Ai Paradigms Techniques and Workflowsmentioning

confidence: 99%

Section: Ai/ml Applications In Manufacturingmentioning

confidence: 99%

A review of artificial intelligence applications in manufacturing operations

Plathottam,

Rzonca,

Lakhnori

et al. 2023

J Adv Manuf & Process

View full text Add to dashboard Cite

show abstract

“…Foerster, Assael, Freitas, and Whiteson proposed a single network with shared parameters to reduce the number of learned parameters and speed up the learning [12]. In a previous work, we applied a similar approach to address a dynamic dispatching problem in an industrial multi-agent environment [13]. Having a shared policy among agents solves the agent failure challenge and addresses the non-stationary problem to some extent.…”

Section: Multi-agent Deep Rl Literature Reviewmentioning

confidence: 99%

K-nearest Multi-agent Deep Reinforcement Learning for Collaborative Tasks with a Variable Number of Agents

Khorasgani¹,

Wang²,

Tang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Traditionally, the performance of multi-agent deep reinforcement learning algorithms are demonstrated and validated in gaming environments where we often have a fixed number of agents. In many industrial applications, the number of available agents can change at any given day and even when the number of agents is known ahead of time, it is common for an agent to break during the operation and become unavailable for a period of time. In this paper, we propose a new deep reinforcement learning algorithm for multi-agent collaborative tasks with a variable number of agents. We demonstrate the application of our algorithm using a fleet management simulator developed by Hitachi to generate realistic scenarios in a production site.Index Terms-Multi-agent deep reinforcement, fleet traffic control, variable number of agents.

show abstract

“…By modeling each router using a DQN, each router is able to account for heterogeneous data about its environment, which allows for the optimization of more complicated cost functions, such as the simultaneous optimization of bag delivery time and energy consumption in a baggage handling system. Zhang et al (2020a) proposed a centralized multi-agent DQN approach for the open-pit mining operational planning (OPMOP) problem (an NP-hard problem that seeks to balance the tradeoffs between mine productivity and operational costs), which works based on learning the memories from heterogeneous agents. Open-pit mine dispatch decisions coordinate the route planning of trucks to shovels and dumps for the loading and delivery of ore.…”

Section: Pathfinding + Schedulingmentioning

confidence: 99%

“… Zhang et al (2020a) proposed a centralized multi-agent DQN approach for the open-pit mining operational planning (OPMOP) problem (an NP-hard problem that seeks to balance the tradeoffs between mine productivity and operational costs), which works based on learning the memories from heterogeneous agents. Open-pit mine dispatch decisions coordinate the route planning of trucks to shovels and dumps for the loading and delivery of ore.…”

Section: Applicationsmentioning

confidence: 99%

A review of the applications of multi-agent reinforcement learning in smart factories

Bahrpeyma

Reichelt

2022

Front. Robot. AI

View full text Add to dashboard Cite

The smart factory is at the heart of Industry 4.0 and is the new paradigm for establishing advanced manufacturing systems and realizing modern manufacturing objectives such as mass customization, automation, efficiency, and self-organization all at once. Such manufacturing systems, however, are characterized by dynamic and complex environments where a large number of decisions should be made for smart components such as production machines and the material handling system in a real-time and optimal manner. AI offers key intelligent control approaches in order to realize efficiency, agility, and automation all at once. One of the most challenging problems faced in this regard is uncertainty, meaning that due to the dynamic nature of the smart manufacturing environments, sudden seen or unseen events occur that should be handled in real-time. Due to the complexity and high-dimensionality of smart factories, it is not possible to predict all the possible events or prepare appropriate scenarios to respond. Reinforcement learning is an AI technique that provides the intelligent control processes needed to deal with such uncertainties. Due to the distributed nature of smart factories and the presence of multiple decision-making components, multi-agent reinforcement learning (MARL) should be incorporated instead of single-agent reinforcement learning (SARL), which, due to the complexities involved in the development process, has attracted less attention. In this research, we will review the literature on the applications of MARL to tasks within a smart factory and then demonstrate a mapping connecting smart factory attributes to the equivalent MARL features, based on which we suggest MARL to be one of the most effective approaches for implementing the control mechanism for smart factories.

show abstract

Dynamic Dispatching for Large-Scale Heterogeneous Fleet via Multi-agent Deep Reinforcement Learning

Cited by 21 publications

References 12 publications

A review of artificial intelligence applications in manufacturing operations

A review of artificial intelligence applications in manufacturing operations

K-nearest Multi-agent Deep Reinforcement Learning for Collaborative Tasks with a Variable Number of Agents

A review of the applications of multi-agent reinforcement learning in smart factories

Contact Info

Product

Resources

About