The dynamics of behavior: Review of Sutton and Barto: Reinforcement Learning: An Introduction (2nd ed.)

Staddon, J. E. R.

doi:10.1002/jeab.587

Cited by 17 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(2) Dispatching Rule 2: Firstly, 2according to Equation (15), the job J i with the largest estimated remaining processing time is selected from the uncompleted jobs and its operation O i(OPi(t)+1) is selected. A suitable machine for O i(OPi(t)+1) then is selected according to Equation (14).…”

Section: Action Setmentioning

confidence: 99%

“…In 2018, some scholars applied deep reinforcement learning (DRL) to the scheduling field and then it was widely used, which attracted the attention and competitive research of scholars in China and abroad. The basic components of reinforcement learning are the environment, agents, the behavior policy, the reward and the value function, where the learning process is usually described by a Markov decision process (MDP) [14]. For large-scale problems, it is necessary to parameterize it through a policy network and to balance exploration and exploitation, which ensures that the scheduling agent converges to the optimal or near-optimal solution in a reasonable time, thus improving the adaptability and self-learning of production scheduling in intelligent manufacturing.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Reinforcement Learning for Dynamic Flexible Job Shop Scheduling with Random Job Arrival

Chang

et al. 2022

Processes

View full text Add to dashboard Cite

The production process of a smart factory is complex and dynamic. As the core of manufacturing management, the research into the flexible job shop scheduling problem (FJSP) focuses on optimizing scheduling decisions in real time, according to the changes in the production environment. In this paper, deep reinforcement learning (DRL) is proposed to solve the dynamic FJSP (DFJSP) with random job arrival, with the goal of minimizing penalties for earliness and tardiness. A double deep Q-networks (DDQN) architecture is proposed and state features, actions and rewards are designed. A soft ε-greedy behavior policy is designed according to the scale of the problem. The experimental results show that the proposed DRL is better than other reinforcement learning (RL) algorithms, heuristics and metaheuristics in terms of solution quality and generalization. In addition, the soft ε-greedy strategy reasonably balances exploration and exploitation, thereby improving the learning efficiency of the scheduling agent. The DRL method is adaptive to the dynamic changes of the production environment in a flexible job shop, which contributes to the establishment of a flexible scheduling system with self-learning, real-time optimization and intelligent decision-making.

show abstract

Section: Action Setmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Deep Reinforcement Learning for Dynamic Flexible Job Shop Scheduling with Random Job Arrival

Chang

et al. 2022

Processes

View full text Add to dashboard Cite

show abstract

“…Reinforcement learning (RL) is a branch of machine learning, [1,2] which is an agent that interacts with an environment through a sequence of state observation, action (a k ) decision, reward (R k ) receive, and value (Q (S, A) ) update. The aim is to obtain a policy consisting of state-action pairs to guide the agent to maximize reward, which is typically used to solve Markov decision process (MDP) or partially observable Markov decision process (POMDP) problems.…”

Section: Introductionmentioning

confidence: 99%

Intrinsic Decay Property of Ti/TiO_x/Pt Memristor for Reinforcement Learning

Dai

Guo

Feng

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

A memristor‐based reinforcement learning (RL) system has shown outstanding performance in achieving efficient autonomous decision‐making and edge computing. Sarsa (λ) is a classical multistep RL algorithm that records state with λ decay and guides policy updates, significantly improving the algorithm convergence speed. However, λ decay implementation of traditional computing hardware is confined by the extensive computation of power exponential decay. Herein, the value update equation for Sarsa (λ) is implemented by using the topological structure of the memristor array, without complex circuits. Where, most importantly, the critical λ decay function is realized by a TiOx‐based memristor with conductance decay property. The energy required for floating‐point operations can be significantly reduced while accelerating the convergence speed. Then, a path planning task is demonstrated based on intrinsic conductance decay property and shows outstanding performance. Finally, the information of rounds used for the task is obtained, which is based on the intrinsic decay property of the TiOx‐based memristor, maps into a 32 × 32 memristor array in parallel to calculate the value of each round. The results indicate that the experimental data have similar results to the simulations. Herein, thus, it provides a hardware‐enabled scheme for the memristor‐based RL algorithm implementation.

show abstract

“…In [22,25,[31][32][33][34][35][36][37][38][39][40][41][42][43], shallow and deep neural networks have been suggested to approximate the Q-value function to achieve better optimisation results with shorter convergence time. Low convergence time is also desirable for online applications.…”

Section: Introductionmentioning

confidence: 99%

Reinforcement Learning for Energy-Storage Systems in Grid-Connected Microgrids: An Investigation of Online vs. Offline Implementation

et al. 2021

View full text Add to dashboard Cite

Grid-connected microgrids consisting of renewable energy sources, battery storage, and load require an appropriate energy management system that controls the battery operation. Traditionally, the operation of the battery is optimised using 24 h of forecasted data of load demand and renewable energy sources (RES) generation using offline optimisation techniques, where the battery actions (charge/discharge/idle) are determined before the start of the day. Reinforcement Learning (RL) has recently been suggested as an alternative to these traditional techniques due to its ability to learn optimal policy online using real data. Two approaches of RL have been suggested in the literature viz. offline and online. In offline RL, the agent learns the optimum policy using predicted generation and load data. Once convergence is achieved, battery commands are dispatched in real time. This method is similar to traditional methods because it relies on forecasted data. In online RL, on the other hand, the agent learns the optimum policy by interacting with the system in real time using real data. This paper investigates the effectiveness of both the approaches. White Gaussian noise with different standard deviations was added to real data to create synthetic predicted data to validate the method. In the first approach, the predicted data were used by an offline RL algorithm. In the second approach, the online RL algorithm interacted with real streaming data in real time, and the agent was trained using real data. When the energy costs of the two approaches were compared, it was found that the online RL provides better results than the offline approach if the difference between real and predicted data is greater than 1.6%.

show abstract

The dynamics of behavior: Review of Sutton and Barto: Reinforcement Learning: An Introduction (2^nd ed.)

Cited by 17 publications

References 18 publications

Deep Reinforcement Learning for Dynamic Flexible Job Shop Scheduling with Random Job Arrival

Deep Reinforcement Learning for Dynamic Flexible Job Shop Scheduling with Random Job Arrival

Intrinsic Decay Property of Ti/TiO_x/Pt Memristor for Reinforcement Learning

Reinforcement Learning for Energy-Storage Systems in Grid-Connected Microgrids: An Investigation of Online vs. Offline Implementation

Contact Info

Product

Resources

About

The dynamics of behavior: Review of Sutton and Barto: Reinforcement Learning: An Introduction (2nd ed.)

Cited by 17 publications

References 18 publications

Deep Reinforcement Learning for Dynamic Flexible Job Shop Scheduling with Random Job Arrival

Deep Reinforcement Learning for Dynamic Flexible Job Shop Scheduling with Random Job Arrival

Intrinsic Decay Property of Ti/TiOx/Pt Memristor for Reinforcement Learning

Reinforcement Learning for Energy-Storage Systems in Grid-Connected Microgrids: An Investigation of Online vs. Offline Implementation

Contact Info

Product

Resources

About

The dynamics of behavior: Review of Sutton and Barto: Reinforcement Learning: An Introduction (2^nd ed.)

Intrinsic Decay Property of Ti/TiO_x/Pt Memristor for Reinforcement Learning