Adaptively Shaping Reinforcement Learning Agents via Human Reward

Yu, Chao; Wang, Dongxu; Yang, Tianpei; Zhu, Wenxuan; Li, Yuchen; Ge, Hongwei; Ren, Jiankang

doi:10.1007/978-3-319-97304-3_7

Cited by 6 publications

(8 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yu et al. [16] proposed an adaptive plasticity (shaping) algorithm that can combine different human–agent RL methods and dynamically select the most favourable method during the learning process. There is also a large body of work using expert advice or demonstrations of how to form rewards in RL problems [29, 30].…”

Section: Related Workmentioning

confidence: 99%

“…As a result, a large body of work has proposed human knowledge‐based RL algorithms [10–15]. These human–agent RL algorithms convert manual instructions into a form that RL agents can recognize in order to embed human knowledge into the learning process [16]. Human–agent reinforcement learning methods have distinct advantages for different reinforcement learning tasks or at different learning stages [10].…”

Section: Introductionmentioning

confidence: 99%

“…The main contributions of this paper are as follows: Similar to Triple‐Gail [8], a new adversarial game framework that extends the original GAIL with the addition of a human decision maker is proposed. The human decision maker and generator in HKGAIL generate policy distributions given expert state‐action pairs, while the discriminator distinguishes whether a policy comes from an expert presentation or not. How to combine a policy‐based approach [11], a reward value‐based approach [16], and DRL to improve the process of policy learning is researched. The algorithm is used for driving and landing tasks, and the experimental results show that HKGAIL is able to match expert policies more effectively than the baseline approach, with a significant performance advantage. …”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

HKGAIL: Policy shaping via integrating human knowledge with generative adversarial imitation learning

Peng

Tan

2022

IET Intelligent Trans Sys

View full text Add to dashboard Cite

Generative adversarial imitation learning (GAIL) has shown good results in several research areas by taking advantage of generative adversarial networks. However, GAIL lacks a reward mechanism and usually adopts a model-free approach based on stochastic policies, which is not ideal for solving complex, dynamically uncertain population intelligence problems, especially in the face of autonomous driving environments. In this paper, a policy framework is shaped by combining the human knowledge with GAIL (HKGAIL). HKGAIL embeds human decision models into the learning process to infer the underlying structure of expert demonstrations. The skills learned from expert demonstrations can directly guide the actions (policies) of the learning process of the agents, and the policies can be optimized through the feedback function of the discriminator. Experiments on both driving and landing tasks show that HKGAIL was able to better fit the policy close to the expert, and was 16.2% safer than GAIL for the driving task.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

HKGAIL: Policy shaping via integrating human knowledge with generative adversarial imitation learning

Peng

Tan

2022

IET Intelligent Trans Sys

View full text Add to dashboard Cite

show abstract

“…The process includedi) Data collection using profile points and analysis. Data logs were created and extracted for a very short intervals of 5 minutes ii) Data extraction and analysis of the of workload demand patterns over a long period of timeduring last one year iii) Generate synthetic workloads patterns iv) Execute stress tests in test environment with large number of virtual users using systemapplications as in real world scenario v) Validate the results by extracting data from different profile points of the application threads and nodes on completion of the tests vi) Training the model using semi-supervised learning approach (deep learning paradigm) [7], [11]. vi) Forecast the likelihood of the traffic burst (excessive CPU usages) using the trained model [4], [6].…”

Section: Experiments For Validationmentioning

confidence: 99%

Using Semi-supervised Classifier to Forecast Extreme CPU Utilization

Khosla¹,

Sharma²

2020

IJAIA

View full text Add to dashboard Cite

A semi-supervised classifier is used in this paper is to investigate a model for forecasting unpredictable load on the IT systems and to predict extreme CPU utilization in a complex enterprise environment with large number of applications running concurrently. This proposed model forecasts the likelihood of a scenario where extreme load of web traffic impacts the IT systems and this model predicts the CPU utilization under extreme stress conditions. The enterprise IT environment consists of a large number of applications running in a real time system. Load features are extracted while analysing an envelope of the patterns of work-load traffic which are hidden in the transactional data of these applications. This method simulates and generates synthetic workload demand patterns, run use-case high priority scenarios in a test environment and use our model to predict the excessive CPU utilization under peak load conditions for validation. Expectation Maximization classifier with forced-learning, attempts to extract and analyse the parameters that can maximize the chances of the model after subsiding the unknown labels. As a result of this model, likelihood of an excessive CPU utilization can be predicted in short duration as compared to few days in a complex enterprise environment. Workload demand prediction and profiling has enormous potential in optimizing usages of IT resources with minimal risk.

show abstract

“…Consequently, the integration of the humans (i.e., doctors) into the learning process, and the interaction of an expert's knowledge with the automatic learning data would greatly enhance the knowledge discovery process [340]. While there is some previous work from other domains, particularly in training of robots [341], [342], human-in-theloop interactive RL is not yet well established in the healthcare domain. It remains open for future research to transfer the insights from existing studies into the healthcare domain to ensure successful applications of existing RL methods.…”

Section: B Integration Of Prior Knowledgementioning

confidence: 99%

Reinforcement Learning in Healthcare: A Survey

Yu¹,

Liu²,

Nemati³

2019

Preprint

Self Cite

View full text Add to dashboard Cite

As a subfield of machine learning, reinforcement learning (RL) aims at empowering one's capabilities in behavioural decision making by using interaction experience with the world and an evaluative feedback. Unlike traditional supervised learning methods that usually rely on one-shot, exhaustive and supervised reward signals, RL tackles with sequential decision making problems with sampled, evaluative and delayed feedback simultaneously. Such distinctive features make RL technique a suitable candidate for developing powerful solutions in a variety of healthcare domains, where diagnosing decisions or treatment regimes are usually characterized by a prolonged and sequential procedure. This survey will discuss the broad applications of RL techniques in healthcare domains, in order to provide the research community with systematic understanding of theoretical foundations, enabling methods and techniques, existing challenges, and new insights of this emerging paradigm. By first briefly examining theoretical foundations and key techniques in RL research from efficient and representational directions, we then provide an overview of RL applications in a variety of healthcare domains, ranging from dynamic treatment regimes in chronic diseases and critical care, automated medical diagnosis from both unstructured and structured clinical data, as well as many other control or scheduling domains that have infiltrated many aspects of a healthcare system. Finally, we summarize the challenges and open issues in current research, and point out some potential solutions and directions for future research.

show abstract

Adaptively Shaping Reinforcement Learning Agents via Human Reward

Cited by 6 publications

References 8 publications

HKGAIL: Policy shaping via integrating human knowledge with generative adversarial imitation learning

HKGAIL: Policy shaping via integrating human knowledge with generative adversarial imitation learning

Using Semi-supervised Classifier to Forecast Extreme CPU Utilization

Reinforcement Learning in Healthcare: A Survey

Contact Info

Product

Resources

About