Inverse reinforcement learning from summary data

Kangasrääsiö, Antti; Kaski, Samuel

doi:10.1007/s10994-018-5730-4

Cited by 12 publications

(11 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It can also be seen as a variant of Maximum Causal Entropy Inverse Reinforcement Learning (Ziebart et al, 2010): while inverse reinforcement learning (IRL) requires demonstrations, or at least state sequences without actions Yu et al, 2018), we learn a reward function from a single state, albeit with the simplifying assumption of known dynamics. This can also be seen as an instance of IRL from summary data (Kangasrääsiö and Kaski, 2018).…”

Section: Related Workmentioning

confidence: 99%

Preferences Implicit in the State of the World

Shah¹,

Dmitrii²,

Alexander³

et al. 2019

Preprint

View full text Add to dashboard Cite

Reinforcement learning (RL) agents optimize only the features specified in a reward function and are indifferent to anything left out inadvertently. This means that we must not only specify what to do, but also the much larger space of what not to do. It is easy to forget these preferences, since these preferences are already satisfied in our environment. This motivates our key insight: when a robot is deployed in an environment that humans act in, the state of the environment is already optimized for what humans want. We can therefore use this implicit preference information from the state to fill in the blanks. We develop an algorithm based on Maximum Causal Entropy IRL and use it to evaluate the idea in a suite of proof-of-concept environments designed to show its properties. We find that information from the initial state can be used to infer both side effects that should be avoided as well as preferences for how the environment should be organized. Our code can be found at https://github.com/HumanCompatibleAI/rlsp.

show abstract

Section: Related Workmentioning

confidence: 99%

Preferences Implicit in the State of the World

Shah¹,

Dmitrii²,

Alexander³

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…expand this work to the situation where the agent's stochastic policy mapping is also unknown and needs to be estimated. Kangasrääsiö and Kaski (2018) present a model-free IRL method where only partial expert trajectories are available. The authors present three methods to estimate the rewards under this assumption.…”

Section: Model-free Irlmentioning

confidence: 99%

A survey of inverse reinforcement learning

2022

View full text Add to dashboard Cite

Learning from demonstration, or imitation learning, is the process of learning to act in an environment from examples provided by a teacher. Inverse reinforcement learning (IRL) is a specific form of learning from demonstration that attempts to estimate the reward function of a Markov decision process from examples provided by the teacher. The reward function is often considered the most succinct description of a task. In simple applications, the reward function may be known or easily derived from properties of the system and hard coded into the learning process. However, in complex applications, this may not be possible, and it may be easier to learn the reward function by observing the actions of the teacher. This paper provides a comprehensive survey of the literature on IRL. This survey outlines the differences between IRL and two similar methods - apprenticeship learning and inverse optimal control. Further, this survey organizes the IRL literature based on the principal method, describes applications of IRL algorithms, and provides areas of future research.

show abstract

“…is framework not only outperforms the state-of-the-art with a large margin but also reveals valuable facts on learning face representation. Literature [9][10][11] proposes the reinforcement learning model, which matches behavioral data to explain the observed strategic behavior. e results show that this method is more effective than similar methods.…”

Section: Related Workmentioning

confidence: 99%

Multiperspective Light Field Reconstruction Method via Transfer Reinforcement Learning

Cai

Luo

Zhou

et al. 2020

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Compared with traditional imaging, the light field contains more comprehensive image information and higher image quality. However, the available data for light field reconstruction are limited, and the repeated calculation of data seriously affects the accuracy and the real-time performance of multiperspective light field reconstruction. To solve the problems, this paper proposes a multiperspective light field reconstruction method based on transfer reinforcement learning. Firstly, the similarity measurement model is established. According to the similarity threshold of the source domain and the target domain, the reinforcement learning model or the feature transfer learning model is autonomously selected. Secondly, the reinforcement learning model is established. The model uses multiagent (i.e., multiperspective) Q-learning to learn the feature set that is most similar to the target domain and the source domain and feeds it back to the source domain. This model increases the capacity of the source-domain samples and improves the accuracy of light field reconstruction. Finally, the feature transfer learning model is established. The model uses PCA to obtain the maximum embedding space of source-domain and target-domain features and maps similar features to a new space for label data migration. This model solves the problems of multiperspective data redundancy and repeated calculations and improves the real-time performance of maneuvering target recognition. Extensive experiments on PASCAL VOC datasets demonstrate the effectiveness of the proposed algorithm against the existing algorithms.

show abstract

Inverse reinforcement learning from summary data

Cited by 12 publications

References 29 publications

Preferences Implicit in the State of the World

Preferences Implicit in the State of the World

A survey of inverse reinforcement learning

Multiperspective Light Field Reconstruction Method via Transfer Reinforcement Learning

Contact Info

Product

Resources

About