Generalized Inverse Reinforcement Learning with Linearly Solvable MDP

Kohjima, Masahiro; Matsubayashi, Tatsushi; Sawada, Hiroshi

doi:10.1007/978-3-319-71246-8_23

Cited by 3 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a special case, Dvijotham and Todorov (2010) showed that maximum entropy IRL is the solution to a LMDP with uniform passive dynamics. Kohjima, Matsubayashi and Sawada (2017) proposed a Bayesian IRL method for learning state values for LMDPs using variational approximation.…”

mentioning

confidence: 99%

“…We present the first application of IRL for collective animal movement using Bayesian learning of state costs-to-go for an LMDP. As an extension of Kohjima, Matsubayashi and Sawada (2017), we reduce the dimension of the state space with basis function approximation, compare variational approximation to MCMC sampling, and consider the multiagent LMDP. We first demonstrate the modeling framework for a simulation of the Vicsek et al (1995) SPP model to illustrate the mechanisms of the LMDP framework in Section 3.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Bayesian Inverse Reinforcement Learning for Collective Animal Movement

Schafer¹,

Wikle²,

Hooten³

2020

Preprint

View full text Add to dashboard Cite

Agent-based methods allow for defining simple rules that generate complex group behaviors. The governing rules of such models are typically set a priori and parameters are tuned from observed behavior trajectories. Instead of making simplifying assumptions across all anticipated scenarios, inverse reinforcement learning provides inference on the short-term (local) rules governing long term behavior policies by using properties of a Markov decision process. We use the computationally efficient linearly-solvable Markov decision process to learn the local rules governing collective movement for a simulation of the self propelled-particle (SPP) model and a data application for a captive guppy population. The estimation of the behavioral decision costs is done in a Bayesian framework with basis function smoothing. We recover the true costs in the SPP simulation and find the guppies value collective movement more than targeted movement toward shelter.

show abstract

mentioning

confidence: 99%

mentioning

confidence: 99%

Bayesian Inverse Reinforcement Learning for Collective Animal Movement

Schafer¹,

Wikle²,

Hooten³

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…As a special case, Dvijotham and Todorov (2010) showed the maximum entropy IRL is the solution to a LMDP with uniform passive dynamics. Kohjima et al (2017) proposed a Bayesian IRL method for learning state values for LMDPs using variational approximation.…”

Section: Introductionmentioning

confidence: 99%

Alternative learning strategies for spatio-temporal processes of complex animal behavior

Schafer¹

View full text Add to dashboard Cite

The estimation of spatio-temporal dynamics of animal behavior processes is complicated by nonlinear interactions. Alternative learning methods such as machine learning, deep learning, and reinforcement learning have proven successful for approximating nonlinear system mechanisms for prediction and classification. These alternative learning frameworks can be linked to statistical models in a hierarchical framework to improve ecological inference and prediction in the presence of uncertainty. This dissertation provides three methodological extensions of alternative learning with statistical uncertainty quantification for modeling animal behavior dynamics at different scales. First, an efficient Bayesian Markov model is developed to provide inference on white-fronted geese behavior from individual accelerometer and location data while accounting for classification uncertainty. Second, nonlinear basis function expansions produced by a spatio-temporal echo state network are used as features in a hierarchical generalized linear model for predicting spatial patterns of mallard duck settling pattern counts. Lastly, Bayesian inverse reinforcement learning is developed to estimate the behavioral state costs for collective animal groups.

show abstract