Variational Bayesian learning of nonlinear hidden state-space models for model predictive control

Raiko, Tapani; Tornio, Matti

doi:10.1016/j.neucom.2009.06.009

Cited by 24 publications

(22 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where T −1 is the precision matrix of the unnormalized Gaussian in (45). If the state x has the same representation as the target vector, T −1 is a diagonal matrix with entries either unity or zero, scaled by 1/σ 2 c .…”

Section: Cost Functionmentioning

confidence: 99%

Gaussian Processes for Data-Efficient Learning in Robotics and Control

Deisenroth

Fox

Rasmussen

2015

IEEE Trans. Pattern Anal. Mach. Intell.

562

482

View full text Add to dashboard Cite

Autonomous learning has been a promising direction in control and robotics for more than a decade since data-driven learning allows to reduce the amount of engineering knowledge, which is otherwise required. However, autonomous reinforcement learning (RL) approaches typically require many interactions with the system to learn controllers, which is a practical limitation in real systems, such as robots, where many interactions can be impractical and time consuming. To address this problem, current learning approaches typically require task-specific knowledge in form of expert demonstrations, realistic simulators, pre-shaped policies, or specific knowledge about the underlying dynamics. In this paper, we follow a different approach and speed up learning by extracting more information from data. In particular, we learn a probabilistic, non-parametric Gaussian process transition model of the system. By explicitly incorporating model uncertainty into long-term planning and controller learning our approach reduces the effects of model errors, a key problem in model-based learning. Compared to state-of-the art RL our model-based policy search method achieves an unprecedented speed of learning. We demonstrate its applicability to autonomous learning in real robot and control tasks.

show abstract

Section: Cost Functionmentioning

confidence: 99%

Gaussian Processes for Data-Efficient Learning in Robotics and Control

Deisenroth

Fox

Rasmussen

2015

IEEE Trans. Pattern Anal. Mach. Intell.

562

482

View full text Add to dashboard Cite

show abstract

“…In essence, these methods work by approximating the intractable posterior distribution of interest with a tractable parametric distribution and then "correcting" the estimation through the use of a cost function that is based on the discrepancy or misfit between the distributions (often Kullback-Leibler divergence). These methods have proven valuable in applications involving control theory (e.g., Raiko and Tornio 2009), and as they continue to develop, may prove to be useful for environmental and ecological applications as well.…”

Section: Gqn Implementationmentioning

confidence: 99%

A general science-based framework for dynamical spatio-temporal models

Wikle

Hooten

2010

TEST

180

203

View full text Add to dashboard Cite

Spatio-temporal statistical models are increasingly being used across a wide variety of scientific disciplines to describe and predict spatially-explicit processes that evolve over time. Correspondingly, in recent years there has been a significant amount of research on new statistical methodology for such models. Although descriptive models that approach the problem from the second-order (covariance) perspective are important, and innovative work is being done in this regard, many real-world processes are dynamic, and it can be more efficient in some cases to characterize the associated spatio-temporal dependence by the use of dynamical models. The chief challenge with the specification of such dynamical models has been related to the curse of dimensionality. Even in fairly simple linear, first-order Markovian, Gaussian error settings, statistical models are often over parameterized. Hierarchical models have proven invaluable in their ability to deal to some extent with this issue by allowing dependency among groups of parameters. In addition, this framework has allowed for the specification of science based parameterizations (and associated prior distributions) in which classes of deterministic dynamical models (e.g., partial differential equations (PDEs), integro-difference equations (IDEs), matrix models, and agent-based models) are used to guide specific parameterizations. Most of the focus for the application of such models in statistics has been in the linear case. The problems mentioned above with linear dynamic models are compounded in the case of nonlinear models. In this sense, the need for coherent and sensible model parameterizations is not only helpful, it is essential. Here, we present an overview of a framework for incorporating scientific information to motivate dynamical spatio-temporal models. First, we illustrate the methodology with the linear case. We then develop a general nonlinear spatio-temporal framework that we call general quadratic nonlinearity and demonstrate that it accommodates many different classes of scientific-based parameterizations as special cases. The model is presented in a hierarchical Bayesian framework and is illustrated with examples from ecology and oceanography.

show abstract

“…Finally, (c) existing methods do not allow an explicit trade-off between the complexity of the hidden state and the quality of the prediction error. Notice, however, that Variational Bayesian methods can be used as a partial solution to the first problem [24,32], as long as the function class covers the underlying data distribution, which remains an open problem for natural images.…”

Section: Related Workmentioning

confidence: 99%

A Separation Principle for Control in the Age of Deep Learning

Achille

Soatto

2018

Annu. Rev. Control Robot. Auton. Syst.

View full text Add to dashboard Cite

We review the problem of defining and inferring a state for a control system based on complex, high-dimensional, highly uncertain measurement streams, such as videos. Such a state, or representation, should contain all and only the information needed for control and discount nuisance variability in the data. It should also have finite complexity, ideally modulated depending on available resources. This representation is what we want to store in memory in lieu of the data, as it separates the control task from the measurement process. For the trivial case with no dynamics, a representation can be inferred by minimizing the information bottleneck Lagrangian in a function class realized by deep neural networks. The resulting representation has much higher dimension than the data (already in the millions) but is smaller in the sense of information content, retaining only what is needed for the task. This process also yields representations that are invariant to nuisance factors and have maximally independent components. We extend these ideas to the dynamic case, where the representation is the posterior density of the task variable given the measurements up to the current time, which is in general much simpler than the prediction density maintained by the classical Bayesian filter. Again, this can be finitely parameterized using a deep neural network, and some applications are already beginning to emerge. No explicit assumption of Markovianity is needed; instead, complexity trades off approximation of an optimal representation, including the degree of Markovianity. 11.1

show abstract

Variational Bayesian learning of nonlinear hidden state-space models for model predictive control

Cited by 24 publications

References 16 publications

Gaussian Processes for Data-Efficient Learning in Robotics and Control

Gaussian Processes for Data-Efficient Learning in Robotics and Control

A general science-based framework for dynamical spatio-temporal models

A Separation Principle for Control in the Age of Deep Learning

Contact Info

Product

Resources

About