Energy-Based Models for Deep Probabilistic Regression

Gustafsson, Fredrik; Danelljan, Martin; Bhat, Goutam; Schön, Thomas B.

doi:10.1007/978-3-030-58565-5_20

Cited by 50 publications

(46 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast to our shallow NN approach, deep learning techniques are also employed in the current applied NN research. Here, the focus is on probabilistic regressions in combination with importance sampling, noise contrastive estimation, and maximum likelihood estimation (Gustafsson et al, 2020;Andersson et al, 2021;Gedon et al, 2021). In the area of probabilistic NN techniques, the estimation of probability density functions in a data-based context should be mentioned as a further approach that might find applications in controloriented system modeling.…”

Section: Preliminariesmentioning

confidence: 99%

Interval Extension of Neural Network Models for the Electrochemical Behavior of High-Temperature Fuel Cells

Rauh

Auer

2022

Front. Control Eng.

View full text Add to dashboard Cite

In various research projects, it has been demonstrated that feedforward neural network models (possibly extended toward dynamic representations) are efficient means for identifying numerous dependencies of the electrochemical behavior of high-temperature fuel cells. These dependencies include external inputs such as gas mass flows, gas inlet temperatures, and the electric current as well as internal fuel cell states such as the temperature. Typically, the research on using neural networks in this context is focused only on point-valued training data. As a result, the neural network provides solely point-valued estimates for such quantities as the stack voltage and instantaneous fuel cell power. Although advantageous, for example, for robust control synthesis, quantifying the reliability of neural network models in terms of interval bounds for the network’s output has not yet received wide attention. In practice, however, such information is essential for optimizing the utilization of the supplied fuel. An additional goal is to make sure that the maximum power point is not exceeded since that would lead to accelerated stack degradation. To solve the data-driven modeling task with the focus on reliability assessment, a novel offline and online parameterization strategy for interval extensions of neural network models is presented in this paper. Its functionality is demonstrated using real-life measured data for a solid oxide fuel cell stack that is operated with temporally varying electric currents and fuel gas mass flows.

show abstract

Section: Preliminariesmentioning

confidence: 99%

Interval Extension of Neural Network Models for the Electrochemical Behavior of High-Temperature Fuel Cells

Rauh

Auer

2022

Front. Control Eng.

View full text Add to dashboard Cite

show abstract

“…In energy-based regression [7,20,22], the task is addressed by learning to model the distribution p(y|x) with a conditional EBM p(y|x; θ), defined according to,…”

Section: Energy-based Regressionmentioning

confidence: 99%

“…where {y (m) i } M m=1 ∼ q(y) are M samples drawn from a proposal distribution q(y). The aforementioned approach is relatively simple, yet it has been shown effective for various regression tasks within computer vision [7,20,22]. In these works, the proposal q(y) is set to a mixture of K Gaussian components centered at the true target y i , i.e.…”

Section: Energy-based Regressionmentioning

confidence: 99%

“…modelling tasks [60,15,58,43,12,18,16,46,1,11]. Recent work has further demonstrated the promise of EBMs for probabilistic regression, achieving impressive results for a variety of important low-dimensional regression tasks, including object detection, visual tracking, pose estimation, age estimation and robot policy learning [7,20,22,21,25,42,14].…”

Section: Introductionmentioning

confidence: 99%

“…Secondly, EBMs are known to be difficult to sample from, which complicates their practical use at test-time. To produce predictions, prior work [7,20,22,21] resort to gradientbased refinement of an initial estimate generated by a separately trained network.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learning Proposals for Practical Energy-Based Regression

Gustafsson¹,

Danelljan²,

Schön³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Energy-based models (EBMs) have experienced a resurgence within machine learning in recent years, including as a promising alternative for probabilistic regression. However, energy-based regression requires a proposal distribution to be manually designed for training, and an initial estimate has to be provided at test-time. We address both of these issues by introducing a conceptually simple method to automatically learn an effective proposal distribution, which is parameterized by a separate network head. To this end, we derive a surprising result, leading to a unified training objective that jointly minimizes the KL divergence from the proposal to the EBM, and the negative log-likelihood of the EBM. At test-time, we can then employ importance sampling with the trained proposal to efficiently evaluate the learned EBM and produce stand-alone predictions. Furthermore, we utilize our derived training objective to learn mixture density networks (MDNs) with a jointly trained energy-based teacher, consistently outperforming conventional MDN training on four real-world regression tasks within computer vision. Code is available at https: //github.com/fregu856/ebms_proposals.

show abstract