Deriving confidence from artificial neural networks for navigation

Curro, Joseph; Raquet, John

doi:10.1109/plans.2018.8373526

Cited by 6 publications

(14 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many works that predict mixture distributions constrain the mixture components to fixed, pre-defined actions or road lanes [26,18]. Optimizing for general, unconstrained mixture distributions requires special initialization and training procedures and suffers from mode collapse; see [44,8,9,35,15,17]. Their findings are consistent with our experiments.…”

Section: Introductionsupporting

confidence: 88%

“…Choi et al [7] utilized MDNs for uncertainties in autonomous driving by using mixture components as samples alternative to dropout [47]. However, optimizing for a general mixture distribution comes with problems, such as numerical instability, requirement for good initializations, and collapsing to a single mode [44,8,9,35,15,17]. The Evolving WTA loss and two stage approach proposed in this work addresses these problems.…”

Section: Related Workmentioning

confidence: 99%

“…Optimizing all parameters jointly in MDNs is difficult, becomes numerically unstable in higher dimensions, and suffers from degenerate predictions [44,8]. Moreover, MDNs are usually prone to overfitting, which requires special regularization techniques and results in mode collapse [9,15,35,17]. We use methodology similar to [17] and sequentially learn first the means, then the variances and finally all parameters jointly.…”

Section: Mdn Baselinementioning

confidence: 99%

See 2 more Smart Citations

Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction

Makansi

Ilg

Çiçek

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

177

174

View full text Add to dashboard Cite

Future prediction is a fundamental principle of intelligence that helps plan actions and avoid possible dangers. As the future is uncertain to a large extent, modeling the uncertainty and multimodality of the future states is of great relevance. Existing approaches are rather limited in this regard and mostly yield a single hypothesis of the future or, at the best, strongly constrained mixture components that suffer from instabilities in training and mode collapse. In this work, we present an approach that involves the prediction of several samples of the future with a winner-takes-all loss and iterative grouping of samples to multiple modes. Moreover, we discuss how to evaluate predicted multimodal distributions, including the common real scenario, where only a single sample from the ground-truth distribution is available for evaluation. We show on synthetic and real data that the proposed approach triggers good estimates of multimodal distributions and avoids mode collapse.

show abstract

Section: Introductionsupporting

confidence: 88%

Section: Related Workmentioning

confidence: 99%

Section: Mdn Baselinementioning

confidence: 99%

See 1 more Smart Citation

Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction

Makansi

Ilg

Çiçek

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

177

174

View full text Add to dashboard Cite

show abstract

“…In the bottom row the results of all three methods exhibit higher uncertainties in the third and forth layers and the mean velocities deviate from the true velocity. This is because surface waves are not sensitive to complex structures with thin low velocity zones (Jan van Heijst et al 1994 of MDNs cannot be resolved by increasing the number of kernels as in these results only a few kernels contribute to the final pdfs (all others are assigned near-zero weights), a property found also in previous studies (Hjorth & Nabney 1999;Rupprecht et al 2017;Curro & Raquet 2018;Cui et al 2019;Makansi et al 2019;.…”

Section: D Surface Wave Dispersion Inversionsupporting

confidence: 56%

“…MDNs have been applied to surface wave dispersion inversion (Meier et al 2007a,b;Cao et al 2020), 2D travel time tomography , petrophysical inversion (Shahraeeni & Curtis 2011;Shahraeeni et al 2012), earthquake source parameter estimation (Käufl et al 2014(Käufl et al , 2015, and Earth's radial seismic structure inversion (de Wit et al 2013). However MDNs become difficult to train in high dimensionality because of numerical instability, and they suffer from mode collapse, that is, some modes of the posterior pdf are missing in the results (Hjorth & Nabney 1999;Rupprecht et al 2017;Curro & Raquet 2018;Cui et al 2019;Makansi et al 2019). Consequently they are less effective at inferring correlations between parameters, so in practice usually very low (often single) dimensional marginal distributions are inferred (Meier et al 2007a,b;).…”

Section: Introductionmentioning

confidence: 99%

Bayesian geophysical inversion using invertible neural networks

Zhang¹,

Curtis²

2021

Preprint

View full text Add to dashboard Cite

Constraining geophysical models with observed data usually involves solving nonlinear and non-unique inverse problems. Mixture density networks (MDNs) produce an efficient way to estimate Bayesian posterior probability density functions (pdf's) that represent the non-unique solution. However it is difficult to infer correlations between parameters using MDNs, and in turn to draw samples from the posterior pdf. We introduce an alternative to resolve these issues: invertible neural networks (INNs). These are simultaneously trained to represent uncertain forward functions and to solve Bayesian inverse problems. In its usual form, the method becomes less effective in high dimensionality because it uses maximum mean discrepancy (MMD) to train the neural network. We show that this issue can be overcome by maximising the likelihood of the data used for training. We apply the method to two types of imaging problems: 1D surface wave dispersion inversion and 2D travel time tomography, and compare the results to those obtained using Monte Carlo and MDNs. Results show that INNs provide comparable posterior pdfs to those obtained using Monte Carlo, including correlations between parameters, and provide more accurate marginal distributions than MDNs. After training, INNs estimate posterior pdfs in seconds on a typical desktop computer. Hence they can be used to provide efficient solutions for repeated inverse problems using different data sets. And even accounting for training time, our results also show that INNs can be more efficient than Monte Carlo methods for solving single inverse problems.

show abstract

Bayesian Geophysical Inversion Using Invertible Neural Networks

Zhang

Curtis

2021

JGR Solid Earth

View full text Add to dashboard Cite

Geoscientists build models of the subsurface to understand properties and processes in the Earth's interior. The models are usually parameterized in some way, so to constrain the models, we must solve a parameter estimation problem. Data are recorded which provide constraints, together with prior knowledge. However, since the physical relationships between parameters and data usually predict data given the parameters (known as the forward calculation) but not the reverse, the solution must be found using inverse theory.Geophysical inverse problems usually have nonunique solutions due to noise in the data, to nonlinearity of the physical relationships between model parameters and data, and to fundamentally unconstrained combinations of parameters. Uncertainties in parameter estimates must therefore be quantified to interpret inversion results correctly. Unfortunately, estimating uncertainty in nonlinear inverse problems can be computationally expensive, and the cost increases both with the number of parameters, and with the computational cost of the forward calculation. In this study, we therefore solve two different types of seismic tomography problems which each have fewer than 100 parameters, and have relatively rapid forward functions (each evaluation takes on the order of seconds). This allows us to evaluate solutions sufficiently accurately to thoroughly test a new method of Geophysical inversion.Geophysical inverse problems are traditionally solved by linearizing (approximating) the nonlinear physics, and using optimization methods which seek a model that minimizes the misfit between observed and predicted data (Aki & Lee, 1976;Dziewonski & Woodhouse, 1987;Iyer & Hirahara, 1993). However, despite their wide applications, linearized procedures cannot produce accurate estimates of uncertainty because data uncertainty distributions can deviate substantially from analytic forms of probability such as Gaussians or Uniform distributions that can be propagated through linearized methods efficiently, and since the distribution of possible models post inversion can be strongly affected by nonlinearity in the physics (Bo-

show abstract

Deriving confidence from artificial neural networks for navigation

Cited by 6 publications

References 17 publications

Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction

Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction

Bayesian geophysical inversion using invertible neural networks

Bayesian Geophysical Inversion Using Invertible Neural Networks

Contact Info

Product

Resources

About