Computer Model Calibration with Confidence and Consistency

Plumlee, Matthew

doi:10.1111/rssb.12314

Cited by 24 publications

(14 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As an alternative to a prior on model parameters, one can take a functional route by directly specifying a prior on the functional space f w (x); commonly done using Gaussian processes (GP). Indeed, GPs have a long history of success in engineering applications [319,253,132,103] and their success in IoFT may pave the way for many new applications. However, the challenge is that GPs are based on correlations and do not conform to the ERM paradigm FL is currently based on.…”

Section: ) Uncertainty Quantification and Bayesian Methodsmentioning

confidence: 99%

The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning

Kontar¹,

Shi²,

Yue³

et al. 2021

Preprint

View full text Add to dashboard Cite

The Internet of Things (IoT) is on the verge of a major paradigm shift. In the IoT system of the future, IoFT, the "cloud" will be substituted by the "crowd" where model training is brought to the edge, allowing IoT devices to collaboratively extract knowledge and build smart analytics/models while keeping their personal data stored locally. This paradigm shift was set into motion by the tremendous increase in computational power on IoT devices and the recent advances in decentralized and privacypreserving model training, coined as federated learning (FL). This article provides a vision for IoFT and a systematic overview of current efforts towards realizing this vision. Specifically, we first introduce the defining characteristics of IoFT and discuss FL data-driven approaches, opportunities, and challenges that allow decentralized inference within three dimensions: (i) a global model that maximizes utility across all IoT devices, (ii) a personalized model that borrows strengths across all devices yet retains its own model, (iii) a meta-learning model that quickly adapts to new devices or learning tasks. We end by describing the vision and challenges of IoFT in reshaping different industries through the lens of domain experts. Those industries include manufacturing, transportation, energy, healthcare, quality & reliability, business, and computing.

show abstract

Section: ) Uncertainty Quantification and Bayesian Methodsmentioning

confidence: 99%

The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning

Kontar¹,

Shi²,

Yue³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…[17] develop variational inference based approach for approximation of posterior densities. [44], [33], and lately [50] show theoretical properties of the framework under some modifications. Despite these efforts, some of the practical challenges for computer enabled predictions with GPs remain.…”

Section: Introductionmentioning

confidence: 98%

A Fast and Calibrated Computer Model Emulator: An Empirical Bayes Approach

Kejzlar,

Son,

Bhattacharya

et al. 2020

Preprint

View full text Add to dashboard Cite

Mathematical models implemented on a computer have become the driving force behind the acceleration of the cycle of scientific processes. This is because computer models are typically much faster and economical to run than physical experiments. In this work, we develop an empirical Bayes approach to predictions of physical quantities using a computer model, where we assume that the computer model under consideration needs to be calibrated and is computationally expensive. We propose a Gaussian process emulator and a Gaussian process model for the systematic discrepancy between the computer model and the underlying physical process. This allows for closed-form and easy-to-compute predictions given by a conditional distribution induced by the Gaussian processes. We provide a rigorous theoretical justification of the proposed approach by establishing posterior consistency of the estimated physical process. The computational efficiency of the methods is demonstrated in an extensive simulation study and a real data example. The newly established approach makes enhanced use of computer models both from practical and theoretical standpoints.

show abstract

“…Kennedy and O'Hagan (2001) is a renowned work on statistical framework for calibration of computer models, which also contains a review of earlier work on the subject. Related references in this field include Higdon et al (2004Higdon et al ( , 2008Higdon et al ( , 2013; Bayarri et al (2007a,b); Tuo andWu (2015, 2016); Joseph and Melkote (2009); Joseph and Yan (2015); Wong et al (2017); Plumlee (2017); Gu and Wang (2018); Plumlee (2019); Tuo (2019); Xie and Xu (2020); Wang et al (2020) and the references therein.…”

Section: Introductionmentioning

confidence: 99%

A Reproducing Kernel Hilbert Space Approach to Functional Calibration of Computer Models

Tuo¹,

He²,

Pourhabib³

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper develops a frequentist solution to the functional calibration problem, where the value of a calibration parameter in a computer model is allowed to vary with the value of control variables in the physical system. The need of functional calibration is motivated by engineering applications where using a constant calibration parameter results in a significant mismatch between outputs from the computer model and the physical experiment. Reproducing kernel Hilbert spaces (RKHS) are used to model the optimal calibration function, defined as the functional relationship between the calibration parameter and control variables that gives the best prediction. This optimal calibration function is estimated through penalized least squares with an RKHS-norm penalty and using physical data. An uncertainty quantification procedure is also developed for such estimates. Theoretical guarantees of the proposed method are provided in terms of prediction consistency and consitency of estimating the optimal calibration function.The proposed method is tested using both real and synthetic data and exhibits more robust performance in prediction and uncertainty quantification than the existing parametric functional calibration method and a state-of-art Bayesian method.

show abstract

Computer Model Calibration with Confidence and Consistency

Cited by 24 publications

References 33 publications

The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning

The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning

A Fast and Calibrated Computer Model Emulator: An Empirical Bayes Approach

A Reproducing Kernel Hilbert Space Approach to Functional Calibration of Computer Models

Contact Info

Product

Resources

About