Model-Averaged Profile Likelihood Intervals

Fletcher, David; Turek, Daniel

doi:10.1007/s13253-011-0064-8

Cited by 27 publications

(47 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This assumption may in part explain the superior performance of AIC model weights, since AIC is known to favor increased model complexity [39]. However we do not consider this an issue, since results from Fletcher and Turek [16] indicate that intervals using AIC weights perform at least as well as those using other information criteria when the most complex model is not the generating model. Furthermore, all simulations presented herein were repeated using data generated under the simpler of the two candidate models ( 1 ).…”

Section: Discussionmentioning

confidence: 99%

“…The frequentist MATA intervals are based upon model averaging the error rates of single-model intervals, rather than constructing an interval around a model-averaged estimator. This construction is analogous to Bayesian model averaging, and the idea was initially motivated using an analogy to a model-averaged Bayesian interval [16]. The MATA construction was studied further in Turek and Fletcher's work [15], where it is shown that, asymptotically, a MATA interval will converge to the single-model interval based upon the candidate model with minimum Kullback-Leibler distance to the true, generating model.…”

Section: Discussionmentioning

confidence: 99%

“…where ( ) is defined in terms of the corresponding likelihood function ( , ), as in (8), and as described by Fletcher and Turek [16]. …”

Section: Mata Profile Likelihoodmentioning

confidence: 99%

“…In a simulation study by Turek and Fletcher [15], the MATA-Wald interval outperformed model-averaged intervals of the form̂±ŝe(̂). Fletcher and Turek [16] applied the MATA construction to profile likelihood intervals to produce a model-averaged tail area profile likelihood (MATA-PL) interval. Coverage properties of MATA confidence intervals are also studied in Kabaila et al [17], and a transformed version of the MATA interval was proposed by Yu et al [18].…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we compare the performance of modelaveraged Bayesian credible intervals and the MATA-Wald and MATA-PL intervals of Turek and Fletcher [15] and Fletcher and Turek [16]. The effect of using various model prior probabilities and parameter prior distributions on Bayesian intervals is considered.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Comparison of the Frequentist MATA Confidence Interval with Bayesian Model-Averaged Confidence Intervals

Turek

2015

Journal of Probability and Statistics

Self Cite

View full text Add to dashboard Cite

Model averaging is a technique used to account for model uncertainty, in both Bayesian and frequentist multimodel inferences. In this paper, we compare the performance of model-averaged Bayesian credible intervals and frequentist confidence intervals. Frequentist intervals are constructed according to the model-averaged tail area (MATA) methodology. Differences between the Bayesian and frequentist methods are illustrated through an example involving cloud seeding. The coverage performance and interval width of each technique are then studied using simulation. A frequentist MATA interval performs best in the normal linear setting, while Bayesian credible intervals yield the best coverage performance in a lognormal setting. The use of a data-dependent prior probability for models improved the coverage of the model-averaged Bayesian interval, relative to that using uniform model prior probabilities. Data-dependent model prior probabilities are philosophically controversial in Bayesian statistics, and our results suggest that their use is beneficial when model averaging.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

“…where ( ) is defined in terms of the corresponding likelihood function ( , ), as in (8), and as described by Fletcher and Turek [16]. …”

Section: Mata Profile Likelihoodmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Comparison of the Frequentist MATA Confidence Interval with Bayesian Model-Averaged Confidence Intervals

Turek

2015

Journal of Probability and Statistics

Self Cite

View full text Add to dashboard Cite

show abstract

On the minimum coverage probability of model averaged tail area confidence intervals

Kabaila

2018

Can J Statistics

View full text Add to dashboard Cite

Key words and phrases: Model averaged confidence intervals; MATA confidence interval; minimum coverage probability. MSC 2010: Primary 62F25; secondary 62P12 Abstract: Frequentist model averaging has been proposed as a method for incorporating "model uncertainty" into confidence interval construction. Such proposals have been of particular interest in the environmental and ecological statistics communities. A promising method of this type is the model averaged tail area (MATA) confidence interval put forward by Turek & Fletcher, 2012. The performance of this interval depends greatly on the data-based model weights on which it is based. A computationally convenient formula for the coverage probability of this interval is provided by Kabaila, Welsh and Abeysekera, 2016, in the simple scenario of two nested linear regression models. We consider the more complicated scenario that there are many (32,768 in the example considered) linear regression models obtained as follows. For each of a specified set of components of the regression parameter vector, we either set the component to zero or let it vary freely. We provide an easily-computed upper bound on the minimum coverage probability of the MATA confidence interval. This upper bound provides evidence against the use of a model weight based on the Bayesian Information Criterion (BIC).

show abstract

Model averaging in ecology: a review of Bayesian, information‐theoretic, and tactical approaches for predictive inference

Dormann

Calabrese

Guillera‐Arroita

et al. 2018

Ecological Monographs

289

205

View full text Add to dashboard Cite

In ecology, the true causal structure for a given problem is often not known, and several plausible models and thus model predictions exist. It has been claimed that using weighted averages of these models can reduce prediction error, as well as better reflect model selection uncertainty. These claims, however, are often demonstrated by isolated examples. Analysts must better understand under which conditions model averaging can improve predictions and their uncertainty estimates. Moreover, a large range of different model averaging methods exists, raising the question of how they differ in their behaviour and performance. Here, we review the mathematical foundations of model averaging along with the diversity of approaches available. We explain that the error in model‐averaged predictions depends on each model's predictive bias and variance, as well as the covariance in predictions between models, and uncertainty about model weights. We show that model averaging is particularly useful if the predictive error of contributing model predictions is dominated by variance, and if the covariance between models is low. For noisy data, which predominate in ecology, these conditions will often be met. Many different methods to derive averaging weights exist, from Bayesian over information‐theoretical to cross‐validation optimized and resampling approaches. A general recommendation is difficult, because the performance of methods is often context dependent. Importantly, estimating weights creates some additional uncertainty. As a result, estimated model weights may not always outperform arbitrary fixed weights, such as equal weights for all models. When averaging a set of models with many inadequate models, however, estimating model weights will typically be superior to equal weights. We also investigate the quality of the confidence intervals calculated for model‐averaged predictions, showing that they differ greatly in behaviour and seldom manage to achieve nominal coverage. Our overall recommendations stress the importance of non‐parametric methods such as cross‐validation for a reliable uncertainty quantification of model‐averaged predictions.

show abstract

Model-Averaged Profile Likelihood Intervals

Cited by 27 publications

References 14 publications

Comparison of the Frequentist MATA Confidence Interval with Bayesian Model-Averaged Confidence Intervals

Comparison of the Frequentist MATA Confidence Interval with Bayesian Model-Averaged Confidence Intervals

On the minimum coverage probability of model averaged tail area confidence intervals

Model averaging in ecology: a review of Bayesian, information‐theoretic, and tactical approaches for predictive inference

Contact Info

Product

Resources

About