A general framework for frequentist model averaging

Abstract

Model selection strategies have been routinely employed to determine a model for data analysis in statistic, and further study and inference then often proceed as though the selected model were the true model that were known a priori. This practice does not account for the uncertainty introduced by the selection process and the fact that the selected model can possibly be a wrong one. Model averaging approaches try to remedy this issue by combining estimators for a set of candidate models. Specifically, instead of deciding which model is the 'right' one, a model averaging approach suggests to fit a set of candidate models and average over the estimators using certain data adaptive weights. In this paper we establish a general frequentist model averaging framework that does not set any restrictions on the set of candidate models. It greatly broadens the scope of the existing methodologies under the frequentist model averaging development. Assuming the data is from an unknown model, we derive the model averaging estimator and study its limiting distributions and related predictions while taking possible modeling biases into account. We propose a set of optimal weights to combine the individual estimators so that the expected mean squared error of the average estimator is minimized. Simulation studies are conducted to compare the performance of the estimator with that of the existing methods. The results show the benefits of the proposed approach over traditional model selection approaches as well as existing model averaging methods.

Solving a Typical Small Sample Size MRSM Dataset Problem Using a Flexible Hybrid Ensemble Approach for Credibility

“…Over the years, many MS criteria have been developed resulting in the problem of MS criterion uncertainty as the criteria often make different choices [8]. Moreover, MS has been criticised for its failure to account for dataset and model uncertainty, model bias and loss of information contained in discarded response models resulting in overly specified prediction accuracy, and understatement of variance [3,9,10]. These problems compound to solution uncertainty which leaves a question on the credibility of the final result.…”

Combining estimators in interlaboratory studies and meta‐analyses

“…25,26 A similar approach known as "model averaging" becomes an important tool to deal with model uncertainty. 27 Liang et al 28 stated, "The main benefit of model averaging over model selection is that it incorporates rather than ignores the uncertainty inherent in the model selection process." Model averaging is used in many application areas in recent years, for example, estimating causal effects in high-dimensional scenarios 23 and estimating a high-dimensional covariance matrix with a network structure.…”

Adaptive optimal stimulus selection in cognitive models using a model averaging approach

“…Bayesian MA has been applied in many fields such as economics, medical science, ecology, and psychology (Fragoso et al 2018;Hinne et al 2020;Hoeting et al 1999;Wasserman 2000). Recently, there has been a growth in literature on frequentist MA as an efficient and theoretically grounded framework to construct an estimator (Mitra et al 2019;Moral-Benito 2013;Wang et al 2009).…”