Optimal Model Averaging of Varying Coefficient Models

Liu, Cong; Li, Qi; Racine, Jeffrey S.; Zhang, Daiqiang

doi:10.5705/ss.202017.0034

Cited by 8 publications

(3 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Semi-/non-parametric averaging was first proposed by Liu (2018) in a non-optimal framework. Li et al (2018) and Zhang and Wang (2019) then studied optimal model averaging in varying-coefficient and partially linear models, respectively. Recently developments on optimal semi-/non-parametric model averaging include Zhu et al (2019), Racine et al (2022) and Zhu et al (2023) in different contexts.…”

Section: Introductionmentioning

confidence: 99%

Optimal Model Averaging for Single-Index Models With Divergent Dimensions

Zou¹,

Wang²,

Zhang³

et al. 2025

STAT SINICA

View full text Add to dashboard Cite

This paper offers a new approach to address the model uncertainty in (potentially) divergent-dimensional single-index models (SIMs). We propose a model-averaging estimator based on cross-validation, which allows the dimension of covariates and the number of candidate models to increase with the sample size. We show that when all candidate models are misspecified, our model-averaging estimator is asymptotically optimal with its squared loss asymptotically identical to that of the infeasible best possible averaging estimator. In a different situation where correct models are available in the model set, the proposed method assigns all weights to the correct models asymptotically.We also propose averaging regularized estimators and prescreening methods to deal with high-dimensional covariates. We illustrate the method via simulations and two empirical applications.

show abstract

Section: Introductionmentioning

confidence: 99%

Optimal Model Averaging for Single-Index Models With Divergent Dimensions

Zou¹,

Wang²,

Zhang³

et al. 2025

STAT SINICA

View full text Add to dashboard Cite

show abstract

“…Zhu et al (2019) proposed a weight choice criterion in a PLM with varying coefficients. Other studies of FMA on the weight choice in nonparametric and semiparametric models include Gao (2015), Chen et al (2018a), Li et al (2018), among others.…”

Section: Introductionmentioning

confidence: 99%

Frequentist Model Averaging for the Nonparametric Additive Model

Liao¹,

Wan²,

He³

et al. 2023

STAT SINICA

View full text Add to dashboard Cite

This paper develops an optimal frequentist model averaging approach for estimating the unknown conditional mean function in the nonparametric additive model when the covariates and the degree of smoothing are subject to uncertainty. Our weight choice criterion selects model weights by minimising a plug-in estimator of the risk of the model average estimator under a squared error loss function. We derive the convergence rate of the model weights obtained from our proposed method to the infeasible optimal weights, and prove that the resultant model average estimators are asymptotically optimal. An extension to the additive autoregressive model for time series data is also considered. Our simulation analysis shows that the proposed model average estimators can significantly outperform several commonly used model selection estimators and their model averaging counterparts in terms of mean squared error in a large part of the parameter space. We further illustrate our methods in two real data studies.

show abstract

“…In the framework of parametric models, we have, for example, Mallows model averaging (MMA; Hansen, 2007), optimal mean squared error averaging (Liang et al, 2011), jackknife model averaging (JMA; Hansen and Racine, 2012), heteroskedasticity-robust C p (Liu and Okui, 2013), optimal model averaging for linear mixed-effects models (Zhang, Zou, and Liang, 2014), multinomial and ordered logit models (Wan, Zhang, and Wang, 2014), Kullback-Leibler model averaging (Zhang, Zou, and Carroll, 2015), optimal model averaging for generalized linear models and generalized linear mixed-effects models (Zhang et al, 2016), model averaging for covariance matrix estimation (Zheng et al, 2017), and model averaging for high-dimensional data Li, 2014, 2017;Zhang et al, 2020). There are also many semiparametric model averaging methods (see, for example, Zhang and Liang, 2011;Li, Linton, and Lu, 2015;Kitagawa and Muris, 2016;Li et al, 2018aLi et al, , 2018bZhang and Wang, 2019;Zhu et al, 2019;Fang, Li, and Xia, 2020).…”

Section: Introductionmentioning

confidence: 99%

An Asymptotic Theory for Least Squares Model Averaging With Nested Models

2022

View full text Add to dashboard Cite

Theoretical results of frequentist model averaging mainly focus on asymptotic optimality and asymptotic distribution of the model averaging estimator. However, even for basic least squares model averaging, many theoretical problems have not been well addressed yet. This article discusses asymptotic properties of a class of least squares model averaging methods with nested candidate models that includes the Mallows model averaging (MMA) of Hansen (2007, Econometrica 75, 1175–1189) as a special case. Two scenarios are considered: (i) all candidate models are under-fitted; and (ii) the true model is included in the candidate models. We find that in the first scenario, the least squares model averaging method asymptotically assigns weight one to the largest candidate model and the resulting model averaging estimator is asymptotically normal. In the second scenario with a slightly special weight space, if the penalty factor in the weight selection criterion is diverging with certain order, the model averaging estimator is asymptotically optimal by putting weight one to the true model. However, MMA with fixed model dimensions is not asymptotically optimal since it puts nonnegligible weights to over-fitted models. The theoretical results are clearly summarized with their restrictions, and some critical implications are discussed. Monte Carlo simulations confirm our theoretical results.

show abstract

Optimal Model Averaging of Varying Coefficient Models

Cited by 8 publications

References 33 publications

Optimal Model Averaging for Single-Index Models With Divergent Dimensions

Optimal Model Averaging for Single-Index Models With Divergent Dimensions

Frequentist Model Averaging for the Nonparametric Additive Model

An Asymptotic Theory for Least Squares Model Averaging With Nested Models

Contact Info

Product

Resources

About