Model selection and prediction: Normal regression

Abstract

A b s t r a c t . This paper discusses the topic of model selection for finitedimensional normal regression models. We compare model selection criteria according to prediction errors based upon prediction with refitting, and prediction without refitting. We provide a new lower bound for prediction without refitting, while a lower bound for prediction with refitting was given by Rissanen. Moreover, we specify a set of sufficient conditions for a model selection criterion to achieve these bounds. Then the achievability of the two bounds by the following selection rules are addressed: Rissanen's accumulated prediction error criterion (APE), his stochastic complexity criterion, AIC, BIC and the FPE criteria. In particular, we provide upper bounds on overfitting and underfitting probabilities needed for the achievability. Finally, we offer a brief discussion on the issue of finite-dimensional vs. infinite-dimensional model assumptions.

Variable Selection in Predictive Regressions

“…Geweke and Meese (1981) show in a stochastic regressors setup that this condition is necessary for consistent model selection. Speed and Yu (1993) also show that the BIC with C T = log T is desirable for prediction. Asymptotic efficiency of the BIC is also shown in Shao (1997).…”

Autoregressive model based algorithm for correcting motion and serially correlated errors in fNIRS

“…The model order can be chosen objectively by minimizing an information criterion function. For example, the Bayesian information criterion (BIC) [20] function is…”

Asymptotics and Coding Theory: One of the n → ∞ Dimensions of Terry

“…The research in Speed and Yu [12] started in 1987. The paper was possibly written in 1989, with many drafts including extensive comments by David Freedman on the first draft and it was a long story regarding why it took four years to publish.…”

“…By then, it was well-known that AIC is prediction optimal and inconsistent (unless the true model is the largest model), while BIC is consistent when the true model is finite and one of the sub-regression models considered. Speed and Yu [12] addresses the prediction optimality question with refitting (causal or on-line prediction) and without refitting (batch prediction). A new lower bound on the latter was derived with sufficient achievability conditions, while a lower bound on the former had been given by Rissanen [8].…”

“…Related trade-offs are seen in analysis of modern model selection (sparse modeling) methods such as Lasso under high-dimensional regression models (large p large n). In fact, Speed and Yu [12] entertained the idea of a high-dimensional model through a discussion of finite dimensional models vs infinite dimensional models. In fact, much insight from this paper is still relevant today: BIC does well both in terms of consistency and prediction when the bias term drastically decreases to a lower level at a certain point (e.g.…”