Economic variable selection

Abstract

Regression plays a key role in many research areas and its variable selection is a classic and major problem. This study emphasizes cost of predictors to be purchased for future use, when we select a subset of them. Its economic aspect is naturally formalized by the decision-theoretic approach. In addition, two Bayesian approaches are proposed to address uncertainty about model parameters and models: the restricted and extended approaches, which lead us to rethink about model averaging. From objective, rule-based, or robust Bayes point of view, the former is preferred. Proposed method is applied to three popular datasets for illustration.

On the two-dataset problem

“…The maximum marginal likelihood criterion indicates that x 2 , x 3 , and x 9 are common across datasets (see Section 2). Miyawaki and MacEachern (2019) report that x 3 , x 5 , and x 9 give the least mean squared loss when using the same model across all observations.…”

“…Another example is the diabetes data analyzed by Efron et al (2004), including diabetes progression measure (Y) as the response and ten predictors (age, sex, body mass index (BMI), average blood pressure (BP), and six blood serum measurements (S1 -S6)). This dataset is used to illustrate variable selection methods (e.g., Efron et al (2004), Hahn and Carvalho (2015), Miyawaki and MacEachern (2019) among others). A close look suggests that the data may come from at least two sources, because the precision of the blood pressure and S4 (the fourth blood serum measurement) is different from that of the other measurements (see ID 95 and 99 in Table 1).…”

“…The remaining parameters are assumed to follow noninformative prior distributions: the prior for the intercept is proportional to a constant, and the priors for the variances of the regression errors are π(σ 2 j ) ∝ σ −2 j (j = 0, 1). With this prior specification, these posterior means are analytically tractable (see, e.g., Miyawaki and MacEachern (2019)).…”

“…A smaller prediction error in the former is an indication that the predictor is common across datasets and a support for the assumption. When several specifications share predictors of interest (or several specifications do not share them), a possible approach is to take average possible models, which is reasonable under the squared loss function (see also Miyawaki and MacEachern (2019)).…”

“…The first evaluates the internal quality of the data. Furthermore, we investigate the heterogeneity/similarity among observations with the second error, which is addressed within the framework of Bayesian model averaging (see, e.g., Raftery et al (1997); Brown et al (2002); Steel (2019); Miyawaki and MacEachern (2019)).…”