Nonparametric Estimation of Global Functionals and a Measure of the Explanatory Power of Covariates in Regression

Doksum, Kjell A.; Samarov, Alexander

doi:10.1214/aos/1176324307

Cited by 125 publications

(70 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Let X −j ≡ X − X j denote X without the X j component. Assume that the squared Pearson nonparametric correlation (Doksum and Samarov [4], Huang and Chen [9]) between X j and X −j is less than 0.5. We call the variable X j relevant for Y if conditionally given X −j , L(Y |X j = x) is not constant in x; and irrelevant otherwise.…”

Section: Consistency Of Catchmentioning

confidence: 99%

Nonparametric variable selection and classification: The CATCH algorithm

Tang

Chen

Tsui

et al. 2014

Computational Statistics & Data Analysis

Self Cite

View full text Add to dashboard Cite

In a nonparametric framework, we consider the problem of classifying a categorical response Y whose distribution depends on a vector of predictors X, where the coordinates X j of X may be continuous, discrete, or categorical. To select the variables to be used for classification, we construct an algorithm which for each variable X j computes an importance score s j to measure the strength of association of X j with Y . The algorithm deletes X j if s j falls below a certain threshold. It is shown in Monte Carlo simulations that the algorithm has a high probability of only selecting variables associated with Y . Moreover when this variable selection rule is used for dimension reduction prior to applying classification procedures, it improves the performance of these procedures. Our approach for computing importance scores is based on root Chisquare type statistics computed for randomly selected regions (tubes) of the sample space. The size and shape of the regions are adjusted iteratively and adaptively using the data to enhance the ability of the importance score to detect local relationships between the response and the predictors. These local scores are then averaged over the tubes to form a global importance score s j for variable X j . When confounding and spurious associations are issues, the nonparametric importance score for variable X j is computed conditionally by using tubes to restrict the other variables . We call this variable selection procedure CATCH (Categorical Adaptive Tube Covariate Hunting). We establish asymptotic properties, including consistency.

show abstract

Section: Consistency Of Catchmentioning

confidence: 99%

Nonparametric variable selection and classification: The CATCH algorithm

Tang

Chen

Tsui

et al. 2014

Computational Statistics & Data Analysis

Self Cite

View full text Add to dashboard Cite

show abstract

“…are considered (Doksum and Samarov (1995), Ait-Sahalia, Bickel, and Stoker (2001)). Similar measures compare prediction errors of the form…”

Section: Bandwidths For Alternatives With Shrinking Supportmentioning

confidence: 99%

Selecting local models in multiple regression by maximizing power

Schafer

Doksum

2008

Metrika

View full text Add to dashboard Cite

This paper considers multiple regression procedures for analyzing the relationship between a response variable and a vector of d covariates in a nonparametric setting where both tuning parameters and the number of covariates need to be selected. We introduce an approach which handles the dilemma that with high dimensional data the sparsity of data in regions of the sample space makes estimation of nonparametric curves and surfaces virtually impossible. This is accomplished by abandoning the goal of trying to estimate true underlying curves and instead estimating measures of dependence that can determine important relationships between variables. These dependence measures are based on local parametric fits on subsets of the covariate space that vary in both dimension and size within each dimension. The subset which maximizes a signal to noise ratio is chosen, where the signal is a local estimate of a dependence parameter which depends on the subset dimension and size, and the noise is an estimate of the standard error (SE) of the estimated signal. This approach of choosing the window size to maximize a signal to noise ratio lifts the curse of dimensionality because for regions with sparsity of data the SE is very large. It corresponds to asymptotically maximizing the probability of correctly finding non-spurious relationships between covariates and a response or, more precisely, maximizing asymptotic power among a class of asymptotic level α t-tests indexed by subsets of the covariate space. Subsets that achieve this goal are called features. We investigate the properties of specific procedures based on the preceding ideas using asymptotic theory and Monte Carlo simulations and find that within a selected dimension, the volume of the optimally selected subset does not tend to zero as n → ∞ unless the volume of the subset of the covariate space where the response depends on the covariate vector tends to zero.

show abstract

“…For exampleτ (x) may be obtained from kernel estimation yielding an estimateθ(x). Doksum and Samarov ( [10]: 1995) and Lavergne and Vuong ( [23]: 1998) investigated nonparametric estimates to obtain a coefficient based on a decomposition of variance.…”

Section: Non-parametric Estimationmentioning

confidence: 99%

On association in regression: the coefficient of determination revisited

Linde

Tutz

2008

Statistics

View full text Add to dashboard Cite

Universal coefficients of determination are investigated which quantify the strength of the relation between a vector of dependent variables Y and a vector of independent covariates X. They are defined as measures of dependence between Y and X through θ(x), with θ(x) parameterizing the conditional distribution of Y given X = x. If θ(x) involves unknown coefficients γ the definition is conditional on γ, and in practice γ, respectively the coefficient of determination has to be estimated. The estimates of quantities we propose generalize R 2 in classical linear regression and are also related to other definitions previously suggested. Our definitions apply to generalized regression models with arbitrary link functions as well as multivariate and nonparametric regression. The definition and use of the proposed coefficients of determination is illustrated for several regression problems with simulated and real data sets.

show abstract

Nonparametric Estimation of Global Functionals and a Measure of the Explanatory Power of Covariates in Regression

Cited by 125 publications

References 37 publications

Nonparametric variable selection and classification: The CATCH algorithm

Nonparametric variable selection and classification: The CATCH algorithm

Selecting local models in multiple regression by maximizing power

On association in regression: the coefficient of determination revisited

Contact Info

Product

Resources

About