On Nonparametric Residual Variance Estimation

Liitiäinen, Elia; Corona, Francesco; Lendasse, Amaury

doi:10.1007/s11063-008-9087-8

Cited by 36 publications

(12 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Delta Test was introduced by Pi and Peterson for time series [42] and recently further analyzed by Liitiäinen et al [43]. However, its applicability to variable selection was proposed in [35].…”

Section: Delta Testmentioning

confidence: 99%

Approximate k-NN delta test minimization method using genetic algorithms: Application to time series

2010

View full text Add to dashboard Cite

In many real world problems, the existence of irrelevant input variables (features) hinders the predictive quality of the models used to estimate the output variables. In particular, time series prediction often involves building large regressors of artificial variables that can contain irrelevant or misleading information. Many techniques have arisen to confront the problem of accurate variable selection, including both local and global search strategies. This paper presents a method based on genetic algorithms that intends to find a global optimum set of input variables that minimize the Delta Test criterion. The execution speed has been enhanced by substituting the exact nearest neighbor computation by its approximate version. The problems of scaling and projection of variables have been addressed. The developed method works in conjunction with MATLAB's Genetic Algorithm and Direct Search Toolbox. The goodness of the proposed methodology has been evaluated on several popular time series examples, and also generalized to other non-time-series datasets.

show abstract

Section: Delta Testmentioning

confidence: 99%

Approximate k-NN delta test minimization method using genetic algorithms: Application to time series

2010

View full text Add to dashboard Cite

show abstract

“…While in practice this might not always be a problem, it is of interest to have a method with better properties in this sense. Here we discuss the method in [12] defined by the formulâ…”

Section: The Modified 1-nn Estimatormentioning

confidence: 99%

“…Some references in machine learning include [5,14]; however, these works make the restrictive homoscedasticity assumption on the noise. This shortcoming has been addressed in [12], where a practical estimator with good convergence properties is derived.…”

Section: Introductionmentioning

confidence: 99%

Residual variance estimation in machine learning

et al. 2009

Self Cite

View full text Add to dashboard Cite

The problem of residual variance estimation consists of estimating the best possible generalization error obtainable by any model based on a finite sample of data. Even though it is a natural generalization of linear correlation, residual variance estimation in its general form has attracted relatively little attention in machine learning.In this paper, we examine four different residual variance estimators and analyze their properties both theoretically and experimentally to understand better their applicability in machine learning problems. The theoretical treatment differs from previous work by being based on a general formulation of the problem covering also heteroscedastic noise in contrary to previous work, which concentrates on homoscedastic and additive noise.In the second part of the paper, we demonstrate practical applications in input and model structure selection. The experimental results show that using residual variance estimators in these tasks gives good results often with a reduced computational complexity, while the nearest neighbor estimators are simple and easy to implement.

show abstract

“…One of the most successful criteria to determine the optimal set of variables in regression applications is a nonparametric noise estimator called Delta Test (DT) ( [2], [3]). …”

Section: Introductionmentioning

confidence: 99%

RCGA-S/RCGA-SP Methods to Minimize the Delta Test for Regression Tasks

Mateo

Sovilj

Gadea

et al. 2009

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. Frequently, the number of input variables (features) involved in a problem becomes too large to be easily handled by conventional machine-learning models. This paper introduces a combined strategy that uses a real-coded genetic algorithm to find the optimal scaling (RCGA-S) or scaling + projection (RCGA-SP) factors that minimize the Delta Test criterion for variable selection when being applied to the input variables. These two methods are evaluated on five different regression datasets and their results are compared. The results confirm the goodness of both methods although RCGA-SP performs clearly better than RCGA-S because it adds the possibility of projecting the input variables onto a lower dimensional space.

show abstract

On Nonparametric Residual Variance Estimation

Cited by 36 publications

References 19 publications

Approximate k-NN delta test minimization method using genetic algorithms: Application to time series

Approximate k-NN delta test minimization method using genetic algorithms: Application to time series

Residual variance estimation in machine learning

RCGA-S/RCGA-SP Methods to Minimize the Delta Test for Regression Tasks

Contact Info

Product

Resources

About