Minimum complexity regression estimation with weakly dependent observations

Modha, Dharmendra S.; Masry, E.

doi:10.1109/wits.1994.513898

Cited by 28 publications

(71 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This implies that E|W m,n | < ∞, hence we now have from Lemma A.6 of Modha & Masry (1996) and from (48) that…”

Section: Proofmentioning

confidence: 84%

“…Implicitly, while establishing the rates of convergence results in Theorem 2.1 and Corollary 2.1, we assumed that these least-squares estimators can indeed be computed. Such an assumption is the very basis for applying Vapnik's empirical risk minimization theory to neural networks, and has been widely used in the literature dealing with rates of convergence results for neural networks and other models, see, for example, Barron (1994), Barron, Birgé, & Massart (1996), Breiman (1993), Haussler (1992), Kearns (1997), Lugosi & Nobel (1995), Lugosi & Zeger (1996), McCaffrey & Gallant (1994), Modha & Masry (1996, 1998, Vapnik (1982Vapnik ( , 1995, and White (1989).…”

Section: Remark 23 (Computational Complexity Of Nonlinear Least-squamentioning

confidence: 99%

“…Following Vapnik (1982Vapnik ( , 1995, exponential probability bounds derived from Bernstein and Hoeffding inequalities (Hoeffding, 1963) have become a standard tool in obtaining rates of convergence results for nonparametric estimators, see, for example, Barron (1991), Barron, Birgé, & Massart (1996), Haussler (1992), Lugosi & Nobel (1995), McCaffrey & Gallant (1994), and Modha & Masry (1996, 1997. Here, we utilize one such exponential probability bound (see (42)) derived in Barron, Birgé, & Massart (1996)-referred to as BBM hereafter.…”

Section: Abstract Estimation Framework and Derivationsmentioning

confidence: 99%

“…3. The technical condition in Assumption 4.4 allows us to employ a simple probability inequality in Lemma A.6 of Modha and Masry (1996) to derive the bounds in (49). 4.…”

Section: Abstract Estimation Framework and Derivationsmentioning

confidence: 99%

“…Roughly speaking, a model with a larger dimension has a smaller approximation error but a larger estimation error, whereas a model with a smaller dimension has a smaller estimation error but a larger approximation error. The problem of model selection is to empirically select the finite-dimensional parametric model (from the permissible collection of models {S m } m ∈ M n ) that achieves the best tradeoff between the competing approximation error and estimation error components-and, consequently, achieves the smallest possible statistical risk in estimating s. For previous theoretical work on model selection in the context of nonparametric regression estimation, see, for example, Barron (1991Barron ( , 1994, Barron, Birgé, & Massart (1996), Birgé & Massart (1994b), Baum & Haussler (1989), Haussler (1992), Lugosi & Nobel (1995), Lugosi & Zeger (1996), McCaffrey & Gallant (1994), Modha & Masry (1996, 1998, Rissanen (1989), Shen & Wong (1994), Vapnik (1982Vapnik ( , 1995, and White (1989).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Untitled

Modha

Masry

1998

Machine Learning

View full text Add to dashboard Cite

Abstract.Prequential model selection and delete-one cross-validation are data-driven methodologies for choosing between rival models on the basis of their predictive abilities. For a given set of observations, the predictive ability of a model is measured by the model's accumulated prediction error and by the model's average-out-ofsample prediction error, respectively, for prequential model selection and for cross-validation. In this paper, given i.i.d. observations, we propose nonparametric regression estimators-based on neural networks-that select the number of "hidden units" (or "neurons") using either prequential model selection or delete-one cross-validation. As our main contributions: (i) we establish rates of convergence for the integrated mean-squared errors in estimating the regression function using "off-line" or "batch" versions of the proposed estimators and (ii) we establish rates of convergence for the time-averaged expected prediction errors in using "on-line" versions of the proposed estimators. We also present computer simulations (i) empirically validating the proposed estimators and (ii) empirically comparing the proposed estimators with certain novel prequential and cross-validated "mixture" regression estimators.

show abstract

“…This implies that E|W m,n | < ∞, hence we now have from Lemma A.6 of Modha & Masry (1996) and from (48) that…”

Section: Proofmentioning

confidence: 84%

Section: Remark 23 (Computational Complexity Of Nonlinear Least-squamentioning

confidence: 99%

Section: Abstract Estimation Framework and Derivationsmentioning

confidence: 99%

“…3. The technical condition in Assumption 4.4 allows us to employ a simple probability inequality in Lemma A.6 of Modha and Masry (1996) to derive the bounds in (49). 4.…”

Section: Abstract Estimation Framework and Derivationsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Untitled

Modha

Masry

1998

Machine Learning

View full text Add to dashboard Cite

show abstract

PAC learning in non-linear FIR models

Najarian

Dumont

Davies

et al. 2001

Int. J. Adapt. Control Signal Process.

View full text Add to dashboard Cite

The PAC learning theory creates a framework to assess the learning properties of static models for which the data are assumed to be independently and identically distributed (i.i.d.). The present paper first extends the idea of PAC learning to cover the learning of modelling tasks with m‐dependent data, and then applies the resulting framework to evaluate learning of non‐linear FIR models. Also, the learning properties of FIR modelling with radial basis function networks are further specified. These results include an upper bound on the size of the data set required to train an FIR radial basis function network, provided that the input data are uniformly distributed. Copyright © 2001 John Wiley & Sons, Ltd.

show abstract

On learning of Sigmoid Neural Networks

Najarian

2001

Complexity

View full text Add to dashboard Cite

The Probably Approximately Correct (PAC) learning theory creates a framework to assess the learning properties of static models for which the data are assumed to be independently and identically distributed (i.i.d.). One important family of dynamic models to which the conventional PAC learning can not be applied is nonlinear Finite Impulse Response (FIR) models. The present article, using an extension of PAC learning that covers learning with m-dependent data, the learning properties of FIR modeling with sigmoid neural networks are evaluated. These results include upper bounds on the size of the data set required to train FIR sigmoid neural networks, provided that the input data are uniformly distributed. ᭧

show abstract

Minimum complexity regression estimation with weakly dependent observations

Cited by 28 publications

References 22 publications

Untitled

Untitled

PAC learning in non-linear FIR models

On learning of Sigmoid Neural Networks

Contact Info

Product

Resources

About