On the Degree of Approximation by Manifolds of Finite Pseudo-Dimension

Maiorov, Vitaly; Ratsaby, Joel

doi:10.1007/s003659900108

Cited by 48 publications

(36 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Lemma 2 is implied by Corollary 2 of Mendelson and Vershinin (2003) together with Property 1 of Maiorov and Ratsaby (1999). It shows that the covering number of a bounded functional space can be also bounded properly.…”

Section: Lemmamentioning

confidence: 86%

“…Suppose that there exists a constant T such that | v ( x )| ≤ T for any v ∈ and x ∈ . Then

log N_{ε} false(V_{k}, {false‖ . false‖}_{2} false) \leq italicck log \frac{T}{ε},

where c is a positive constant and ||.|| 2 denotes the function L 2 norm . Lemma 2 is implied by Corollary 2 of Mendelson and Vershinin (2003) together with Property 1 of Maiorov and Ratsaby (1999). It shows that the covering number of a bounded functional space can be also bounded properly.…”

mentioning

confidence: 86%

See 1 more Smart Citation

Prediction-based Termination Rule for Greedy Learning with Massive Data

Lin

Fang

et al. 2016

STAT SINICA

View full text Add to dashboard Cite

The appearance of massive data has become increasingly common in contemporary scientific research. When sample size n is huge, classical learning methods become computationally costly for the regression purpose. Recently, the orthogonal greedy algorithm (OGA) has been revitalized as an efficient alternative in the context of kernel-based statistical learning. In a learning problem, accurate and fast prediction is often of interest. This makes an appropriate termination crucial for OGA. In this paper, we propose a new termination rule for OGA via investigating its predictive performance. The proposed rule is conceptually simple and convenient for implementation, which suggests an Ofalse(n/lognfalse) number of essential updates in an OGA process. It therefore provides an appealing route to conduct efficient learning for massive data. With a sample dependent kernel dictionary, we show that the proposed method is strongly consistent with an Ofalse(logn/nfalse) convergence rate to the oracle prediction. The promising performance of the method is supported by both simulation and real data examples.

show abstract

Section: Lemmamentioning

confidence: 86%

“…Suppose that there exists a constant T such that | v ( x )| ≤ T for any v ∈ and x ∈ . Then

log N_{ε} false(V_{k}, {false‖ . false‖}_{2} false) \leq italicck log \frac{T}{ε},

mentioning

confidence: 86%

Prediction-based Termination Rule for Greedy Learning with Massive Data

Lin

Fang

et al. 2016

STAT SINICA

View full text Add to dashboard Cite

show abstract

“…We halt the discussion about ρ d and refer the interested reader to [23] where we estimate it for a standard Sobolev class W r, l p , 1 ≤ p, q ≤ ∞.…”

Section: A New Nonlinear Approximation Widthmentioning

confidence: 99%

On the Value of Partial Information for Learning from Examples

Maiorov

1997

Journal of Complexity

View full text Add to dashboard Cite

The PAC model of learning and its extension to real valued function classes provides a well-accepted theoretical framework for representing the problem of learning a target function g(x) using a random sample {(Based on the uniform strong law of large numbers the PAC model establishes the sample complexity, i.e., the sample size m which is sufficient for accurately estimating the target function to within high confidence. Often, in addition to a random sample, some form of prior knowledge is available about the target. It is intuitive that increasing the amount of information should have the same effect on the error as increasing the sample size. But quantitatively how does the rate of error with respect to increasing information compare to the rate of error with increasing sample size? To answer this we consider a new approach based on a combination of information-based complexity of Traub et al. and Vapnik-Chervonenkis (VC) theory. In contrast to VC-theory where function classes of finite pseudo-dimension are used only for statistical-based estimation, we let such classes play a dual role of functional estimation as well as approximation. This is captured in a newly introduced quantity, ρ d (F ), which represents a nonlinear width of a function class F. We then extend the notion of the nth minimal radius of information and define a quantity I n, d (F ) which measures the minimal approximation error of the worst-case target g ∈ F by the family of function classes having pseudo-dimension d given partial information on g consisting of values taken by n linear operators. The error rates are calculated which leads to a quantitative notion of the value of partial information for the paradigm of learning from examples.

show abstract

“…To obtain the lower bounds of approximation, we use estimates of the number of connected components of polynomial manifolds. These estimates are related to the problem of calculation of the pseudo-dimension of manifolds, which is widely studied within the neural network community (see [25,27,8,12,23]). …”

Section: Some Estimates In the Finite-dimensional Spacementioning

confidence: 99%