2019
DOI: 10.48550/arxiv.1911.01544
|View full text |Cite
Preprint
|
Sign up to set email alerts
|

The generalization error of max-margin linear classifiers: High-dimensional asymptotics in the overparametrized regime

Abstract: Modern machine learning models are often so complex that they achieve vanishing classification error on the training set. Max-margin linear classifiers are among the simplest classification methods that have zero training error (with linearly separable data). Despite this simplicity, their high-dimensional behavior is not yet completely understood. We assume to be given i.i.d. data (yi, xi), i ≤ n with xi ∼ N(0, Σ) a p-dimensional Gaussian feature vector, and yi ∈ {+1, −1} a label whose distribution depends on… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
2
1
1
1

Citation Types

1
25
0

Year Published

2020
2020
2024
2024

Publication Types

Select...
4
3

Relationship

0
7

Authors

Journals

citations
Cited by 33 publications
(26 citation statements)
references
References 38 publications
1
25
0
Order By: Relevance
“…These are the regularity assumptions for the feature matrix in [33]. The same intuition also appears in the analysis of the unperturbed random kernel models, in particular, the random feature model [18]. In this paper, we suppose that the feature matrix and the activation function satisfy the regularity assumptions in [33] and conjecture that the Gaussian equivalence is valid for (ν 1 , ν 2 , .…”
Section: Gaussian Equivalence Conjecture With An Intuitive Explanationmentioning
confidence: 65%
See 3 more Smart Citations
“…These are the regularity assumptions for the feature matrix in [33]. The same intuition also appears in the analysis of the unperturbed random kernel models, in particular, the random feature model [18]. In this paper, we suppose that the feature matrix and the activation function satisfy the regularity assumptions in [33] and conjecture that the Gaussian equivalence is valid for (ν 1 , ν 2 , .…”
Section: Gaussian Equivalence Conjecture With An Intuitive Explanationmentioning
confidence: 65%
“…ϕ(•) is the identity function and ∆ = 0 in (3)) is precisely analyzed in [14] where the feature matrix is Gaussian. In a subsequent work, [18] uses the CGMT to accurately analyze the maximum-margin linear classifier in the overparametrized regime. The work in [15] precisely characterizes the performance of the standard formulation, i.e.…”
Section: Related Workmentioning
confidence: 99%
See 2 more Smart Citations
“…When the number of parameters in the models is excessively large, there are multiple techniques to precisely measure generalization errors. To name a few, the spectrum-based analysis [45,46,47,48,49,50,51,52], and the utilization of loss functions whose shapes are almost convex or approaches zero due to the excess parameters [53,54,55]. A disadvantage of this approach is that until now it can only deal with linear or two-layer neural network models.…”
Section: Definition 2 (Population Minimummentioning
confidence: 99%