A precise high-dimensional asymptotic theory for boosting and minimum-ℓ1-norm interpolated classifiers

Liang, Tengyuan; Sur, Pragya

doi:10.1214/22-aos2170

Cited by 14 publications

(1 citation statement)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These classifiers are usually used for classification in two or more classes and are based on various principles. For example, linear combination (Orozco-Alzate et al 2019), use of mutual distances (Shahid and Singh 2019;Liu and Wang 2022;Veneri et al 2022), variants of gradient boosting (Bentejac et al 2021;Liang and Sur 2022), adaboost (Hu et al 2020), and bagging (Medina-Pérez et al 2017;Jafarzadeh et al 2021).…”

Section: Introductionmentioning

confidence: 99%

Concept of hidden classes in pattern classification

Hrebik

Kukal

2023

Artif Intell Rev

View full text Add to dashboard Cite

Our paper presents a novel approach to pattern classification. The general disadvantage of a traditional classifier is in too different behaviour and optimal parameter settings during training on a given pattern set and the following cross-validation. We describe the term critical sensitivity, which means the lowest reached sensitivity for an individual class. This approach ensures a uniform classification quality for individual class classification. Therefore, it prevents outlier classes with terrible results. We focus on the evaluation of critical sensitivity, as a quality criterion. Our proposed classifier eliminates this disadvantage in many cases. Our aim is to present that easily formed hidden classes can significantly contribute to improving the quality of a classifier. Therefore, we decided to propose classifier will have a relatively simple structure. The proposed classifier structure consists of three layers. The first is linear, used for dimensionality reduction. The second layer serves for clustering and forms hidden classes. The third one is the output layer for optimal cluster unioning. For verification of the proposed system results, we use standard datasets. Cross-validation performed on standard datasets showed that our critical sensitivity-based classifier provides comparable sensitivity to reference classifiers.

show abstract

Section: Introductionmentioning

confidence: 99%

Concept of hidden classes in pattern classification

Hrebik

Kukal

2023

Artif Intell Rev

View full text Add to dashboard Cite

show abstract

Equivalence of state equations from different methods in high-dimensional regression

Luo,

Tian

2024

Communications in Statistics - Theory and Methods

View full text Add to dashboard Cite

Deterministic equivalent and error universality of deep random features learning^*

Schröder,

Cui,

Dmitriev

et al. 2024

J. Stat. Mech.

View full text Add to dashboard Cite

This manuscript considers the problem of learning a random Gaussian network function using a fully connected network with frozen intermediate layers and trainable readout layer. This problem can be seen as a natural generalization of the widely studied random features model to deeper architectures. First, we prove Gaussian universality of the test error in a ridge regression setting where the learner and target networks share the same intermediate layers, and provide a sharp asymptotic formula for it. Establishing this result requires proving a deterministic equivalent for traces of the deep random features sample covariance matrices which can be of independent interest. Second, we conjecture the asymptotic Gaussian universality of the test error in the more general setting of arbitrary convex losses and generic learner/target architectures. We provide extensive numerical evidence for this conjecture. In light of our results, we investigate the interplay between architecture design and implicit regularization.

show abstract

A precise high-dimensional asymptotic theory for boosting and minimum-ℓ1-norm interpolated classifiers

Cited by 14 publications

References 61 publications

Concept of hidden classes in pattern classification

Concept of hidden classes in pattern classification

Equivalence of state equations from different methods in high-dimensional regression

Deterministic equivalent and error universality of deep random features learning^*

Contact Info

Product

Resources

About

A precise high-dimensional asymptotic theory for boosting and minimum-ℓ1-norm interpolated classifiers

Cited by 14 publications

References 61 publications

Concept of hidden classes in pattern classification

Concept of hidden classes in pattern classification

Equivalence of state equations from different methods in high-dimensional regression

Deterministic equivalent and error universality of deep random features learning*

Contact Info

Product

Resources

About

Deterministic equivalent and error universality of deep random features learning^*