2021
DOI: 10.1088/2632-2153/abfbbb
|View full text |Cite
|
Sign up to set email alerts
|

Large deviations in the perceptron model and consequences for active learning

Abstract: Active learning (AL) is a branch of machine learning that deals with problems where unlabeled data is abundant yet obtaining labels is expensive. The learning algorithm has the possibility of querying a limited number of samples to obtain the corresponding labels, subsequently used for supervised learning. In this work, we consider the task of choosing the subset of samples to be labeled from a fixed finite pool of samples. We assume the pool of samples to be a random matrix and the ground truth labels to be g… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
2
2
1

Citation Types

0
5
0

Year Published

2021
2021
2024
2024

Publication Types

Select...
3
3

Relationship

0
6

Authors

Journals

citations
Cited by 6 publications
(5 citation statements)
references
References 18 publications
0
5
0
Order By: Relevance
“…Such perceptrons have also been analyzed in an active learning setting where the learner is free to design any new input to be labeled [27,28], rather than choose from a fixed set of inputs, as in data-pruning. Recent work [29] has analyzed this scenario but focused on message passing algorithms that are tailored to the case of Gaussian inputs and perceptrons, and are hard to generalize to real world settings. In contrast we analyze margin based pruning algorithms that are used in practice in diverse settings, as in [8,9].…”
Section: Statistical Mechanics Of Perceptron Learningmentioning
confidence: 99%
“…Such perceptrons have also been analyzed in an active learning setting where the learner is free to design any new input to be labeled [27,28], rather than choose from a fixed set of inputs, as in data-pruning. Recent work [29] has analyzed this scenario but focused on message passing algorithms that are tailored to the case of Gaussian inputs and perceptrons, and are hard to generalize to real world settings. In contrast we analyze margin based pruning algorithms that are used in practice in diverse settings, as in [8,9].…”
Section: Statistical Mechanics Of Perceptron Learningmentioning
confidence: 99%
“…In this work we study the impact of curriculum using the analytically tractable teacher-student framework and the tools of statistical physics [19,20,21,22]. High-dimensional teacher-student models are a popular approach for systematically studying learning behaviour in neural networks [23,24,20], and have recently been leveraged to analyse a variety of phenomena [25,26,27,28,29,30].…”
Section: Introductionmentioning
confidence: 99%
“…Again, we see that negative values of q are more effective in increasing the minimum energy gap of this system. In this section, we consider the binary perceptron problem to check the algorithm's performance in a computationally challenging problem [24,[60][61][62][63][64]. In a simple binary perceptron we have N artificial neurons of states ξ = {ξ i = ±1 : i = 1 .…”
Section: The Mean-field P-spin Modelmentioning
confidence: 99%
“…In the following, we consider M random and independent patterns with equal probabilities for the ±1 values of the ξ a i and s a . It is known that in this case we can store up to M c 0.83N random patterns with an exponentially large number of isolated solutions σ * in the space of weights [61,62,64]. Because of the complex energy landscape of the problem, it is difficult for a classical simulated annealing algorithm to find a solution for large N as the number of patterns approaches M c .…”
Section: The Mean-field P-spin Modelmentioning
confidence: 99%