Large Margin Distribution Learning

Zhou, Zhi‐Hua

doi:10.1007/978-3-319-11656-3_1

Cited by 36 publications

(27 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, recent theoretical results disclosed that the margin distribution, rather than the minimum margin, was more crucial to the generalization performance [41]. Such a conjecture has been recently proved by Gao and Zhou [16].…”

Section: Formulationmentioning

confidence: 96%

Large Margin Distribution Learning with Cost Interval and Unlabeled Data

Zhou

2016

IEEE Trans. Knowl. Data Eng.

Self Cite

View full text Add to dashboard Cite

In many real-world applications, different types of misclassification usually suffer from different costs, but the accurate cost is often hard to be determined and usually one can only get an interval-estimation like that one type of mistake is about five to ten times more serious than the other type. On the other hand, there are usually abundant unlabeled data available, leading to great research effort about semi-supervised learning. It is noticeable that cost interval and unlabeled data usually appear simultaneously in practice tasks; however, there is rare study tackling them together. In this paper, we propose the cisLDM approach which is able to handle cost interval and exploit unlabeled data in a principled way. Rather than maximizing the minimum margin like traditional large margin classifiers, cisLDM tries to optimize the margin distribution on both labeled and unlabeled data when minimizing the worst-case totalcost and the mean total-cost simultaneously according to the cost interval. Experiments on a broad range of datasets and cost settings exhibit the impressive performance of cisLDM. In particular, cisLDM is able to reduce 47% more total-cost than standard SVM and 27% more total-cost than cost-sensitive semi-supervised SVM which assumes the true cost value is known in advance.

show abstract

Section: Formulationmentioning

confidence: 96%

Large Margin Distribution Learning with Cost Interval and Unlabeled Data

Zhou

2016

IEEE Trans. Knowl. Data Eng.

Self Cite

View full text Add to dashboard Cite

show abstract

“…According to Occam's razor [11], when a classifier was trained too complex, the performance of it would be even worse rather than better. This phenomena is called overfitting, which means that the trained model is so adaptable to the training data that it would exaggerate the slight fluctuations in the training data, leading to poor generalization performance [4], [12]. However, the AdaBoost algorithm has the superiority of resisting overfitting, which has been observed by many researches [13]- [15].…”

Section: Introductionmentioning

confidence: 99%

“…[4], [20]- [26]. However, the biggest problem of the statistical view is that these algorithms do not explain well why AdaBoost is resistant to overfitting [4], [12].…”

Section: Introductionmentioning

confidence: 99%

Feature Learning Viewpoint of Adaboost and a New Algorithm

Wang

He³

et al. 2019

IEEE Access

View full text Add to dashboard Cite

The AdaBoost algorithm has the superiority of resisting overfitting. Understanding the mysteries of this phenomena is a very fascinating fundamental theoretical problem. Many studies are devoted to explaining it from statistical view and margin theory. In this paper, we illustrate it from feature learning viewpoint, and propose the AdaBoost+SVM algorithm, which can explain the resistant to overfitting of AdaBoost directly and easily to understand. Firstly, we adopt the AdaBoost algorithm to learn the base classifiers. Then, instead of directly weighted combination the base classifiers, we regard them as features and input them to SVM classifier. With this, the new coefficient and bias can be obtained, which can be used to construct the final classifier. We explain the rationality of this and illustrate the theorem that when the dimension of these features increases, the performance of SVM would not be worse, which can explain the resistant to overfitting of AdaBoost.

show abstract

“…Several approaches have been proposed to control the margin distribution in SVM-like classifiers [17][18][19][20][21][22]. In [18], a maximum average margin classifier (MAMC) is proposed, in which instead of maximizing the minimum margin, the margin mean for the training data is maximized without slack variables.…”

Section: Introductionmentioning

confidence: 99%

“…In [18], a maximum average margin classifier (MAMC) is proposed, in which instead of maximizing the minimum margin, the margin mean for the training data is maximized without slack variables. In [21,22], in addition to maximizing the margin mean, the margin variance is minimized and the classifier is called large margin distribution machine (LDM).…”

Section: Introductionmentioning

confidence: 99%

Improving Generalization Abilities of Maximal Average Margin Classifiers

Abe

2016

Artificial Neural Networks in Pattern Recognition

View full text Add to dashboard Cite

Maximal average margin classifiers (MAMCs) maximize the average margin without constraints. Although training is fast, the generalization abilities are usually inferior to support vector machines (SVMs). To improve the generalization abilities of MAMCs, in this paper, we propose optimizing slopes and bias terms of separating hyperplanes after the coefficient vectors of the hyperplanes are obtained. The bias term is optimized so that the number of misclassifications is minimized. To optimized the slope, we introduce a weight to the average of mapped training data for one class and optimize the weight by cross-validation. To improve the generalization ability further, we propose equally constrained MAMCs and show that they reduce to least squares SVMs. Using two-class problems, we show that the generalization ability of the unconstrained MAMCs are inferior to those of the constrained MAMCs and SVMs.

show abstract

Large Margin Distribution Learning

Cited by 36 publications

References 17 publications

Large Margin Distribution Learning with Cost Interval and Unlabeled Data

Large Margin Distribution Learning with Cost Interval and Unlabeled Data

Feature Learning Viewpoint of Adaboost and a New Algorithm

Improving Generalization Abilities of Maximal Average Margin Classifiers

Contact Info

Product

Resources

About