Reweighting Forest for Extreme Multi-label Classification

Lin, Z. C.; Dai, Bi-Ru

doi:10.1007/978-3-319-64283-3_21

Cited by 2 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We used the machine learning datasets shown in Table 2 . The “australian” data set is from LIBSVM Data (Lin, 2017 ). The “MNIST” data set contains handwritten “0” and “1” digits and is provided by LeCun et al ( 1998b ).…”

Section: Methodsmentioning

confidence: 99%

“…We compared the proposed algorithms with Pegasos (Shalev-Shwartz et al, 2011 ) and the SMO algorithm on various datasets (LeCun et al, 1998b ; Dheeru and Karra Taniskidou, 2017 ; Lin, 2017 ) for binary and multiclass classification. The results of the comparison demonstrated that the proposed algorithms perform better than the existing ones in terms of the value of the objective function for learning with a support vector machine and in terms of computational time.…”

Section: Introductionmentioning

confidence: 99%

“…Section 4 presents convergence analyses. Section 5 describes numerical comparisons of the proposed algorithms with the existing ones in Nedić and Bertsekas ( 2001 ), Shalev-Shwartz et al ( 2011 ), and Hishinuma and Iiduka ( 2015 ) using concrete machine learning datasets (LeCun et al, 1998b ; Dheeru and Karra Taniskidou, 2017 ; Lin, 2017 ). In this section, we also describe how to use one of the proposed algorithms to train a multilayer neural network for recognizing handwritten digits.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Incremental and Parallel Machine Learning Algorithms With Automated Learning Rate Adjustments

Hishinuma

Iiduka

2019

Front. Robot. AI

View full text Add to dashboard Cite

The existing machine learning algorithms for minimizing the convex function over a closed convex set suffer from slow convergence because their learning rates must be determined before running them. This paper proposes two machine learning algorithms incorporating the line search method, which automatically and algorithmically finds appropriate learning rates at run-time. One algorithm is based on the incremental subgradient algorithm, which sequentially and cyclically uses each of the parts of the objective function; the other is based on the parallel subgradient algorithm, which uses parts independently in parallel. These algorithms can be applied to constrained nonsmooth convex optimization problems appearing in tasks of learning support vector machines without adjusting the learning rates precisely. The proposed line search method can determine learning rates to satisfy weaker conditions than the ones used in the existing machine learning algorithms. This implies that the two algorithms are generalizations of the existing incremental and parallel subgradient algorithms for solving constrained nonsmooth convex optimization problems. We show that they generate sequences that converge to a solution of the constrained nonsmooth convex optimization problem under certain conditions. The main contribution of this paper is the provision of three kinds of experiment showing that the two algorithms can solve concrete experimental problems faster than the existing algorithms. First, we show that the proposed algorithms have performance advantages over the existing ones in solving a test problem. Second, we compare the proposed algorithms with a different algorithm Pegasos, which is designed to learn with a support vector machine efficiently, in terms of prediction accuracy, value of the objective function, and computational time. Finally, we use one of our algorithms to train a multilayer neural network and discuss its applicability to deep learning.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%