BDUOL: Double Updating Online Learning on a Fixed Budget

Zhao, Peilin; Hoi, Steven C. H.

doi:10.1007/978-3-642-33460-3_57

Cited by 27 publications

(28 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Perceptron algorithm (Rosenblatt 1958;Freund and Schapire 1999) is one important online approach which updates the learning function by adding a new example with a constant weight when it is misclassified. Recently a number of online learning algorithms have been proposed based on the criterion of maximum margin (Li and Long 1999;Gentile 2001;Kivinen et al 2001;Crammer and Singer 2003;Crammer et al 2006;Zhao et al 2011). For example, Relaxed Online Maximum Margin (ROMMA) (Li and Long 1999) algorithm repeatedly chooses the hyper-planes that correctly classify the existing training examples with the maximum margin.…”

Section: Online Learningmentioning

confidence: 99%

PAMR: Passive aggressive mean reversion strategy for portfolio selection

et al. 2012

Self Cite

View full text Add to dashboard Cite

This article proposes a novel online portfolio selection strategy named "Passive Aggressive Mean Reversion" (PAMR). Unlike traditional trend following approaches, the proposed approach relies upon the mean reversion relation of financial markets. Equipped with online passive aggressive learning technique from machine learning, the proposed portfolio selection strategy can effectively exploit the mean reversion property of markets. By analyzing PAMR's update scheme, we find that it nicely trades off between portfolio return and volatility risk and reflects the mean reversion trading principle. We also present several variants of PAMR algorithm, including a mixture algorithm which mixes PAMR and other strategies. We conduct extensive numerical experiments to evaluate the empirical performance of the proposed algorithms on various real datasets. The encouraging results show that in most cases the proposed PAMR strategy outperforms all benchmarks and almost all state-of-the-art portfolio selection strategies under various performance metrics. In addition to its superior performance, the proposed PAMR runs extremely fast and thus is very suitable for real-life online trading applications. The experimental testbed including source codes and data sets is available at http://www.cais.ntu.edu.sg/~chhoi/PAMR/.

show abstract

Section: Online Learningmentioning

confidence: 99%

PAMR: Passive aggressive mean reversion strategy for portfolio selection

et al. 2012

Self Cite

View full text Add to dashboard Cite

show abstract

“…Compared to offline learning, online learning technique is more efficient and suitable to handle massive and sequential data [23,24,25]. Task structure has been exploited by using a global loss function to evaluate the prediction [26], or assuming that a few experts can perform well on the entire task set [27,28].…”

Section: Related Workmentioning

confidence: 99%

Learning Correlative and Personalized Structure for Online Multi-Task Classification

Yang

Zhao

et al. 2016

Proceedings of the 2016 SIAM International Conference on Data Mining

Self Cite

View full text Add to dashboard Cite

Multi-Task Learning (MTL) can enhance the classifier's generalization performance by learning multiple related tasks simultaneously. Conventional MTL works under the offline or batch learning setting and suffers from the expensive training cost together with the poor scalability. To address such inefficiency issues, online learning technique has been applied to solve MTL problems. However, most existing algorithms for online MTL constrain task relatedness into a presumed structure via a single weight matrix, a strict restriction that does not always hold in practice. In this paper, we propose a general online MTL framework that overcomes this restriction by decomposing the weight matrix into two components: the first component captures the correlative structure among tasks in a low-rank subspace, and the second component identifies the personalized patterns for the outlier tasks. A projected gradient scheme is devised to learn such components adaptively. Theoretical analysis shows that the proposed algorithm can achieve a sub-linear regret with respect to the best linear model in hindsight. Experimental investigation on a number of real-world datasets also verifies the efficacy of our approach.

show abstract

“…One classical online learning method is the wellknown Perceptron algorithm [35], [16]. Recently, a large number of online learning algorithms have been proposed [22], [6], [49], [21], [39], in which many of them follow the criterion of maximum margin principle [17], [24], [6], [49]. For example, the Passive-Aggressive algorithm [6] proposes to update a classifier when the incoming training example is either misclassified or fall into the range of classification margin.…”

Section: Related Workmentioning

confidence: 99%

“…Finally, we would like to distinguish our work from budget online learning [3], [11], [31], [48], [51] which aims to learn a kernel-based classifier with a bounded number of support vectors. A common strategy behind many budget online learning algorithms is to remove the "oldest" support vector when the maximum number of support vectors is reached, which however is not applicable to online feature selection.…”

Section: Related Workmentioning

confidence: 99%

Online Feature Selection and Its Applications

Wang

Zhao

Hoi

et al. 2014

IEEE Trans. Knowl. Data Eng.

Self Cite

209

View full text Add to dashboard Cite

Abstract-Feature selection is an important technique for data mining. Despite its importance, most studies of feature selection are restricted to batch learning. Unlike traditional batch learning methods, online learning represents a promising family of efficient and scalable machine learning algorithms for large-scale applications. Most existing studies of online learning require accessing all the attributes/features of training instances. Such a classical setting is not always appropriate for real-world applications when data instances are of high dimensionality or it is expensive to acquire the full set of attributes/features. To address this limitation, we investigate the problem of Online Feature Selection (OFS) in which an online learner is only allowed to maintain a classifier involved only a small and fixed number of features. The key challenge of Online Feature Selection is how to make accurate prediction for an instance using a small number of active features. This is in contrast to the classical setup of online learning where all the features can be used for prediction. We attempt to tackle this challenge by studying sparsity regularization and truncation techniques. Specifically, this article addresses two different tasks of online feature selection: (1) learning with full input where an learner is allowed to access all the features to decide the subset of active features, and (2) learning with partial input where only a limited number of features is allowed to be accessed for each instance by the learner. We present novel algorithms to solve each of the two problems and give their performance analysis. We evaluate the performance of the proposed algorithms for online feature selection on several public datasets, and demonstrate their applications to real-world problems including image classification in computer vision and microarray gene expression analysis in bioinformatics. The encouraging results of our experiments validate the efficacy and efficiency of the proposed techniques.

show abstract

BDUOL: Double Updating Online Learning on a Fixed Budget

Cited by 27 publications

References 5 publications

PAMR: Passive aggressive mean reversion strategy for portfolio selection

PAMR: Passive aggressive mean reversion strategy for portfolio selection

Learning Correlative and Personalized Structure for Online Multi-Task Classification

Online Feature Selection and Its Applications

Contact Info

Product

Resources

About