A new multi-task learning method with universum data

Xiao, Yanshan; Wen, Jing; Liu, Bo

doi:10.1007/s10489-020-01954-3

Cited by 16 publications

(4 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Typical methods include precision-based weighted ensemble methods [29] and weight-based update ensemble model methods [30]. In addition, some algorithms based on transfer learning [31], multi task learning [32][33], and deep learning [34][35][36] to the ensemble methods are proposed. The passive adaptive strategies enable models can achieve high classification accuracy.…”

Section: Related Workmentioning

confidence: 99%

Adaptive Hybrid Ensemble Method for Accelerate Adaptation of Concept Drift

Wang

Guo

Zhang

et al. 2023

Preprint

View full text Add to dashboard Cite

Because the insufficient new distribution training samples after concept drift occurs in streaming data, the performance of online learning model degrades and cannot quickly recover. Therefore, an adaptive hybrid ensemble method for accelerate adaptation of concept drift(AHE\_A$^2$CD) is proposed. After concept drift occurs, the proposed method extracts local information from the streaming data through the weighted base classifiers located in the classifier pool. The local information is supplemented into the current data block through expanding the data to make up for the lack of current distribution data after concept drift occurs and to build an efficient local base learner that conforms to the current data distribution. On this basis, the key data information at different stages is extracted by local base learner, and the current data is adaptively selected by the data distribution to construct diverse global base learner. Through the hybrid ensemble of the high-performance local base learner and the diverse global base learner, this method can adaptively learn the changing streaming data and improve the adaptability after concept drift occurs. Experimental results show that this method can accelerate the convergence of the online learning model after concept drift occurs and improve the real-time performance of streaming data classification.

show abstract

Section: Related Workmentioning

confidence: 99%

Adaptive Hybrid Ensemble Method for Accelerate Adaptation of Concept Drift

Wang

Guo

Zhang

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Xie et al [35] presented a novel multi-task twin pinball loss SVM, which enhances the noise insensitivity of multi-task twin hinge loss SVM. For more information on MTLSVM, see [36][37][38][39]. Although some attempts have been made to introduce MTL into SVM, and they perform better than STL in most cases, they do not address the problems of classifier sensitivity to noise and resampling instability.…”

Section: Introductionmentioning

confidence: 99%

Multi-task support vector machine with generalized huber loss

Liu

Zhu

2022

Preprint

View full text Add to dashboard Cite

The main goal of multi-task learning (MTL) is to improve generalization ability by utilizing domain-specific information implicit in training signals of multiple related tasks. MTL achieves this goal by training multiple tasks in parallel using shared representations. The adaptation of MTL to support vector machines (SVM) is a pretty successful example. Loss function plays a crucial role in the algorithm implementation and classification accuracy of SVM. Classical support vector machine(SVM) and multi-task learning support vector machine(MTLSVM) utilize the non-differentiable and noise sensitive hinge loss function. SVM with pinball loss which penalizes correctly classified points enhances the noise insensitivity of classical SVM, but it is also non-differentiable. Both pinball loss SVM and hinge loss SVM suffer from higher computational cost because they have to solve a quadratic programming problem (QPP) whose complexity is proportional to the cube of the dataset size. In contrast, the differentiability of huber loss function reduces the sensitivity to noise compared to hinge loss and pinball loss. The differentiability of huber loss function can greatly improve the running speed of SVM algorithm. Inspired by the recently published generalized huber SVM(GHSVM) and regularized multi-task learning(RMTL),we propose a novel multi-task support vector machine with generalized huber loss, named as GHMTSVM. The new method extends the GHSVM for single task learning to the multi-task learning SVM. To our knowledge, the application of the generalized huber loss to MTLSVM is innovative. The proposed method mainly has two advantages: on the one hand, compared with pinball loss and hinge loss, the huber loss is less sensitive to outliers and has better robustness; on the other hand, it adopts functional iterative to find the optimal solution, and does not need to solve QPP, which improves the solving speed. Numerical experiments are carried out on five real datasets and the result is compared with SVM, GHSVM and MTLSVM methods to verify the good performance of the proposed algorithm. Experimental results show that the GHMTSVM outperforms existing single-task learning SVMs and multi-task learning SVM.

show abstract

“…This model has experimentally demonstrated that it delivers a better accuracy compared to the methods that only make use of labeled data. Further, many Universum models have been recently proposed that improve their parental models by increasing the classification accuracy [25,37,30].…”

Section: Introductionmentioning

confidence: 99%

Sparse Universum Quadratic Surface Support Vector Machine Models for Binary Classification

Moosaei,

Mousavi,

Hladík

et al. 2021

Preprint

View full text Add to dashboard Cite

In binary classification, kernel-free linear or quadratic support vector machines are proposed to avoid dealing with difficulties such as finding appropriate kernel functions or tuning their hyper-parameters. Furthermore, Universum data points, which do not belong to any class, can be exploited to embed prior knowledge into the corresponding models so that the generalization performance is improved. In this paper, we design novel kernel-free Universum quadratic surface support vector machine models. Further, we propose the L1 norm regularized version that is beneficial for detecting potential sparsity patterns in the Hessian of the quadratic surface and reducing to the standard linear models if the data points are (almost) linearly separable. The proposed models are convex such that standard numerical solvers can be utilized for solving them. Nonetheless, we formulate a least squares version of the L1 norm regularized model and next, design an effective tailored algorithm that only requires solving one linear system. Several theoretical properties of these models are then reported/proved as well. We finally conduct numerical experiments on both artificial and public benchmark data sets to demonstrate the feasibility and effectiveness of the proposed models.

show abstract

A new multi-task learning method with universum data

Cited by 16 publications

References 44 publications

Adaptive Hybrid Ensemble Method for Accelerate Adaptation of Concept Drift

Adaptive Hybrid Ensemble Method for Accelerate Adaptation of Concept Drift

Multi-task support vector machine with generalized huber loss

Sparse Universum Quadratic Surface Support Vector Machine Models for Binary Classification

Contact Info

Product

Resources

About