Multi-Task Learning with Group-Specific Feature Space Sharing

Yousefi, Niloofar; Georgiopoulos, Michael; Anagnostopoulos, Georgios C.

doi:10.1007/978-3-319-23525-7_8

Cited by 7 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[46] provides a bound on the empirical GRC of the aforementioned hypothesis space. However, similar to the proof of Corollary 21, we can easily convert it to a distribution dependent GRC bound which matches our global bound in (56) (for the defined hypothesis space (65)) and in our notation reads as…”

Section: Comparisons To Related Workmentioning

confidence: 90%

“…At the core of each MTL formulation lies a mechanism that encodes task relatedness into the learning problem [24]. Such relatedness mechanism can always be thought of as jointly constraining the tasks' hypothesis spaces, so that their geometry is mutually coupled, e.g., via a block norm constraint [65]. Thus, from a regularization perspective, the tasks mutually regularize their learning based on their inter-task relatedness.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Local Rademacher Complexity-based Learning Guarantees for Multi-Task Learning

Yousefi,

Lei,

Kloft

et al. 2016

Preprint

Self Cite

View full text Add to dashboard Cite

We show a Talagrand-type concentration inequality for Multi-Task Learning (MTL), using which we establish sharp excess risk bounds for MTL in terms of distribution-and data-dependent versions of the Local Rademacher Complexity (LRC). We also give a new bound on the LRC for norm regularized as well as strongly convex hypothesis classes, which applies not only to MTL but also to the standard i.i.d. setting. Combining both results, one can now easily derive fast-rate bounds on the excess risk for many prominent MTL methods, including-as we demonstrate-Schatten-norm, group-norm, and graph-regularized MTL. The derived bounds reflect a relationship akeen to a conservation law of asymptotic convergence rates. This very relationship allows for trading off slower rates w.r.t. the number of tasks for faster rates with respect to the number of available samples per task, when compared to the rates obtained via a traditional, global Rademacher analysis.

show abstract

Section: Comparisons To Related Workmentioning

confidence: 90%

Section: Introductionmentioning

confidence: 99%

Local Rademacher Complexity-based Learning Guarantees for Multi-Task Learning

Yousefi,

Lei,

Kloft

et al. 2016

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Nair et al [25] evaluated the Spark primary architecture running ML algorithms and reported extensive results that emphasized Spark's advantages. Other groups [26,27] have employed Apache Spark for Twitter data sentiment analysis, and there have been many important contributions to large data collection analysis (interested readers are referred to [28][29][30][31] for further details).…”

Section: Background and Related Workmentioning

confidence: 99%

A Two-Stage Big Data Analytics Framework with Real World Applications Using Spark Machine Learning and Long Short-Term Memory Network

2018

View full text Add to dashboard Cite

Every day we experience unprecedented data growth from numerous sources, which contribute to big data in terms of volume, velocity, and variability. These datasets again impose great challenges to analytics framework and computational resources, making the overall analysis difficult for extracting meaningful information in a timely manner. Thus, to harness these kinds of challenges, developing an efficient big data analytics framework is an important research topic. Consequently, to address these challenges by exploiting non-linear relationships from very large and high-dimensional datasets, machine learning (ML) and deep learning (DL) algorithms are being used in analytics frameworks. Apache Spark has been in use as the fastest big data processing arsenal, which helps to solve iterative ML tasks, using distributed ML library called Spark MLlib. Considering real-world research problems, DL architectures such as Long Short-Term Memory (LSTM) is an effective approach to overcoming practical issues such as reduced accuracy, long-term sequence dependency, and vanishing and exploding gradient in conventional deep architectures. In this paper, we propose an efficient analytics framework, which is technically a progressive machine learning technique merged with Spark-based linear models, Multilayer Perceptron (MLP) and LSTM, using a two-stage cascade structure in order to enhance the predictive accuracy. Our proposed architecture enables us to organize big data analytics in a scalable and efficient way. To show the effectiveness of our framework, we applied the cascading structure to two different real-life datasets to solve a multiclass and a binary classification problem, respectively. Experimental results show that our analytical framework outperforms state-of-the-art approaches with a high-level of classification accuracy.

show abstract

“…As shown by the past empirical works [5,1,2,31,17], it is beneficial to learn multiple related tasks simultaneously instead of independently as typically done in practice. A commonly utilized information sharing strategy for Multi-Task Learning (MTL) is to use a (partially) common feature mapping φ to map the data from all tasks to a (partially) shared feature space H. Such a method, named kernel-based MTL, not only allows information sharing across tasks, but also enjoys the non-linearity that is brought by the feature mapping φ.…”

Section: Introductionmentioning

confidence: 99%

Multi-Task Learning Using Neighborhood Kernels

Yousefi¹,

Liu²,

Mollaghasemi³

et al. 2017

Preprint

Self Cite

View full text Add to dashboard Cite

This paper introduces a new and effective algorithm for learning kernels in a Multi-Task Learning (MTL) setting. Although, we consider a MTL scenario here, our approach can be easily applied to standard single task learning, as well. As shown by our empirical results, our algorithm consistently outperforms the traditional kernel learning algorithms such as uniform combination solution, convex combinations of base kernels as well as some kernel alignment-based models, which have been proven to give promising results in the past. We present a Rademacher complexity bound based on which a new Multi-Task Multiple Kernel Learning (MT-MKL) model is derived. In particular, we propose a Support Vector Machine (SVM)-regularized model in which, for each task, an optimal kernel is learned based on a neighborhood-defining kernel that is not restricted to be Positive Semi-Definite (PSD). Comparative experimental results are showcased that underline the merits of our neighborhood-defining framework in both classification and regression problems.

show abstract

Multi-Task Learning with Group-Specific Feature Space Sharing

Cited by 7 publications

References 17 publications

Local Rademacher Complexity-based Learning Guarantees for Multi-Task Learning

Local Rademacher Complexity-based Learning Guarantees for Multi-Task Learning

A Two-Stage Big Data Analytics Framework with Real World Applications Using Spark Machine Learning and Long Short-Term Memory Network

Multi-Task Learning Using Neighborhood Kernels

Contact Info

Product

Resources

About