A Convex Formulation of SVM-Based Multi-task Learning

Multitask Support Vector Regression for Solar and Wind Energy Prediction

“…where α, β, and 1 are defined as in (2). However, observe now the multiple equality constraints in (6), in contrast with the single equality constraint in (2); they appear as a consequence of the multiple biases b r in (5). The dual ( 6) is again obtained through Lagrangian theory; see in [6] for more details, where it is also shown that the multi-task kernel matrix Q has the form…”

“…Moreover, it is not clear beforehand if there is any limiting relation between the MTL model and the common and independent tasks. To avoid this, we have proposed in [6] a new convex formulation of the MTL-SVM, in which this balancing parameter is replaced by a convex combination of the common and task-specific models, governed by a parameter whose domain is the range [0, 1]. In particular, the extreme 0 and 1 values correspond to common task learning (CTL) and independent task learning (ITL), respectively.…”

“…The second is to fit a specific model for each task subsample; we will call this independent task learning SVR (itlSVR). However, it is possible to use task information using an MTL-SVR formulation, as proposed by Cai and Cherkasski in [35], and in [6] we have given a simpler, convex MTL-SVR formulation which we showed to be equivalent to that in [35], but with the advantage of a clear identification of the interplay between the common and independent tasks. In more detail, the primal problem of this approach, which we will call mtlSVR, is arg min…”

“…MTL has grown enormously since its proposal by R. Caruana [2], and MTL approaches have been proposed for the main ML paradigms; recent surveys can be found in [3,4], see also Section 2. In this work, we will apply an MTL approach to support vector regression (SVR), first proposed in [5], using the convex formulation of the work in [6] for the prediction of wind and solar energy. More precisely, and besides a thorough yet still simple review of the main ideas and issues in convex MTL-SVR which may be of independent interest, our main contributions are as follows.…”