Curriculum learning of multiple tasks

Abstract

Sharing information between multiple tasks enables algorithms to achieve good generalization performance even from small amounts of training data. However, in a realistic scenario of multi-task learning not all tasks are equally related to each other, hence it could be advantageous to transfer information only between the most related tasks.In this work we propose an approach that processes multiple tasks in a sequence with sharing between subsequent tasks instead of solving all tasks jointly. Subsequently, we address the question of curriculum learning of tasks, i.e. finding the best order of tasks to be learned. Our approach is based on a generalization bound criterion for choosing the task order that optimizes the average expected classification performance over all tasks. Our experimental results show that learning multiple related tasks sequentially can be more effective than learning them jointly, the order in which tasks are being solved affects the overall performance, and that our model is able to automatically discover the favourable order of tasks.

Joint Manifold Diffusion for Combining Predictions on Decoupled Observations

“…5 Similarly to RankSVM, MTL 1 can also construct non-linear predictors using Gaussian kernels (with hyperparameter σ 2 S ). D) MTL 2 adapts Pentina et al's curriculum learning approach [19], which penalizes the deviation of the main predictor parameter w from a single best reference predictor w k . Pentina et al's original algorithm uses a bound on the generalization accuracy to select the reference predictor, which is not directly applicable to our rank learning problem.…”

Ambiguity Helps: Classification with Disagreements in Crowdsourced Annotations

“…In the second experiment, we focus on differentiating between 'easy' and 'hard' images of animal classes. We use a subset of Animals with Attributes dataset (AwA) 5 [13] for which the annotation of easy-hard scores is available 6 [22]. For each class the annotation specifies ranking scores of its images from easiest to hardest.…”

“…With MTurk, now it becomes possible to collect annotations for large datasets such as ImageNet [26], TinyImages [31], COCO [14], and Places [38]. Moreover, it becomes prevalent to collect task-specific datasets, for example for studying the attributes and their strength [20] and for determining the easiness or hardness of a particular classification task [22]. Those task-specific datasets often require annotations that are more ambiguous than typical object annotations 'present' or 'not present'.…”

“…A confidence in the label annotations, ranging from a simple percentage agreement to the kappa statistic, is then computed from the multiple annotations at each data point. For sufficiently high confidence annotations, a majority voting scheme is then widely adopted as the ground-truth label [31,14,38,20,22,26].…”

From Easy to Difficult: A Self-Paced Multi-Task Joint Sparse Representation Method

“…Some researchers have combined the self-paced learning with the multi-task learning. Pentina [20] proposed a curriculum learning of multiple tasks to solve multiple tasks in a sequential manner. The limitation of this model was that it currently allowed to transfer only from the previous task to solve the current one.…”