Context-sensitive Multiple Task Learning, or csMTL, is presented as a method of inductive transfer which uses a single output neural network and additional contextual inputs for learning multiple tasks. Motivated by problems with the application of MTL networks to machine lifelong learning systems, csMTL encoding of multiple task examples was developed and found to improve predictive performance. As evidence, the csMTL method is tested on seven task domains and shown to produce hypotheses for primary tasks that are often better than standard MTL hypotheses when learning in the presence of related and unrelated tasks. We argue that the reason for this performance improvement is a reduction in the number of effective free parameters in the csMTL network brought about by the shared output node and weight update constraints due to the context inputs. An examination of IDT and SVM models developed from csMTL encoded data provides initial evidence that this improvement is not shared across all machine learning models.

No abstract

No abstract

Fundamental to the problem of lifelong machine learning is how to consolidate the knowledge of a learned task within a long-term memory structure (domain knowledge) without the loss of prior knowledge. We investigate the effect of curriculum, ie. the order in which tasks are learned, on the consolidation of task knowledge. Relevant background material on knowledge transfer and consolidation using multiple task learning (MTL) neural networks is reviewed. A large MTL network is used as the long-term memory structure and task rehearsal overcomes the stability-plasticity problem and the loss of prior knowledge. Experimental results demonstrate that curriculum has an important effect on the accuracy of consolidated knowledge particularly for the first few tasks that are learned. The results also suggest that, for given set of tasks and training examples, the mean accuracy of consolidated domain knowledge converges to the same level regardless of the curriculum.

No abstract