A brief review on multi-task learning

Thung, Kim-Han; Wee, Chong-Yaw

doi:10.1007/s11042-018-6463-x

Cited by 181 publications

(84 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In this work, we introduce a learning scheme, termed multi-task (MT) SISSO, within the framework of the wider class of learning schemes known as multi-task learning (MTL) [32][33][34][35][36][37][38][39]. A task for a learning algorithm is the learning of a target property starting from a single input source (set of features).…”

Section: Introductionmentioning

confidence: 99%

“…A task for a learning algorithm is the learning of a target property starting from a single input source (set of features). The learning of multiple tasks (or MTL) is an umbrella term that refers to [38] (i) the learning of multiple target properties using a single input source, or (ii) the joint learning of a single target property using multiple input sources, or (iii) a mixture of both.…”

Section: Introductionmentioning

confidence: 99%

“…The key aspect is the parallel learning of multiple tasks, with the (sometimes implicit) assumption that the shared information among different tasks can lead to better learning performance if all the tasks are learned jointly, as compared to learning them independently. In other words, MTL assumes that the learning of one task can improve the learning of the other tasks [38]. Though MTL has not yet been applied to materials-science problems so far, it has already been widely applied in other fields, such as in the handwriting recognition problem, self-driving automation system, computer vision, bioinformatics and health informatics, speech and language recognition, and more [32,33,38,39].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Simultaneous learning of several materials properties from incomplete databases with multi-task SISSO

et al. 2019

View full text Add to dashboard Cite

The identification of descriptors of materials properties and functions that capture the underlying physical mechanisms is a critical goal in data-driven materials science. Only such descriptors will enable a trustful and efficient scanning of materials spaces and possibly the discovery of new materials. Recently, the sure-independence screening and sparsifying operator (SISSO) has been introduced and was successfully applied to a number of materials-science problems. SISSO is a compressed sensing based methodology yielding predictive models that are expressed in form of analytical formulas, built from simple physical properties. These formulas are systematically selected from an immense number (billions or more) of candidates. In this work, we describe a powerful extension of the methodology to a 'multi-task learning' approach, which identifies a single descriptor capturing multiple target materials properties at the same time. This approach is specifically suited for a heterogeneous materials database with scarce or partial data, e.g. in which not all properties are reported for all materials in the training set. As showcase examples, we address the construction of materials properties maps for the relative stability of octet-binary compounds, considering several crystal phases simultaneously, and the metal/insulator classification of binary materials distributed over many crystal prototypes.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Simultaneous learning of several materials properties from incomplete databases with multi-task SISSO

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Multitask learning (MTL) is an active area of research in machine learning and is concerned with simultaneously learning multiple related tasks from a common dataset [21]. Inspired by this idea, many methods have been developed to determine multiple tasks simultaneously rather than separately [22]. All these techniques have the tendency to jointly determine multiple tasks within a common environment, which can complicate the learning process, but can enhance the performance of single-task learning models.…”

Section: Introductionmentioning

confidence: 99%

Multitask-based association rule mining

Taşer¹,

Birant²,

Birant³

2020

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

Recently, there has been a growing interest in association rule mining (ARM) in various fields. However, standard ARM algorithms fail to discover rules for multitask problems as they do not consider task-oriented investigation and, therefore, they ignore the correlation among the tasks. Considering this situation, this paper proposes a novel algorithm, named multitask association rule miner (MTARM), that tends to jointly discover rules by considering multiple tasks. This paper also introduces two novel concepts: single-task rule and multiple-task rule. In the first phase of the proposed approach, highly frequent local rules (single-task rules) are explored for each task separately and then these local rules are combined to produce the global result (multitask rules) using a majority voting mechanism. Experiments were conducted on four different real-world multitask learning datasets. The experimental results indicated that the proposed MTARM approach discovers more information than that of traditional ARM algorithms by jointly considering the relationships among multiple tasks. This work is licensed under a Creative Commons Attribution 4.0 International License. 933 YILDIRIM TAŞER et al./Turk J Elec Eng & Comp Scisubset of them, are familiar to one mutual rule set with a small difference. The proposed approach can discover common association rules by responding to different tasks and by applying an algorithm to consider all the tasks collectively. Accordingly, the rule X → Y can have high support and confidence in a task. A rule may not be frequent in the entire dataset; however, it may be frequent in several specific tasks. Therefore, finding task-based frequent rules in an effective way is important. This paper proposes a novel algorithm, named multitask association rule miner (MTARM), that tends to discover rules by jointly considering multiple tasks. The proposed algorithm mines datasets that include task information. The MTARM approach consists of two phases. In the first phase, the algorithm discovers local frequent rules (single-task rules) from the data of each task separately, and in the second phase, it combines these local rules to produce the global result (multitask rules). The discovered multitask rules can be used to deploy information systems that support the execution of their associated task or to improve decision-making in applications.The main contributions and novelty of this study are threefold: (i) this is the first study that applies ARM to multitask problems; (ii) it proposes the MTARM approach, which attempts to find intertask rules based solely on the task identities and the observed data for each task; and, finally (iii) it also introduces two novel concepts: single-task rule and multitask rule. This paper also provides a brief survey of ARM as it is an active research area of data mining. In addition, it presents the results of experimental studies conducted on four different real-world datasets to demonstrate the capability of the proposed algorithm.The rest of the paper is organized...

show abstract

“…A possible method, which recently attracted great interest, is the use of feed-forward neural network (FFNN) architectures, with a certain number of hidden layers and an appropriate number of output neurons, each responsible for predicting the desired variables y i with i = 1, ..., d. In the example of oxygen sensing, the output layer would have a neuron for the oxygen concentration [O 2 ] and one for the temperature T. This work shows that, since the output neurons must use the same features (the output of the last hidden layer) for all variables [10,11], FFNNs are insufficiently flexible. For the cases when the variables depend on fundamentally different ways from the inputs, this approach will give a result that is at best acceptable, and at worst unusable.…”

Section: Introductionmentioning

confidence: 99%

Multi-Task Learning for Multi-Dimensional Regression: Application to Luminescence Sensing

Umberto¹,

Michelucci

Venturini³

2019

Applied Sciences

View full text Add to dashboard Cite

The classical approach to non-linear regression in physics is to take a mathematical model describing the functional dependence of the dependent variable from a set of independent variables, and then using non-linear fitting algorithms, extract the parameters used in the modeling. Particularly challenging are real systems, characterized by several additional influencing factors related to specific components, like electronics or optical parts. In such cases, to make the model reproduce the data, empirically determined terms are built in the models to compensate for the difficulty of modeling things that are, by construction, difficult to model. A new approach to solve this issue is to use neural networks, particularly feed-forward architectures with a sufficient number of hidden layers and an appropriate number of output neurons, each responsible for predicting the desired variables. Unfortunately, feed-forward neural networks (FFNNs) usually perform less efficiently when applied to multi-dimensional regression problems, that is when they are required to predict simultaneously multiple variables that depend from the input dataset in fundamentally different ways. To address this problem, we propose multi-task learning (MTL) architectures. These are characterized by multiple branches of task-specific layers, which have as input the output of a common set of layers. To demonstrate the power of this approach for multi-dimensional regression, the method is applied to luminescence sensing. Here, the MTL architecture allows predicting multiple parameters, the oxygen concentration and temperature, from a single set of measurements.

show abstract

A brief review on multi-task learning

Cited by 181 publications

References 70 publications

Simultaneous learning of several materials properties from incomplete databases with multi-task SISSO

Simultaneous learning of several materials properties from incomplete databases with multi-task SISSO

Multitask-based association rule mining

Multi-Task Learning for Multi-Dimensional Regression: Application to Luminescence Sensing

Contact Info

Product

Resources

About