Multi-Task Learning for Multi-Dimensional Regression: Application to Luminescence Sensing

Abstract

The classical approach to non-linear regression in physics is to take a mathematical model describing the functional dependence of the dependent variable from a set of independent variables, and then using non-linear fitting algorithms, extract the parameters used in the modeling. Particularly challenging are real systems, characterized by several additional influencing factors related to specific components, like electronics or optical parts. In such cases, to make the model reproduce the data, empirically determined terms are built in the models to compensate for the difficulty of modeling things that are, by construction, difficult to model. A new approach to solve this issue is to use neural networks, particularly feed-forward architectures with a sufficient number of hidden layers and an appropriate number of output neurons, each responsible for predicting the desired variables. Unfortunately, feed-forward neural networks (FFNNs) usually perform less efficiently when applied to multi-dimensional regression problems, that is when they are required to predict simultaneously multiple variables that depend from the input dataset in fundamentally different ways. To address this problem, we propose multi-task learning (MTL) architectures. These are characterized by multiple branches of task-specific layers, which have as input the output of a common set of layers. To demonstrate the power of this approach for multi-dimensional regression, the method is applied to luminescence sensing. Here, the MTL architecture allows predicting multiple parameters, the oxygen concentration and temperature, from a single set of measurements.

Multi-task learning architecture for brain tumor detection and segmentation in MRI images

“…The suggested MTL is a strategy for machine learning in which n learning tasks are simultaneously executed using commonalities and differences across tasks. 23 To be clinically approved for brain tumor diagnosis, a model must be lightweight, generic, and capable of handling multiclass segmentation robustly. This was the primary reason for developing this encoder-decoderbased MTL architecture.…”

“…In our proposed model, besides the main task of segmentation, we added classification as an auxiliary task, which helps to force the model to preserve as much relevant and essential information as possible. 23 It also helps in smooth segmentation prediction and joint learning helps optimize feature extraction. As mentioned in Sec.…”

“…Rather than learning the main concepts of data, the model memorizes all the inputs, reducing model efficiency. 23 To overcome this problem, one dropout layer after the concatenation layer is used at the decoder side to help decrease the trainable parameters. At the same time, max-pooling was also used to reduce the feature map.…”

“…The shared encoder learns efficient data coding through convolutional layers and keeps as much pertinent data as feasible. 23 The encoder component is the shared backbone, generating an output known as a "shared representation." The name is derived from the fact that the output of those layers is used to assess the output tasks.…”

“…Each loss is given a weight, combined using an algebraic sum and is then used for optimization. 23 For the proposed scheme, the loss weight used for classification was w 1 ¼ 0.001, whereas for segmentation, it was w 2 ¼ 0.1. These values for loss weights gave the optimum results.…”