Deep multi-task learning with flexible and compact architecture search

Recent advances in domain-driven data mining

“…The results showed that deep learning model could perform better and bring valuable information to meet users' needs. Focusing on more fundamental deep learning techniques, Zhao et al [54] developed a flexible approach to compact architecture search for deep multi-task learning (MTL) problems. Though sharing model architectures is a popular method for MTL problems, identifying the appropriate components to be shared by multiple tasks is still a challenge.…”

“…We now see more examples of transfer learning, where models trained on one (source) domain are applied in another (target) domain suffering from data scarcity. However, learning generalized models that perform well on multiple tasks could be a challenging process [54]. These models are often trained with self-supervision on large data and contain millions or billions of learned parameters, such as models for language processing (e.g., BERT, GPT-3, and XLNet) and image classification (ResNet, EfficientNet, Inception).…”

“…In many domains, where there is a need for scalable low-latency inferences and when the cost of training new models and deploying them could get high, it becomes necessary to restrict the model size. There are several techniques to accomplish these objectives including pruning, distilling, and training with constraints as Zhao et al [54] demonstrated in this special issue.…”

Recent advances in domain-driven data mining

“…The results showed that deep learning model could perform better and bring valuable information to meet users' needs. Focusing on more fundamental deep learning techniques, Zhao et al [53] developed a flexible approach to compact architecture search for deep multitask learning (MTL) problems. Though sharing model architectures is a popular method for MTL problems, identifying the appropriate components to be shared by multiple tasks is still a challenge.…”

“…We now see more examples of transfer learning, where models trained on one (source) domain are applied in another (target) domain suffering from data scarcity. However, learning generalized models that perform well on multiple tasks could be a challenging process [53]. These models are often trained with self-supervision on large data and contain millions or billions of learned parameters, such as models for language processing (e.g., BERT, GPT-3, XLNet) and image classification (ResNet, EfficientNet, Inception).…”