Auxiliary Learning by Implicit Differentiation

Navon, Aviv; Achituve, Idan; Maron, Haggai; Chechik, Gal; Fetaya, Ethan

doi:10.48550/arxiv.2007.02693

Cited by 6 publications

(7 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, Reinforcement Learning strategies may be concerned with further future research. More precisely, we are currently investigating a Machine Learning field known as Auxiliary Learning [71] after introducing routing effectively to Reinforcement Learning. In this implementation setting, we assume that since the capsule routing helps us model the hierarchical relationships available in these systems, the model we built (A2C with routing) may provide satisfactory results.…”

Section: Discussionmentioning

confidence: 99%

An Empirical Review of Automated Machine Learning

2021

View full text Add to dashboard Cite

In recent years, Automated Machine Learning (AutoML) has become increasingly important in Computer Science due to the valuable potential it offers. This is testified by the high number of works published in the academic field and the significant efforts made in the industrial sector. However, some problems still need to be resolved. In this paper, we review some Machine Learning (ML) models and methods proposed in the literature to analyze their strengths and weaknesses. Then, we propose their use—alone or in combination with other approaches—to provide possible valid AutoML solutions. We analyze those solutions from a theoretical point of view and evaluate them empirically on three Atari games from the Arcade Learning Environment. Our goal is to identify what, we believe, could be some promising ways to create truly effective AutoML frameworks, therefore able to replace the human expert as much as possible, thereby making easier the process of applying ML approaches to typical problems of specific domains. We hope that the findings of our study will provide useful insights for future research work in AutoML.

show abstract

Section: Discussionmentioning

confidence: 99%

An Empirical Review of Automated Machine Learning

2021

View full text Add to dashboard Cite

show abstract

“…Specifically, Shi et al [51] used a similar concept to Lin et al [52], aiming to ensure that the weighted sum of gradients was close to the primary task gradient. Furthermore, Navon et al [54] suggested learning a nonlinear fusion of auxiliary loss. In contrast, Chen et al [55] proposed selecting tasks and individual data samples within each task to maximize auxiliary information utilization.…”

Section: Auxiliary Learningmentioning

confidence: 99%

Auxcoformer: Auxiliary and Contrastive Transformer for Robust Crack Detection in Adverse Weather Conditions

Yoon,

Jung,

Yoo

2024

Mathematics

View full text Add to dashboard Cite

Crack detection is integral in civil infrastructure maintenance, with automated robots for detailed inspections and repairs becoming increasingly common. Ensuring fast and accurate crack detection for autonomous vehicles is crucial for safe road navigation. In these fields, existing detection models demonstrate impressive performance. However, they are primarily optimized for clear weather and struggle with occlusions and brightness variations in adverse weather conditions. These problems affect automated robots and autonomous vehicle navigation that must operate reliably in diverse environmental conditions. To address this problem, we propose Auxcoformer, designed for robust crack detection in adverse weather conditions. Considering the image degradation caused by adverse weather conditions, Auxcoformer incorporates an auxiliary restoration network. This network efficiently restores damaged crack details, ensuring the primary detection network obtains better quality features. The proposed approach uses a non-local patch-based 3D transform technique, emphasizing the characteristics of cracks and making them more distinguishable. Considering the connectivity of cracks, we also introduce contrastive patch loss for precise localization. Then, we demonstrate the performance of Auxcoformer, comparing it with other detection models through experiments.

show abstract

“…The main methodology of the CoLab strategy is inspired by some recently proposed methods which aim to generate the weights for pre-defined auxiliary tasks or labels through a similar meta-learning framework [23], [27]. In this study, CoLab is specifically designed for semantic segmentation with heterogeneous background classes which is a common scenario in medical imaging.…”

Section: B Multi-task Learningmentioning

confidence: 99%

Context Label Learning: Improving Background Class Representations in Semantic Segmentation

Kamnitsas

Ouyang

et al. 2023

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Background samples provide key contextual information for segmenting regions of interest (ROIs). However, they always cover a diverse set of structures, causing difficulties for the segmentation model to learn good decision boundaries with high sensitivity and precision. The issue concerns the highly heterogeneous nature of the background class, resulting in multi-modal distributions. Empirically, we find that neural networks trained with heterogeneous background struggle to map the corresponding contextual samples to compact clusters in feature space. As a result, the distribution over background logit activations may shift across the decision boundary, leading to systematic over-segmentation across different datasets and tasks. In this study, we propose context label learning (CoLab) to improve the context representations by decomposing the background class into several subclasses. Specifically, we train an auxiliary network as a task generator, along with the primary segmentation model, to automatically generate context labels that positively affect the ROI segmentation accuracy. Extensive experiments are conducted on several challenging segmentation tasks and datasets. The results demonstrate that CoLab can guide the segmentation model to map the logits of background samples away from the decision boundary, resulting in significantly improved segmentation accuracy. Code is available 1 .

show abstract

Auxiliary Learning by Implicit Differentiation

Cited by 6 publications

References 28 publications

An Empirical Review of Automated Machine Learning

An Empirical Review of Automated Machine Learning

Auxcoformer: Auxiliary and Contrastive Transformer for Robust Crack Detection in Adverse Weather Conditions

Context Label Learning: Improving Background Class Representations in Semantic Segmentation

Contact Info

Product

Resources

About