AM-LFS: AutoML for Loss Function Search

Li, Chuming; Yuan, Xin; Chen, Lin; Guo, Minghao; Wu, Wei; Yan, Junjie; Ouyang, Wanli

doi:10.1109/iccv.2019.00850

Cited by 63 publications

(41 citation statements)

References 42 publications

(78 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…It is almost impossible to explore specific Gaussian distributions for various landmarks and tasks with hand-crafted heuristics, especially when the number of landmarks increases. Inspired by the work [30], we can consider the variances or standard deviations of the Gaussian distributions as the network hyperparameters to determine the distance metric between the predicted values and the ground truth. We propose a learning-to-learn framework equipping U-Net with an RL-based approach, which can steadily optimize the network parameters and the Gaussian standard deviations of the heatmap ground truth simultaneously.…”

Section: A Landmark-aware Objective Metric Learningmentioning

confidence: 99%

Learn Fine-Grained Adaptive Loss for Multiple Anatomical Landmark Detection in Medical Images

Zhou

Miao

et al. 2021

IEEE J. Biomed. Health Inform.

View full text Add to dashboard Cite

Automatic and accurate detection of anatomical landmarks is an essential operation in medical image analysis with a multitude of applications. Recent deep learning methods have improved results by directly encoding the appearance of the captured anatomy with the likelihood maps (i.e., heatmaps). However, most current solutions overlook another essence of heatmap regression, the objective metric for regressing target heatmaps and rely on hand-crafted heuristics to set the target precision, thus being usually cumbersome and task-specific. In this paper, we propose a novel learning-to-learn framework for landmark detection to optimize the neural network and the target precision simultaneously. The pivot of this work is to leverage the reinforcement learning (RL) framework to search objective metrics for regressing multiple heatmaps dynamically during the training process, thus avoiding setting problem-specific target precision. We also introduce an early-stop strategy for active termination of the RL agent's interaction that adapts the optimal precision for separate targets considering exploration-exploitation tradeoffs. This approach shows better stability in training and improved localization accuracy in inference. Extensive experimental results on two different applications of landmark localization: 1) our in-house prenatal ultrasound (US) dataset and 2) the publicly available dataset of cephalometric X-Ray landmark detection, demonstrate the effectiveness of our proposed method. Our proposed framework is general and shows the potential to improve the efficiency of anatomical landmark detection.

show abstract

Section: A Landmark-aware Objective Metric Learningmentioning

confidence: 99%

Learn Fine-Grained Adaptive Loss for Multiple Anatomical Landmark Detection in Medical Images

Zhou

Miao

et al. 2021

IEEE J. Biomed. Health Inform.

View full text Add to dashboard Cite

show abstract

“…Moreover, some auxiliary loss [25,26] are employed to train models together with classification loss functions. Recently, some approaches focus on hard examples [27] or AutoML searching [28,29] to get better loss functions. Though these loss functions achieve better and better performance on FR, but they all depend on fulllabeled identity datasets, such as MS-Celeb-1M and Casia, and cannot treat face sequences as the training data to learn discriminative face features.…”

Section: Deep Face Recognitionmentioning

confidence: 99%

SeqFace: Learning discriminative features by using face sequences

Huang

Zhang

et al. 2021

IET Image Processing

View full text Add to dashboard Cite

Deep convolutional neural networks (CNNs) have greatly improved the Face Recognition (FR) performance in recent years. Almost all CNNs in FR are trained on the carefully labeled datasets containing plenty of identities. However, such high‐quality datasets are very expensive to collect, which restricts many researchers to achieve state‐of‐the‐art performance. In this paper, a framework, called SeqFace, for learning discriminative face features is proposed. Besides a traditional identity training dataset, the designed SeqFace can train CNNs by using an additional dataset which includes a large number of face sequences collected from videos. Moreover, the label smoothing regularization (LSR) and a new proposed discriminative sequence agent (DSA) loss are employed to enhance the discrimination power of deep face features via making full use of the sequence data. Only with a single ResNet model, the method achieves very competitive performance on several face recognition benchmarks, including LFW, YTF, CFP, AgeDB, and MegaFace. The code and model are publicly available at the website https://github.com/huangyangyu/SeqFace.

show abstract

“…Previous methods relay on hand-crafted heuristics that require much effort from experts to explore the large design space. To address this issue, Li et al (Li et al, 2019) propose a new AutoML for Loss Function Search (AM-LFS) to automatically determine the search space. Specifically, the formulation of AM-LFS is written as follows:…”

Section: Preliminary Knowledgementioning

confidence: 99%

“…Recently, Li et al (Li et al, 2019) propose an AutoML for loss function search method (AM-LFS) from a hyperparameter optimization perspective. Specifically, they formulate hyper-parameters of loss functions as a parameterized probability distribution sampling and achieve promising results on several different vision tasks.…”

Section: Introductionmentioning

confidence: 99%

A joint loss function for deep face recognition

Wang

Chen

2018

Multidim Syst Sign Process

View full text Add to dashboard Cite

In face recognition, designing margin-based (e.g., angular, additive, additive angular margins) softmax loss functions plays an important role in learning discriminative features. However, these hand-crafted heuristic methods are sub-optimal because they require much effort to explore the large design space. Recently, an AutoML for loss function search method AM-LFS has been derived, which leverages reinforcement learning to search loss functions during the training process. But its search space is complex and unstable that hindering its superiority. In this paper, we first analyze that the key to enhance the feature discrimination is actually how to reduce the softmax probability. We then design a unified formulation for the current margin-based softmax losses. Accordingly, we define a novel search space and develop a reward-guided search method to automatically obtain the best candidate. Experimental results on a variety of face recognition benchmarks have demonstrated the effectiveness of our method over the state-of-the-art alternatives.

show abstract

AM-LFS: AutoML for Loss Function Search

Cited by 63 publications

References 42 publications

Learn Fine-Grained Adaptive Loss for Multiple Anatomical Landmark Detection in Medical Images

Learn Fine-Grained Adaptive Loss for Multiple Anatomical Landmark Detection in Medical Images

SeqFace: Learning discriminative features by using face sequences

A joint loss function for deep face recognition

Contact Info

Product

Resources

About