Regularizing Deep Networks by Modeling and Predicting Label Structure

Mostajabi, Mohammadreza; Maire, Michael; Shakhnarovich, Gregory

doi:10.1109/cvpr.2018.00590

Cited by 31 publications

(16 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Formulating it as an independent pixel labeling problem not only makes the pixellevel classification unnecessarily hard, but also leads to artifacts and spatially incoherent results. Several ways to incorporate structure information into segmentation have been investigated [15,8,37,19,17,4,24]. For example, Chen et al [6] utilized denseCRF [15] as post-processing to refine the final segmentation results.…”

Section: Related Workmentioning

confidence: 99%

Adaptive Affinity Fields for Semantic Segmentation

Hwang

Liu

et al. 2018

Lecture Notes in Computer Science

159

View full text Add to dashboard Cite

Semantic segmentation has made much progress with increasingly powerful pixel-wise classifiers and incorporating structural priors via Conditional Random Fields (CRF) or Generative Adversarial Networks (GAN). We propose a simpler alternative that learns to verify the spatial structure of segmentation during training only. Unlike existing approaches that enforce semantic labels on individual pixels and match labels between neighbouring pixels, we propose the concept of Adaptive Affinity Fields (AAF) to capture and match the semantic relations between neighbouring pixels in the label space. We use adversarial learning to select the optimal affinity field size for each semantic category. It is formulated as a minimax problem, optimizing our segmentation neural network in a best worst-case learning scenario. AAF is versatile for representing structures as a collection of pixel-centric relations, easier to train than GAN and more efficient than CRF without run-time inference. Our extensive evaluations on PASCAL VOC 2012, Cityscapes, and GTA5 datasets demonstrate its above-par segmentation performance and robust generalization across domains.

show abstract

Section: Related Workmentioning

confidence: 99%

Adaptive Affinity Fields for Semantic Segmentation

Hwang

Liu

et al. 2018

Lecture Notes in Computer Science

159

View full text Add to dashboard Cite

show abstract

“…Recently, there were newly-built label regularization methods (Mostajabi, Maire, and Shakhnarovich 2018;Hao et al 2020) using an isolated network to explicitly model labels as features for supervision,w.r.t. semantic segmentation and detection.…”

Section: Teacher-free Methodsmentioning

confidence: 99%

LGD: Label-guided Self-distillation for Object Detection

Zhang¹,

Kang²,

Yang³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper, we propose the first self-distillation framework for general object detection, termed LGD (Label-Guided self-Distillation). Previous studies rely on a strong pretrained teacher to provide instructive knowledge for distillation. However, this could be unavailable in real-world scenarios. Instead, we generate an instructive knowledge by inter-andintra relation modeling among objects, requiring only student representations and regular labels. In detail, our framework involves sparse label-appearance encoding, inter-object relation adaptation and intra-object knowledge mapping to obtain the instructive knowledge. Modules in LGD are trained end-to-end with student detector and are discarded in inference. Empirically, LGD obtains decent results on various detectors, datasets, and extensive task like instance segmentation. For example in MS-COCO dataset, LGD improves Reti-naNet with ResNet-50 under 2× single-scale training from 36.2% to 39.0% mAP (+ 2.8%). For much stronger detectors like FCOS with ResNeXt-101 DCN v2 under 2× multi-scale training (46.1%), LGD achieves 47.9% (+ 1.8%). For pedestrian detection in CrowdHuman dataset, LGD boosts mMR by 2.3% for Faster R-CNN with ResNet-50. Compared with a classical teacher-based method FGFI, LGD not only performs better without requiring pretrained teacher but also with 51% lower training cost beyond inherent student learning.

show abstract

“…However, such strategy ignores the internal structure of the perspective map and may result in very poor results. Motivated by [18], we suggest a three-phase procedure to train an auto-encoder.…”

Section: Perspective Estimationmentioning

confidence: 99%

Perspective-Guided Convolution Networks for Crowd Counting

Yan

Yuan

Zuo

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

197

101

View full text Add to dashboard Cite

In this paper, we propose a novel perspective-guided convolution (PGC) for convolutional neural network (CNN) based crowd counting (i.e. PGCNet), which aims to overcome the dramatic intra-scene scale variations of people due to the perspective effect. While most state-of-the-arts adopt multi-scale or multi-column architectures to address such issue, they generally fail in modeling continuous scale variations since only discrete representative scales are considered. PGCNet, on the other hand, utilizes perspective information to guide the spatially variant smoothing of feature maps before feeding them to the successive convolutions. An effective perspective estimation branch is also introduced to PGCNet, which can be trained in either supervised setting or weakly-supervised setting when the branch has been pre-trained. Our PGCNet is single-column with moderate increase in computation, and extensive experimental results on four benchmark datasets show the improvements of our method against the state-of-the-arts. Additionally, we also introduce Crowd Surveillance, a large scale dataset for crowd counting that contains 13,000+ high-resolution images with challenging scenarios. † This work was done when Zhaoyi Yan was a research intern at Baidu * Corresponding author (a) Input (d) Ours

show abstract

Regularizing Deep Networks by Modeling and Predicting Label Structure

Cited by 31 publications

References 36 publications

Adaptive Affinity Fields for Semantic Segmentation

Adaptive Affinity Fields for Semantic Segmentation

LGD: Label-guided Self-distillation for Object Detection

Perspective-Guided Convolution Networks for Crowd Counting

Contact Info

Product

Resources

About