Soft Transfer Learning via Gradient Diagnosis for Visual Relationship Detection

Chen, Diqi; Liang, Xiaodan; Wang, Yizhou; Gao, Wen

doi:10.1109/wacv.2019.00124

Cited by 15 publications

(12 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We note that while Chen et al [4] graphs as we do in this paper. They propose a three-stage approach which generates pseudo-labels for the unlabeled examples with a biased trained model, followed by training a less biased model with the additional "positive" examples.…”

Section: Positive Unlabeled (Pu) Learningmentioning

confidence: 76%

See 1 more Smart Citation

Recovering the Unbiased Scene Graphs from the Biased Ones

Chiou

Ding²,

Yan

et al. 2021

Preprint

View full text Add to dashboard Cite

Given input images, scene graph generation (SGG) aims to produce comprehensive, graphical representations describing visual relationships among salient objects. Recently, more efforts have been paid to the long tail problem in SGG; however, the imbalance in the fraction of missing labels of different classes, or reporting bias, exacerbating the long tail is rarely considered and cannot be solved by the existing debiasing methods. In this paper we show that, due to the missing labels, SGG can be viewed as a "Learning from Positive and Unlabeled data" (PU learning) problem, where the reporting bias can be removed by recovering the unbiased probabilities from the biased ones by utilizing label frequencies, i.e., the per-class fraction of labeled, positive examples in all the positive examples. To obtain accurate label frequency estimates, we propose Dynamic Label Frequency Estimation (DLFE) to take advantage of training-time data augmentation and average over multiple training iterations to introduce more valid examples. Extensive experiments show that DLFE is more effective in estimating label frequencies than a naive variant of the traditional estimate, and DLFE significantly alleviates the long tail and achieves state-of-the-art debiasing performance on the VG dataset. We also show qualitatively that SGG models with DLFE produce prominently more balanced and unbiased scene graphs. The source code will be publicly available 1 .

show abstract

Section: Positive Unlabeled (Pu) Learningmentioning

confidence: 76%

“…However, their approach is time and resource consuming since it requires re-generating pseudo labels if different SGG models are used. Unlike [4], our approach not only can be easily adapted for any SGG model with minimal modification, but is superior in terms of debiasing performance.…”

Section: Positive Unlabeled (Pu) Learningmentioning

confidence: 99%

Recovering the Unbiased Scene Graphs from the Biased Ones

Chiou

Ding²,

Yan

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Yang et al [136] presented an approach for inferring accurate support relations between objects from given RGBD images of cluttered indoor scenes. Yet additional models include LinkNet [45], STL [137], CogTree [183], HetH [195], TCN-VRP [190], RONNIE [119], Px2graph [100], BLOCK [130], and MR-NET [131].…”

Section: Other Sgg Methodsmentioning

confidence: 99%

Scene Graph Generation: A Comprehensive Survey

Zhu¹,

Zhang²,

Jiang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep learning techniques have led to remarkable breakthroughs in the field of generic object detection and have spawned a lot of scene-understanding tasks in recent years. Scene graph has been the focus of research because of its powerful semantic representation and applications to scene understanding. Scene Graph Generation (SGG) refers to the task of automatically mapping an image into a semantic structural scene graph, which requires the correct labeling of detected objects and their relationships. Although this is a challenging task, the community has proposed a lot of SGG approaches and achieved good results. In this paper, we provide a comprehensive survey of recent achievements in this field brought about by deep learning techniques. We review 138 representative works that cover different input modalities, and systematically summarize existing methods of image-based SGG from the perspective of feature extraction and fusion. We attempt to connect and systematize the existing visual relationship detection methods, to summarize, and interpret the mechanisms and the strategies of SGG in a comprehensive way. Finally, we finish this survey with deep discussions about current existing problems and future research directions. This survey will help readers to develop a better understanding of the current research status and ideas.

show abstract

“…Tang et al also try to use causal analysis (Tang et al 2020) to reduce the influence of training data distribution on the final model. Some other works (Chen et al 2019a;Zhan et al 2020;Chiou et al 2021) address this issue in a positive-unlabeled learning manner, and typical imbalance learning methods, such as re-sampling and costsensitive learning, are also introduced for scene graph generation (Li et al 2021a;Yan et al 2020). Unlike these approaches, we adopt the resistance training strategy using prior bias (RTPB), which utilizes a resistance bias item for the relationship classifier during training to optimize the loss value and the classification margin of each type of relationship.…”

Section: Scene Graph Generationmentioning

confidence: 99%

Resistance Training using Prior Bias: toward Unbiased Scene Graph Generation

Chen¹,

Zhan²,

Yu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Scene Graph Generation (SGG) aims to build a structured representation of a scene using objects and pairwise relationships, which benefits downstream tasks. However, current SGG methods usually suffer from sub-optimal scene graph generation because of the long-tailed distribution of training data. To address this problem, we propose Resistance Training using Prior Bias (RTPB) for the scene graph generation. Specifically, RTPB uses a distributed-based prior bias to improve models' detecting ability on less frequent relationships during training, thus improving the model generalizability on tail categories. In addition, to further explore the contextual information of objects and relationships, we design a contextual encoding backbone network, termed as Dual Transformer (DTrans). We perform extensive experiments on a very popular benchmark, VG150, to demonstrate the effectiveness of our method for the unbiased scene graph generation. In specific, our RTPB achieves an improvement of over 10% under the mean recall when applied to current SGG methods. Furthermore, DTrans with RTPB outperforms nearly all state-ofthe-art methods with a large margin. 1 * This work was done when Chao Chen was a research intern at JD Explore Academy.† Corrsponding Author.

show abstract

Soft Transfer Learning via Gradient Diagnosis for Visual Relationship Detection

Cited by 15 publications

References 17 publications

Recovering the Unbiased Scene Graphs from the Biased Ones

Recovering the Unbiased Scene Graphs from the Biased Ones

Scene Graph Generation: A Comprehensive Survey

Resistance Training using Prior Bias: toward Unbiased Scene Graph Generation

Contact Info

Product

Resources

About