Instance-Specific Feature Propagation for Referring Segmentation

Liu, Chang; Jiang, Xudong; Ding, Henghui

doi:10.1109/tmm.2022.3163578

Cited by 32 publications

(12 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It outperforms LAVT in most datasets slightly, but our method still outperforms it in UNC, UNC+ and G-Ref(Google) datasets. We compare our method with 11 state-of-the-art referring image segmentation methods in the validation set of UNC from 2019-2022, including CMSA [13], STEP [59], CMPC [11], LSCM [39], MCN [6], LELA-CLAF [5], CEFNet [42], VLT [55], ISF-FPN [56], CRIS [57] and LAVT [19]. Table 2 shows the qualitative comparison results using Prec@X and mIoU in UNC dataset.…”

Section: Quantitative Resultsmentioning

confidence: 99%

Cross‐modal fusion encoder via graph neural network for referring image segmentation

Zhang,

Piao

et al. 2024

IET Image Processing

View full text Add to dashboard Cite

Referring image segmentation identifies the object masks from images with the guidance of input natural language expressions. Nowadays, many remarkable cross‐modal decoder are devoted to this task. But there are mainly two key challenges in these models. One is that these models usually lack to extract fine‐grained boundary information and gradient information of images. The other is that these models usually lack to explore language associations among image pixels. In this work, a Multi‐scale Gradient balanced Central Difference Convolution (MG‐CDC) and a Graph convolutional network‐based Language and Image Fusion (GLIF) for cross‐modal encoder, called Graph‐RefSeg, are designed. Specifically, in the shallow layer of the encoder, the MG‐CDC captures comprehensive fine‐grained image features. It could enhance the perception of target boundaries and provide effective guidance for deeper encoding layers. In each encoder layer, the GLIF is used for cross‐modal fusion. It could explore the correlation of every pixel and its corresponding language vectors by a graph neural network. Since the encoder achieves robust cross‐modal alignment and context mining, a light‐weight decoder could be used for segmentation prediction. Extensive experiments show that the proposed Graph‐RefSeg outperforms the state‐of‐the‐art methods on three public datasets. Code and models will be made publicly available at https://github.com/ZYQ111/Graph_refseg.

show abstract

Section: Quantitative Resultsmentioning

confidence: 99%

Cross‐modal fusion encoder via graph neural network for referring image segmentation

Zhang,

Piao

et al. 2024

IET Image Processing

View full text Add to dashboard Cite

show abstract

“…• Referring VOS. Referring video object segmentation [44,45,46,47,48,49,50] is an emerging setting that involves multi-modal information. It gives a natural language expression to indicate the target object and aims at segmenting the target object throughout the video clips.…”

Section: Video Object Segmentation (Vos)mentioning

confidence: 99%

MOSE: A New Dataset for Video Object Segmentation in Complex Scenes

Ding¹,

Liu²,

He³

et al. 2023

Preprint

View full text Add to dashboard Cite

4 ByteDance https://henghuiding.github.io/MOSE Figure 1. Examples of video clips from the coMplex video Object SEgmentation (MOSE) dataset. The selected target objects are masked in orange ◼. The most notable feature of MOSE is complex scenes, including the disappearance-reappearance of objects, small/inconspicuous objects, heavy occlusions, crowded environments, etc. For example, the target player in the 2nd row turns around when reappearing in the 4th and 5th columns after disappearing in the 3rd column, bringing challenges in re-identifying him. Most videos in MOSE contain crowded and occluded objects with the target object seldom being the salient one. The goal of MOSE dataset is to provide a platform that promotes the development of more comprehensive and robust video object segmentation algorithms.

show abstract

“…Model Pretraining. Different from model pretraining of 2D vision tasks [17,[21][22][23][40][41][42]46,59], recent researches on deep model pretraining for understanding 3D objects mainly focus on unsupervised learning methods. Typical unsupervised tasks include self-reconstruction [2,24,28,30,76] and self-supervised learning pretext tasks [25,54,57,66,74].…”

Section: Deep Learning For 3d Object Understandingmentioning

confidence: 99%

Primitive3D: 3D Object Dataset Synthesis from Randomly Assembled Primitives

Li¹,

Ding²,

Tong³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Numerous advancements in deep learning can be attributed to the access to large-scale and well-annotated datasets. However, such a dataset is prohibitively expensive in 3D computer vision due to the substantial collection cost. To alleviate this issue, we propose a cost-effective method for automatically generating a large amount of 3D objects with annotations. In particular, we synthesize objects simply by assembling multiple random primitives. These objects are thus auto-annotated with part labels originating from primitives. This allows us to perform multi-task learning by combining the supervised segmentation with unsupervised reconstruction. Considering the large overhead of learning on the generated dataset, we further propose a dataset distillation strategy to remove redundant samples regarding a target dataset. We conduct extensive experiments for the downstream tasks of 3D object classification. The results indicate that our dataset, together with multitask pretraining on its annotations, achieves the best performance compared to other commonly used datasets. Further study suggests that our strategy can improve the model performance by pretraining and fine-tuning scheme, especially for the dataset with a small scale. In addition, pretraining with the proposed dataset distillation method can save 86% of the pretraining time with negligible performance degradation. We expect that our attempt provides a new datacentric perspective for training 3D deep models.

show abstract

Instance-Specific Feature Propagation for Referring Segmentation

Cited by 32 publications

References 56 publications

Cross‐modal fusion encoder via graph neural network for referring image segmentation

Cross‐modal fusion encoder via graph neural network for referring image segmentation

MOSE: A New Dataset for Video Object Segmentation in Complex Scenes

Primitive3D: 3D Object Dataset Synthesis from Randomly Assembled Primitives

Contact Info

Product

Resources

About