A closer look at referring expressions for video object segmentation

Bellver, Miriam; Ventura, Carles; Silberer, Carina; Kazakos, Ioannis; Torres, Jordi; Giró-i-Nieto, Xavier

doi:10.1007/s11042-022-13413-x

Cited by 17 publications

(2 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The first stage extracts language and image feature, respectively, and the second stage fuses the multi-modal features to predict the segmentation result. In the first stage, these methods usually use ResNet [5,34], DeeplabV3 [28,35,36] or Darknet [6] to extract image feature, and use LSTM [9,10,36], simple recurrent units [37] or Transformer [35,38] to extract language features. In the second stage, most methods construct cross-modal decoder to fuse image and language features.…”

Section: Referring Image Segmentationmentioning

confidence: 99%

Cross‐modal fusion encoder via graph neural network for referring image segmentation

Zhang,

Piao

et al. 2024

IET Image Processing

View full text Add to dashboard Cite

Referring image segmentation identifies the object masks from images with the guidance of input natural language expressions. Nowadays, many remarkable cross‐modal decoder are devoted to this task. But there are mainly two key challenges in these models. One is that these models usually lack to extract fine‐grained boundary information and gradient information of images. The other is that these models usually lack to explore language associations among image pixels. In this work, a Multi‐scale Gradient balanced Central Difference Convolution (MG‐CDC) and a Graph convolutional network‐based Language and Image Fusion (GLIF) for cross‐modal encoder, called Graph‐RefSeg, are designed. Specifically, in the shallow layer of the encoder, the MG‐CDC captures comprehensive fine‐grained image features. It could enhance the perception of target boundaries and provide effective guidance for deeper encoding layers. In each encoder layer, the GLIF is used for cross‐modal fusion. It could explore the correlation of every pixel and its corresponding language vectors by a graph neural network. Since the encoder achieves robust cross‐modal alignment and context mining, a light‐weight decoder could be used for segmentation prediction. Extensive experiments show that the proposed Graph‐RefSeg outperforms the state‐of‐the‐art methods on three public datasets. Code and models will be made publicly available at https://github.com/ZYQ111/Graph_refseg.

show abstract

Section: Referring Image Segmentationmentioning

confidence: 99%

Cross‐modal fusion encoder via graph neural network for referring image segmentation

Zhang,

Piao

et al. 2024

IET Image Processing

View full text Add to dashboard Cite

show abstract

“…Image information undergoes contextual self calibration structure operations to obtain deeper features, including a large number of spatial and semantic features [28][29]. However, this approach solely extracts contextual features, resulting in incomplete information features, leading to significant feature information waste, ultimately compromising the network's output.…”

mentioning

confidence: 99%

Interactive Image Segmentation Technique Based on Improved Residual Network

Yang,

Geng

2023

IEEE Access

View full text Add to dashboard Cite

Interactive image segmentation offers useful guidance to users and can be applied in practical settings for production and daily life purposes. Nonetheless, the technology's intrinsic limitations, including complicated interaction methods and high error rates, have impeded its further advancement with the development of computer vision. To address the issue, this study introduces a method to determine the extreme point of center point prediction. The target center serves as the "symmetric center" of the extreme point, which facilitates the search for other extreme points. Furthermore, the Canny algorithm is combined to achieve edge image detection. Moreover, the residual network is enhanced through embedding a pre-activation step, introducing a BatchNorm layer, and adding a pyramid scene parsing network. Finally, the performance of this method is verified by analyzing its Intersection over Union, segmentation accuracy, efficiency, F1 value, and other indicators. The results show that on the Pascalvoc2012 dataset, the segmentation accuracy obtained through the extreme point method can reach over 90%. The addition of the pyramid scene analysis network stabilizes its accuracy on urban landscape datasets between 92% and 96%. When the proposed image segmentation method is applied to the Grabcut dataset, its union intersection can reach 88.7%. On the self-generated complex daily scenery dataset, this method achieves segmentation accuracy of 95% with superior stability and precision. This provides a fresh methodological reference for optimizing interactive image segmentation technology further.

show abstract