Multi-Scale Cropping Mechanism for Remote Sensing Image Captioning

Zhang, Xueting; Wang, Qi; Chen, Shangdong; Li, Xuelong

doi:10.1109/igarss.2019.8900503

Cited by 38 publications

(12 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides tailored attention mechanisms, other previous studies on remote sensing image captioning have explored alternative paths for improving the results of standard encoderdecoder neural models. These include studies (a) exploring multi-scale feature representations [15], [16]; (b) using novel loss functions [17] or training procedures based on reinforcement learning [18], improving on the standard cross-entropy loss [17]; (c) extending and combining the set of reference captions, associated to each image, through summarization [19] or retrieval [20] approaches; or (d) using decoder components based on the Transformer architecture [18].…”

Section: Related Workmentioning

confidence: 99%

Using Neural Encoder-Decoder Models With Continuous Outputs for Remote Sensing Image Captioning

Ramos

Martins

2022

IEEE Access

View full text Add to dashboard Cite

This research was supported through COST Action Multi3Generation Ref. CA18231, and also through Fundação para a Ciência e Tecnologia (FCT), namely through the FCT project grant with reference PTDC/CCI-CIF/32607/2017 (MIMU) and through the Ph.D. scholarship with reference 2020.06106.BD, as well as through the INESC-ID multi-annual funding from the PIDDAC programme with reference UIDB/50021/2020. We also gratefully acknowledge the support of NVIDIA Corporation, with the donation of the two Titan Xp GPUs used in our experiments.

show abstract

Section: Related Workmentioning

confidence: 99%

Using Neural Encoder-Decoder Models With Continuous Outputs for Remote Sensing Image Captioning

Ramos

Martins

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Also, because the training 6 Complexity labels which are texts are different from the features obtained from the images, language model techniques are required to analyze the form, meaning, and context of a sequence of words. is becomes even more complex as keywords are required to be identified for emphasizing the action or scene being described [117].…”

Section: Image Captioningmentioning

confidence: 99%

Features to Text: A Comprehensive Survey of Deep Learning on Semantic Segmentation and Image Captioning

et al. 2021

View full text Add to dashboard Cite

With the emergence of deep learning, computer vision has witnessed extensive advancement and has seen immense applications in multiple domains. Specifically, image captioning has become an attractive focal direction for most machine learning experts, which includes the prerequisite of object identification, location, and semantic understanding. In this paper, semantic segmentation and image captioning are comprehensively investigated based on traditional and state-of-the-art methodologies. In this survey, we deliberate on the use of deep learning techniques on the segmentation analysis of both 2D and 3D images using a fully convolutional network and other high-level hierarchical feature extraction methods. First, each domain’s preliminaries and concept are described, and then semantic segmentation is discussed alongside its relevant features, available datasets, and evaluation criteria. Also, the semantic information capturing of objects and their attributes is presented in relation to their annotation generation. Finally, analysis of the existing methods, their contributions, and relevance are highlighted, informing the importance of these methods and illuminating a possible research continuation for the application of semantic image segmentation and image captioning approaches.

show abstract

“…Recently, to realize the end-to-end identification of remote sensing objects and their spatial relationships, a multi-scale remote sensing image interpretation network based on a FCN, a U-Net and a LSTM is proposed [4]. In addition, Zhang et al [49] propose a training mechanism of multi-scale cropping for remote sensing image captioning based on an encoder-decoder model to improve the generalization of feature representation. Although image captioning has been applied to the semantic understanding of remote sensing images in recent years, there are still many problems to be solved.…”

Section: Remote Sensing Image Captioningmentioning

confidence: 99%

Multi-Scale Remote Sensing Semantic Analysis Based on a Global Perspective

Cui

Zhang

et al. 2019

IJGI

View full text Add to dashboard Cite

Remote sensing image captioning involves remote sensing objects and their spatial relationships. However, it is still difficult to determine the spatial extent of a remote sensing object and the size of a sample patch. If the patch size is too large, it will include too many remote sensing objects and their complex spatial relationships. This will increase the computational burden of the image captioning network and reduce its precision. If the patch size is too small, it often fails to provide enough environmental and contextual information, which makes the remote sensing object difficult to describe. To address this problem, we propose a multi-scale semantic long short-term memory network (MS-LSTM). The remote sensing images are paired into image patches with different spatial scales. First, the large-scale patches have larger sizes. We use a Visual Geometry Group (VGG) network to extract the features from the large-scale patches and input them into the improved MS-LSTM network as the semantic information, which provides a larger receptive field and more contextual semantic information for small-scale image caption so as to play the role of global perspective, thereby enabling the accurate identification of small-scale samples with the same features. Second, a small-scale patch is used to highlight remote sensing objects and simplify their spatial relations. In addition, the multi-receptive field provides perspectives from local to global. The experimental results demonstrated that compared with the original long short-term memory network (LSTM), the MS-LSTM’s Bilingual Evaluation Understudy (BLEU) has been increased by 5.6% to 0.859, thereby reflecting that the MS-LSTM has a more comprehensive receptive field, which provides more abundant semantic information and enhances the remote sensing image captions.

show abstract

Multi-Scale Cropping Mechanism for Remote Sensing Image Captioning

Cited by 38 publications

References 8 publications

Using Neural Encoder-Decoder Models With Continuous Outputs for Remote Sensing Image Captioning

Using Neural Encoder-Decoder Models With Continuous Outputs for Remote Sensing Image Captioning

Features to Text: A Comprehensive Survey of Deep Learning on Semantic Segmentation and Image Captioning

Multi-Scale Remote Sensing Semantic Analysis Based on a Global Perspective

Contact Info

Product

Resources

About