Semantic‐meshed and content‐guided transformer for image captioning

Abstract

The transformer architecture has been the dominant framework for today's image captioning tasks because of its superior performance. However, existing methods based on transformer often lack the integrated use of multi-level semantic information and are weak in maintaining the relevance of captions to the image. In this paper, a semanticmeshed and content-guided transformer network is introduced for image captioning to solve these problems. The semantic-meshed mechanism allows the model to generate words by selecting semantic information of multiple interaction levels adaptively through attention-based reconstruction. And the content-guided module guides the words generation by using attribute features that represent the image content, which aims to keep the generated caption consistent with the main content of the image. Experiments on dataset on the MSCOCO captioning dataset are conducted to validate the authors' model and achieve superior results compared to other state-of-the-art method approaches.

The Application of FLICSP Algorithm Based on Multi-Feature Fusion in Image Saliency Detection

“…Computer vision is an important branch of computer technology, and it is a complex field. An important task of computer vision is to process the collected image and video information, and the processing effect can be similar to that of human processing [1][2][3]. This technology has broad applications, which involve high processing requirements, and some tasks require human assistance.…”