Text to multi-object images synthesis based on non-local self-attention

Abstract

Deep convolutional network (CNN) can make the pictures generated by GAN more reasonable, but limited by the local receptive field of CNN, there are still many unreasonable places in the authenticity and semantics of the multi-object images generated from text. Therefore, a GAN-based method that incorporates a non-local self-attention mechanism is proposed. By embedding a non-local self-attention structure in the network, the network obtains global semantic information and detailed features, and uses the obtained information to perform level-by-level encoding to generate the final relatively reasonable image. The amount of parameters and calculation of the entire model is also reduced a lot. The proposed method is verified on the public COCO-stuff dataset and uses multiple indicators such as Inception Score, FID and classification accuracy score to evaluate the authenticity and diversity of the generated images. Experimental results show that the quality of the generated images is superior to that of previously proposed methods.