Automatic Image Caption Generation Using Deep Learning

Abstract

Image captioning is an interesting and challenging task with applications in diverse domains such as image retrieval, organizing and locating images of users’ interest etc. It has huge potential for replacing manual caption generation for images and is especially suitable for large scale image data. Recently, deep neural network based methods have achieved great success in the field of computer vision, machine translation and language generation. In this paper, we propose an encoder-decoder based model that is capable of generating grammatically correct captions for images. This model makes use of VGG16 Hybrid Places 1365 as encoder and LSTM as decoder. To ensure the complete ground truth accuracy, the model is trained on the labelled Flickr8k and MSCOCO Captions datasets. Further, the model is evaluated using all standard metrics such as BLEU, METEOR, GLEU and ROUGE L. Experimental results indicate that the proposed model obtained a BLEU-1 score 0.6666, METEOR score 0.5060 and GLEU score 0.2469 on Flickr8k dataset and BLEU-1 score 0.7350, METEOR score 0.4768 and GLEU score 0.2798 on the MSCOCO Captions dataset. Thus, the proposed method achieved a significant performance as compared to the state-of-art approaches. To evaluate the efficacy of the model further, we also show the results of a caption generation from live sample images that reinforces the validity of the proposed approach.

Transforming Healthcare: Leveraging Vision-Based Neural Networks for Smart Home Patient Monitoring

“…Moreover, the proposed transformer model has demonstrated superior performance (Table 1) in terms of the BLEU-4 score (0.71) and METEOR score (0.81), indicating higher accuracy and fluency in caption generation compared with the traditional CNN model proposed by Akash Verma et al [21]. The authors of the study demonstrated that BLEU-4 score of the generated picture was (0.66) and a METEOR score of (0.50) using the "Flickr8k" dataset.…”

Study and Development of Image Caption Generation using Various Encoders for Different Image Categories

“…Traditional methods relied upon search based and template based techniques which came with the drawback of major dependency on the datasets to generate captions [11] [12]. On the other hand, the deep learning methods turned the direction by introducing encoder-decoder framework [13], attention based model, reinforcement learning [25] and so on.…”

“…The different datasets such as Flickr8k, Flickr30k, MSCOCO and Pascal1K used for training the model[24]. Verma et al has described neural network-based model for automatic image captioning and has presented BLEU scores comparison of proposed model with other existing models for different images[25].…”

“…[25].Luo et al has presented image captioning using different models and compared their BLEU scores. The different datasets such as Flickr8k, Flickr30k, MSCOCO and Pascal1K used for training the model[24].…”