Chittron: An Automatic Bangla Image Captioning System

Abstract

Automatic image caption generation aims to produce an accurate description of an image in natural language automatically. However, Bangla, the fifth most widely spoken language in the world, is lagging considerably in the research and development of such domain. Besides, while there are many established data sets to related to image annotation in English, no such resource exists for Bangla yet. Hence, this paper outlines the development of "Chittron", an automatic image captioning system in Bangla. Moreover, to address the data set availability issue, a collection of 16, 000 Bangladeshi contextual images has been accumulated and manually annotated in Bangla. This data set is then used to train a model which integrates a pre-trained VGG16 image embedding model with stacked LSTM layers. The model is trained to predict the caption when the input is an image, one word at a time. The results show that the model has successfully been able to learn a working language model and to generate captions of images quite accurately in many cases. The results are evaluated mainly qualitatively. However, BLEU scores are also reported. It is expected that a better result can be obtained with a bigger and more varied data set.

A Hybridized Deep Learning Method for Bengali Image Captioning

“…Moreover, P. Blandfort et al [32] systematically characterize diverse image captions that appear "in the wild" in order to understand how people caption images naturally. Alongside English researchers also generated captions in Chinese [15], [16], Japanese [1], Arabic [12], Bahasa Indonesia [13], Hindi [26] German [29] and Bengali [5], [23]. M. Rahman et al [23] generated image caption in Bengali for the first time followed by T. Deb et al [5].…”

“…M. Rahman et al [23] generated image caption in Bengali for the first time followed by T. Deb et al [5]. Researchers of paper [23] used VGG-16 to extract image features and stacked LSTMs. On the contrary, researchers of paper [5] generated image caption using InceptionResnetV2 or VGG-16 and LSTM.…”

“…In [23], author's first paper, was where in image captioning in Bengali followed by [5] and [37]. Rahman et al [23] have aimed to outline an automatic image captioning system in Bengali called 'Chittron'. Their model was trained to predict Bengali caption from input image one word at a time.…”

“…Research done on Image captioning using other languages [13], [15], [16] is still limited. Few works until now have been conducted on image captioning in Bengali [5], [23], [37] so we aim to explore image captioning in the Bengali language further.…”

Bornon: Bengali Image Captioning with Transformer-Based Deep Learning Approach

“…Only seven works have been done on image captioning in Bengali till now. [8] was the first paper in image captioning in Bengali followed by [9][10][11] and [12]. Rahman et al [8] have aimed to outline an automatic image captioning system in Bengali called 'Chittron'.…”