An accurate generation of image captions for blind people using extended convolutional atom neural network

Abstract

Recently, the progress on image understanding and AIC (Automatic Image Captioning) has attracted lots of researchers to make use of AI (Artificial Intelligence) models to assist the blind people. AIC integrates the principle of both computer vision and NLP (Natural Language Processing) to generate automatic language descriptions in relation to the image observed. This work presents a new assistive technology based on deep learning which helps the blind people to distinguish the food items in online grocery shopping. The proposed AIC model involves the following steps such as Data Collection, Noncaptioned image selection, Extraction of appearance, texture features and Generation of automatic image captions. Initially, the data is collected from two public sources and the selection of non-captioned images are done using the ARO (Adaptive Rain Optimization). Next, the appearance feature is extracted using SDM (Spatial Derivative and Multiscale) approach and WPLBP (Weighted Patch Local Binary Pattern) is used in the extraction of texture features. Finally, the captions are automatically generated using ECANN (Extended Convolutional Atom Neural Network). ECANN model combines the CNN (Convolutional Neural Network) and LSTM (Long Short-Term Memory) architectures to perform the caption reusable system to select the most accurate caption. The loss in the ECANN architecture is minimized using AAS (Adaptive Atom Search) Optimization algorithm. The implementation tool used is PYTHON and the dataset used for the analysis are Grocery datasets (Freiburg Groceries and Grocery Store Dataset). The proposed ECANN model acquired accuracy (99.46%) on Grocery Store Dataset and (99.32%) accuracy on Freiburg Groceries dataset. Thus, the performance of the proposed ECANN model is compared with other existing models to verify the supremacy of the proposed work over the other existing works.

Overcoming the Challenge of Cyberbullying Detection in Images: A Deep Learning Approach with Image Captioning and OCR Integration

“…T Tiwary et al [11] have introduced the Automatic Image Captioning (AIC) technique to help visually impaired consumers identify products in online grocery stores. To solve this problem, they proposed an ECANN (Extended Convolutional Atom Neural Network).…”

PerceptGuide: A Perception Driven Assistive Mobility Aid Based on Self-Attention and Multi-Scale Feature Fusion

“…Advancements in scene understanding [13], [14], [15], [16] and image captioning [17], [18], [19], [20], [21] are driving the evolution of novel assistive solutions, enabling visually impaired individuals to navigate the world more effectively. The pursuit of improved quality of life for blind and visually impaired individuals fuels the increasing research and development of enhanced assistive devices.…”

Automated Image Captioning Using Sparrow Search Algorithm With Improved Deep Learning Model

“…Various research was dedicated to automatic image captions, as well as it is categorized into various approaches [6]. The image caption technique is designated from similar images through captioning, and the retrieval-based method identifies visually similar images through the captioning from the training data [7,8].…”