An augmented reality for an arabic text reading and visualization assistant for the visually impaired

Ouali, Imene; Halima, Mohamed Ben; Wali, Ali

doi:10.1007/s11042-023-14880-6

Cited by 7 publications

(1 citation statement)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, we propose converting traffic scene keyframes into natural language captions and using richer semantic information can replace detecting individual entities. This approach shows promise for assisting visually impaired individuals [12,23], driving safety [1], and describing traffic accidents [18].…”

Section: Introductionmentioning

confidence: 99%

TSIC-CLIP: Traffic Scene Image Captioning Model Based on Clip

Zhang,

Xu,

et al. 2024

ITC

View full text Add to dashboard Cite

Image captioning in traffic scenes presents several challenges, including imprecise caption generation, lack of personalization, and an unwieldy number of model parameters. We propose a new image captioning model for traffic scenes to address these issues. The model incorporates an adapter-based fine-tuned feature extraction part to enhance personalization and a caption generation module using global weighted attention pooling to reduce model parameters and improve accuracy. The proposed model consists of four main stages. In the first stage, the Image-Encoder extracts the global features of the input image and divides it into nine sub-regions, encoding each sub-region separately. In the second stage, the Text-Encoder encodes the text dataset to obtain text features. It then calculates the similarity between the image sub-region features and encoded text features, selecting the text features with the highest similarity. Subsequently, the pre-trained Faster RCNN model extracts local image features. The model then splices together the text features, global image features, and local image features to fuse the multimodal information. In the final stage, the extracted features are fed into the Captioning model, which effectively fuses the different features using a novel global weighted attention pooling layer. The Captioning model then generates natural language image captions. The proposed model is evaluated on the MS-COCO dataset, Flickr 30K dataset, and BUUISE-Image dataset, using mainstream evaluation metrics. Experiments demonstrate significant improvements across all evaluation metrics on the public datasets and strong performance on the BUUISE-Image traffic scene dataset.

show abstract

Section: Introductionmentioning

confidence: 99%

TSIC-CLIP: Traffic Scene Image Captioning Model Based on Clip

Zhang,

Xu,

et al. 2024

ITC

View full text Add to dashboard Cite

show abstract

Digital Muhadathah: Framework Model Development for Digital Arabic Language Learning

Hehsan,

Huda,

Mahsun

et al. 2024

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

The use of CNNs in VR/AR/MR/XR: a systematic literature review

Cortes,

Bermejo,

Juiz

2024

Virtual Reality

View full text Add to dashboard Cite

This study offers a systematic literature review on the application of Convolutional Neural Networks in Virtual Reality, Augmented Reality, Mixed Reality, and Extended Reality technologies. We categorise these applications into three primary classifications: interaction, where the networks amplify user engagements with virtual and augmented settings; creation, showcasing the networks’ ability to assist in producing high-quality visual representations; and execution, emphasising the optimisation and adaptability of apps across diverse devices and situations. This research serves as a comprehensive guide for academics, researchers, and professionals in immersive technologies, offering profound insights into the cross-disciplinary realm of network applications in these realities. Additionally, we underscore the notable contributions concerning these realities and their intersection with neural networks.

show abstract

An augmented reality for an arabic text reading and visualization assistant for the visually impaired

Cited by 7 publications

References 39 publications

TSIC-CLIP: Traffic Scene Image Captioning Model Based on Clip

TSIC-CLIP: Traffic Scene Image Captioning Model Based on Clip

Digital Muhadathah: Framework Model Development for Digital Arabic Language Learning

The use of CNNs in VR/AR/MR/XR: a systematic literature review

Contact Info

Product

Resources

About