Semantic Enhanced Encoder-Decoder Network (SEN) for Video Captioning

Gui, Yuling; Guo, Dan; Zhao, Ye

doi:10.1145/3347319.3356839

Cited by 2 publications

(2 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In spite of the fact that the essential centre is on trim acknowledgement, the worldly modelling capabilities of LSTM systems adjust with their utilisation in picture captioning, where capturing transient connections between objects in a picture is vital. Guo et al [17] presented a semantic direction arranged for video captioning, emphasizing the part of consideration instruments in improving the quality of produced captions. The consideration instrument makes a difference the show centres on particular districts of intrigued inside the video outlines, making strides in the significance and instruction of the captions.…”

Section: Related Workmentioning

confidence: 99%

Recurrent Neural Networks for Image Captioning: A Case Study with LSTM

Devi K, Manish Sharma, R. N. Patil, Anurag Shrivastava

2024

jes

View full text Add to dashboard Cite

This research investigates the viability of Long Short-Term Memory (LSTM) systems, a subtype of Recurrent Neural Networks (RNNs), for picture captioning. Leveraging the MS COCO dataset, the study compares the execution of LSTM-based RNNs with Vanilla RNN, Gated Recurrent Unit (GRU), consideration components, and transformer-based models. Experimental comes about to illustrate that the LSTM-based RNN shows competitive execution, accomplishing a BLEU-4 score of 0.72, a METEOR score of 0.68, and a CIDEr score of 2.1. The comparative investigation uncovers its prevalence over Vanilla RNN and GRU, highlighting its capability to capture long-range conditions inside successive picture information. Moreover, the study investigates the effect of consideration instruments and transformer designs, exhibiting their potential improvements in the context-aware caption era. The transformer-based show outflanks all other models, accomplishing a BLEU-4 score of 0.78, a METEOR score of 0.72, and a CIDEr score of 2.5. The findings give important bits of knowledge toward the creating scene of picture captioning strategy, which makes LSTM-based RNNs solid and productive approaches for capturing worldly groupings in visual substance. In achieving these, the study provides a framework for future developments in hybrid models and manufacturing processes that push boundaries of smart image perception and understanding

show abstract

Section: Related Workmentioning

confidence: 99%

Recurrent Neural Networks for Image Captioning: A Case Study with LSTM

Devi K, Manish Sharma, R. N. Patil, Anurag Shrivastava

2024

jes

View full text Add to dashboard Cite

show abstract

“…The spatial information from the different regions was passed through a decoder with detailed features. A semantic enhanced encoder-decoder network method with LSTM was used by researchers in [24]. The method presented extracts motion features, appearance features, and global features for video description generations.…”

Section: End-to-end Modelmentioning

confidence: 99%

Att-BiL-SL: Attention-Based Bi-LSTM and Sequential LSTM for Describing Video in the Textual Formation

et al. 2021

View full text Add to dashboard Cite

With the advancement of the technological field, day by day, people from around the world are having easier access to internet abled devices, and as a result, video data is growing rapidly. The increase of portable devices such as various action cameras, mobile cameras, motion cameras, etc., can also be considered for the faster growth of video data. Data from these multiple sources need more maintenance to process for various usages according to the needs. By considering these enormous amounts of video data, it cannot be navigated fully by the end-users. Throughout recent times, many research works have been done to generate descriptions from the images or visual scene recordings to address the mentioned issue. This description generation, also known as video captioning, is more complex than single image captioning. Various advanced neural networks have been used in various studies to perform video captioning. In this paper, we propose an attention-based Bi-LSTM and sequential LSTM (Att-BiL-SL) encoder-decoder model for describing the video in textual format. The model consists of two-layer attention-based bi-LSTM and one-layer sequential LSTM for video captioning. The model also extracts the universal and native temporal features from the video frames for smooth sentence generation from optical frames. This paper includes the word embedding with a soft attention mechanism and a beam search optimization algorithm to generate qualitative results. It is found that the architecture proposed in this paper performs better than various existing state of the art models.

show abstract

Semantic Enhanced Encoder-Decoder Network (SEN) for Video Captioning

Cited by 2 publications

References 30 publications

Recurrent Neural Networks for Image Captioning: A Case Study with LSTM

Recurrent Neural Networks for Image Captioning: A Case Study with LSTM

Att-BiL-SL: Attention-Based Bi-LSTM and Sequential LSTM for Describing Video in the Textual Formation

Contact Info

Product

Resources

About