Region-Object Relation-Aware Dense Captioning via Transformer

Shao, Zhuang; Han, Jungong; Marnerides, Demetris; Debattista, Kurt

doi:10.1109/tnnls.2022.3152990

Cited by 75 publications

(35 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared to handcrafted features, deep features are more discriminative. With deep features, the computer vision tasks make a great progress in recent years, such as detection [24], [44], [57], segmentation [9], [55], and classification [58], [60]. These methods improve feature representation via better network or architecture design.…”

Section: Feature Representation and Distillationmentioning

confidence: 99%

ESGN: Efficient Stereo Geometry Network for Fast 3D Object Detection

Gao

Pang

Nie

et al. 2024

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

show abstract

Section: Feature Representation and Distillationmentioning

confidence: 99%

ESGN: Efficient Stereo Geometry Network for Fast 3D Object Detection

Gao

Pang

Nie

et al. 2024

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

show abstract

“…Wang et al [19] leverage a graph neural network to implicitly model the visual relationship among the objects/regions of interest in an image, which does not need pre-defined relationship classes. Shao et al [20] propose a region-object correlation score unit to measure the importance of each region via transformer-based architecture.…”

Section: End-to-end Dense Video Captioningmentioning

confidence: 99%

“…Shao et al. [20] propose a region‐object correlation score unit to measure the importance of each region via transformer‐based architecture.…”

Section: Related Workmentioning

confidence: 99%

A latent topic‐aware network for dense video captioning

Xu,

Cui,

et al. 2023

IET Computer Vision

View full text Add to dashboard Cite

Multiple events in a long untrimmed video possess the characteristics of similarity and continuity. These characteristics can be considered as a kind of topic semantic information, which probably behaves as same sports, similar scenes, same objects etc. Inspired by this, a novel latent topic‐aware network (LTNet) is proposed in this article. The LTNet explores potential themes within videos and generates more continuous captions. Firstly, a global visual topic finder is employed to detect the similarity among events and obtain latent topic‐level features. Secondly, a latent topic‐oriented relation learner is designed to further enhance the topic‐level representations by capturing the relationship between each event and the video themes. Benefiting from the finder and the learner, the caption generator is capable of predicting more accurate and coherent descriptions. The effectiveness of our proposed method is demonstrated on ActivityNet Captions and YouCook2 datasets, where LTNet shows a relative performance of over 3.03% and 0.50% in CIDEr score respectively.

show abstract

“…[14][15][16][17][18] We have seen that there has been massive progress in vision tasks, whether related to classification 19 work (action classification, image classification, and attribute classification), [20][21][22][23][24] or related to recognition work (object and scene recognition). [25][26][27][28][29] Generating an automatic description of the image is a new work. As we know, most communication between machines and humans depends on natural language understanding and explaining.…”

Section: Image Captioningmentioning

confidence: 99%

“…Mainly image captioning has been tackled via RNNs models, 9‐13 and Long Short‐Term Memory (LSTMs) 14‐18 . We have seen that there has been massive progress in vision tasks, whether related to classification 19 work (action classification, image classification, and attribute classification), 20‐24 or related to recognition work (object and scene recognition) 25‐29 …”

Section: Introductionmentioning

confidence: 99%

Deep learning and knowledge graph for image/video captioning: A review of datasets, evaluation metrics, and methods

Wajid,

Terashima‐Marin,

Najafirad

et al. 2023

Engineering Reports

View full text Add to dashboard Cite

Generating an image/video caption has always been a fundamental problem of Artificial Intelligence, which is usually performed using the potential of Deep Learning Methods, Computer Vision, Knowledge Graphs, and Natural Language Processing (NLP). The significant task of image/video captioning is to describe visual content in terms of natural language. Due to a semantic gap, this presents a massive problem in understanding and explaining images or videos syntactically and semantically. The current systems need somewhere to fill the gap between low‐level and high‐level features while mapping. Therefore, to tackle this problem, there is a need to describe the latest research and methods to overcome difficulties and to propose effective solutions. This work thoroughly analyses and investigates the most related methods (deep learning and knowledge graph‐based approaches), benchmark datasets, and evaluation metrics with their benefits and limitations. Here we have also reviewed the state‐of‐the‐art methods related to image/video captioning and their applications in the current scenario. Finally, we provide thorough information on existing research with comparisons of results on benchmark datasets. We have also mentioned the existing challenges and future direction of research.

show abstract

Region-Object Relation-Aware Dense Captioning via Transformer

Cited by 75 publications

References 35 publications

ESGN: Efficient Stereo Geometry Network for Fast 3D Object Detection

ESGN: Efficient Stereo Geometry Network for Fast 3D Object Detection

A latent topic‐aware network for dense video captioning

Deep learning and knowledge graph for image/video captioning: A review of datasets, evaluation metrics, and methods

Contact Info

Product

Resources

About