Few shot object detection for headdresses and seats in Thangka Yidam based on ResNet and deformable convolution

Hu, Wenjin; Xue, Ping; He, Guoyuan; Tang, Huiyuan; Song, Huafei; Yue, Changtao

doi:10.1080/09540091.2022.2041554

Cited by 5 publications

(1 citation statement)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Currently, most scholars primarily focus on researching Thangka images from the perspectives of image restoration [2,3], object detection [4,5], and headgear segmentation [6,7], and overlook the study of Thangka images from a semantic viewpoint. Existing literature indicates a scarcity of research on image description generation for Thangka im-ages.…”

Section: Introductionmentioning

confidence: 99%

WITHDRAWN: Thangka Image Caption Generation Method Combining Multi-scale and Multi-level Aggregation

YUE,

Hu,

Zhang

et al. 2024

Preprint

View full text Add to dashboard Cite

Thangka, as a distinctive form of painting in China, plays a crucial role in facilitating a more profound appreciation and comprehension of Thangka through automated image description. Considering the diverse semantic objects and varying scales present in Thangka images, as well as their distinct spatial distribution characteristics, along with the challenge of potential information loss in image key features using Transformer-based encoding layers, this paper proposes a novel approach for generating Thangka descriptions, integrating multi-scale and multi-level aggregation. The proposed method, named Multi-scale and Multi-level Aggregation (MMA), addresses these challenges and enhances the quality of Thangka image description. At the encoding stage, we employ asymmetric convolutions to enhance the spatial information-capturing capability of convolutional layers. Additionally, we utilize a pyramid pooling module to further integrate multi-scale contextual information from both global and local regions of Thangka images, resulting in feature representations that possess rich semantic information. In the decoding stage, a multi-level aggregation network is designed to aggregate features from different encoding layers, thereby improving the utilization of semantic information from higher-level encoding layers and content information from lower-level encoding layers. This effectively addresses the issue of information loss. The experimental results demonstrate that the proposed model achieves promising performance on the Thangka dataset. Compared to the NIC model, it achieves a significant improvement of 26.7% in BLEU-4 and 0.9% in METEOR, while generating descriptions with higher accuracy.

show abstract