Attribute Attention for Semantic Disambiguation in Zero-Shot Learning

Yang, Liu; Guo, Jishun; Cai, Deng; He, Xiaofei

doi:10.1109/iccv.2019.00680

Cited by 148 publications

(73 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The application of zero-shot in super-resolution, i.e, the ZSSR prescription [23], is among the most widely used models in superresolution and has gained increasing interest recently. In addition, the majority of zero-shot methods that have provided a large number of excellent results in recent years are mainly based on segmentation [39], emotion recognition [40], object detection [41], image retrieval [42][43][44][45], image classification [46][47][48] and intelligent learning in machines or robots [49]. In the ZSSR formalism, LR images are downsampled to generate many lower-resolution images (I = I 0 , I 1 , I 2 , ..., I n ), which serve as the HR supervision information called "HR fathers, " then, each HR father is downscaled by the required scale factor s to obtain the corresponding "LR sons.…”

Section: Zero-shotmentioning

confidence: 99%

A Single Historical Painting Super-Resolution via a Reference-Based Zero-Shot Network

Shi¹,

Xu²,

Zhang³

et al. 2021

IJCIS

View full text Add to dashboard Cite

As an important part of human cultural heritage, many ancient paintings have suffered from various deteriorations that have led to texture blurring, color fading, etc. Single image super-resolution (SISR) which aims to recover a high-resolution (HR) version from a low-resolution (LR) input is actively engaged in the digital preservation of cultural relics. Currently, only traditional superresolution is widely studied and used in cultural heritage, and it is difficult to apply learning-based SISR to unique historical paintings because of the absence of both ground truth and datasets. Fortunately, the recently proposed ZSSR method suggests that it is feasible to generate a small dataset and extract self-supervised information from a single image. However, when applied to the preservations of historical paintings, the performance of ZSSR is highly limited due to the lack of image knowledge. To address the above issues and to unleash the great potential of learning-based methods in heritage conservation, we present Ref-ZSSR, which is the first attempt to combine zero-shot and reference-based methods to achieve SISR. In our model, both global and local multi-scale similar information is fully exploited from the painting itself. In an end-to-end manner, this information provides consistent artistic style image knowledge and helps synthesize SR images with sharp texture details. Compared with the ZSSR method, our approach shows improvement in both precision (approximately 4.68 dB for scale ×2) and visual perception. It is worth mentioning that all image knowledge required in our method can be extracted from the painting itself, i.e., external examples are not required. Therefore, this approach can be easily generalized to any damaged historical paintings, broken murals, noisy old photos, incomplete art works, etc.

show abstract

Section: Zero-shotmentioning

confidence: 99%

A Single Historical Painting Super-Resolution via a Reference-Based Zero-Shot Network

Shi¹,

Xu²,

Zhang³

et al. 2021

IJCIS

View full text Add to dashboard Cite

show abstract

“…With augmented projections, we adopt the same process as in [3,4] and provide a hybrid way for inference. For visualsemantic projection, given a test image x, it can be projected to semantic space as φ(x), our goal is to assign a class label with the maximal overall compatibility score, i.e., y * = arg max c∈Y u ( φ(x), ϕ(x) ).…”

Section: Inference Algorithmmentioning

confidence: 99%

“…Depending on whether additional data is generated during the training phase, existing ZSL methods can be categorized into two groups, i.e., the generative model-based methods [2] and the correlation-based methods [3,4,5,6,7]. The first group generates features for unseen classes according to their semantic descriptions, i.e., human expert annotated attributes [1] and word embeddings [8].…”

Section: Introductionmentioning

confidence: 99%

Channel-Wise Mix-Fusion Deep Neural Networks for Zero-Shot Learning

Wang¹,

Guau²,

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Zero-shot learning (ZSL), with the assistance of the seen class image and additional semantic knowledge, generalizes its classification ability to the unseen class by aligning the visual-semantic space embeddings. Few previous methods have researched whether discriminative visual features are helpful to recognize different classes while neglecting the rich semantic information from the surrounding background. This paper proposes a channel-wise mix-fusion ZSL model (CMFZ) to contextualize the ZSL classifier's discriminative information by incorporating much richer visual semantic information from both objects and their semantic surrounding environments. In particular, the channel-wise connection module (CCM) learns to construct the relationship between the object and its surroundings. A collaborative channel-wise activation module (CAM) is adopted to learn from a more delicate scale image attained from the cropping module. It highlights the most distinct channels representing the object's discriminative regions to eliminate inadvertently introduced background noise. Furthermore, the representation ability of the learned mapping is enhanced by integrating the visual semantic features processed by CCM and CAM. Experimental results show that CMFZ outperforms the state-of-the-art ZSL methods and verifies the effectiveness of incorporating visual semantic information.

show abstract

“…The crowd counting model is computed by a joint regularization through learning the crowd pattern intrinsic distribution (geometric) structure and imposing temporal smoothness of activity patterns of the scene. The selected share attributes by the semantic relations [28] of visual attributes are used in zero-shot learning [29] and image retrieval [26]. In many learning tasks, the learning target space structure of the function holds rich relationship information that is the topology relations of the outputs of the learning function on the different inputs data points.…”

Section: Semi-supervised Learning For Attribute Recognitionmentioning

confidence: 99%

Scene Attribute Semantic Relational Regularization for Transport-Travel Scene Understanding

et al. 2020

View full text Add to dashboard Cite

Attribute learning has improved the performance in scene understanding and scene recognition. However, there are many attributes described by words or short texts in a static scene and traffic crowd scene. If there are two similar scenes, the semantic relationship topology structures of corresponding attribute groups of the two scenes are also homogeneity. But it is difficult to learn a semantic relation topology projection across semantic text data and visual data. To solve the problem, we construct approximate homeomorphism mapping based on the scene attributes semantic relational regularization. Hence, we propose a novel attribute semantic topological relationship regularization based scene attribute semantic learning(ARSL) method for scene semantic understanding. And we establish a transport and travel scene recognition model based on attribute semantic features which are achieved by the proposed ARSL algorithm. In order to verify the proposed method, the experiments are implemented on the static transporttravel scene dataset and dynamic transport-travel crowds scene dataset respectively. And the static transporttravel scene dataset is constructed by the SUN Attribute dataset including images and texts. However, the dynamic transport-travel crowds scene dataset is constructed through the WWW Crowd dataset including videos and texts, and the dynamic transport-travel crowd scene dataset is named as the WWW Crowd-Sub dataset. The performances of the proposed method are improved by 38.48% and 17.51% on the SUN Attribute dataset and WWW Crowd-Sub dataset respectively. The experimental results on the SUN Attribute dataset and WWW Crowd-Sub dataset demonstrate that the proposed approach has superior performance compared to state of the art. And it can be demonstrated that the performance of the proposed ARSL method is effective against static transport-travel scene and dynamic transport-travel crowd scene.

show abstract

Attribute Attention for Semantic Disambiguation in Zero-Shot Learning

Cited by 148 publications

References 32 publications

A Single Historical Painting Super-Resolution via a Reference-Based Zero-Shot Network

A Single Historical Painting Super-Resolution via a Reference-Based Zero-Shot Network

Channel-Wise Mix-Fusion Deep Neural Networks for Zero-Shot Learning

Scene Attribute Semantic Relational Regularization for Transport-Travel Scene Understanding

Contact Info

Product

Resources

About