Double Attention for Multi-Label Image Classification

Zhao, Haiying; Zhou, Wei; Hou, Xiaogang; Zhu, Hui

doi:10.1109/access.2020.3044446

Cited by 15 publications

(5 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is calculated by the weighted harmonic average of precision and recall rate. Therefore, it is a comprehensive calculation method for prediction results and samples [ 17 , 18 ], expressed as

…”

Section: Methodsmentioning

confidence: 99%

Clothing Design Style Recommendation Using Decision Tree Algorithm Combined with Deep Learning

Yang¹

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Most clothing recommendation methods have problems such as high resource consumption and inconsistent subjectively labeled clothing labels. Based on this, a multilabel classification algorithm based on deep learning (DL) theory is introduced, based on which the clothing style recognition model is constructed. Next, the concept of the decision tree algorithm is given, and the clothing recommendation model is built based on this algorithm. Moreover, the clothing style recognition model based on a multilabel classification algorithm and the clothing recommendation system based on a decision tree algorithm are tested by building simulation experiments and combining neural network technology. Finally, the application of the decision tree algorithm and DL theory in clothing recommendation design is studied through the literature collection method. The research focus is to realize the recognition of clothing through decision tree algorithm and DL method to achieve the intelligent recommendation of clothing style. The results show that: (1) the neural network technology in DL theory can realize efficient recognition and classification of clothing style by automatically extracting image features and combining with a multilabel classification algorithm. (2) The decision tree algorithm can make an initial recommendation according to users’ style preferences, then make implicit recommendations through user retrieval, browsing, and other operations, and make dynamic clothing style recommendations to users. (3) When the neural network based on a multilabel classification algorithm is trained, the precision, recall rate, and F1 values are 0.73, 0.43, and 0.55, respectively. (4) After using the clothing recommendation system based on the decision tree algorithm, the subjects’ average satisfaction is 86.25%, indicating that this system can give users a better clothing recommendation experience. This exploration aims to provide a crucial reference for further improving the quality of clothing recommendation services. It has important theoretical significance and practical value for the development of artificial intelligence in the field of fashion design, and is expected to provide a reference for the development of bionics.

show abstract

“…It is calculated by the weighted harmonic average of precision and recall rate. Therefore, it is a comprehensive calculation method for prediction results and samples [ 17 , 18 ], expressed as

…”

Section: Methodsmentioning

confidence: 99%

Clothing Design Style Recommendation Using Decision Tree Algorithm Combined with Deep Learning

Yang¹

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…Image Multi-Label Classiication. Convolutional neural network (CNN) has achieved great success in the image multi-label classiication tasks [4,6,24,55,57,60]. For example, DELTA [57] obtains global prior information and local instance information by the multi-instance network and the global priors network, so as to make full use of global and local information for multi-label classiication task.…”

Section: Related Workmentioning

confidence: 99%

“…In addition, RIA [20] compares the efects of four diferent label input ordering on RNN, and inds the label ordering in the training stage has a great inluence on the model performance. Moreover, some researchers [15,60,62] apply attention mechanisms to multi-label classiication.…”

Section: Related Workmentioning

confidence: 99%

“…For instance, SRN [62] exploits spatial regularization network to learn the attention maps among all labels, and excavates the potential relationships among labels through learnable convolution. DAN [60] exploits spatial attention to learn the relationship between image regions and labels, and utilizes channel attention to capture the correlation between labels. However, the above methods fail to fully capture the correlation between labels.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Double Attention Based on Graph Attention Network for Image Multi-Label Classification

Zhou

Xia

Dou

et al. 2023

ACM Trans. Multimedia Comput. Commun. Appl.

Self Cite

View full text Add to dashboard Cite

The task of image multi-label classification is to accurately recognize multiple objects in an input image. Most of the recent works need to leverage the label co-occurrence matrix counted from training data to construct the graph structure, which are inflexible and may degrade model generalizability. In addition, these methods fail to capture the semantic correlation between the channel feature maps to further improve the model performance. To address these issues, we propose a D ouble A ttention framework based on G raph A ttention ne T work (DA-GAT) to effectively learn the correlation between labels from training data. Firstly, we devise a new channel attention mechanism to enhance the semantic correlation between channel feature maps, so as to implicitly capture the correlation between labels. Secondly, we propose a new label attention mechanism to avoid the adverse impact of manually constructed label co-occurrence matrix. It only needs to leverage the label embedding as the input of network, and then automatically constructs the label relation matrix to explicitly establish the correlation between labels. Finally, we effectively fuse the output of these two attention mechanisms to further improve the model performance. Extensive experiments are conducted on three public multi-label classification benchmarks. Our DA-GAT model achieves the mAPs of 87.1%, 96.6%, and 64.3% on MS-COCO 2014, PASCAL VOC 2007, and NUS-WIDE respectively, and obviously outperforms other existing state-of-the-art methods. In addition, visual analysis experiments demonstrate that each attention mechanism can capture the correlation between labels well, and significantly promote the model performance.

show abstract

“…The burger buns and bagel buns have a difference of ~100 calories (Nutritionix, 2023 ), and hence, positioning the recipes in the right cluster is essential. Several studies (George and Floerkemeier, 2014 ; Silva et al, 2020 ; Zhao et al, 2020 ) have explored food classification as a multi-label problem that will require extensive manual annotations of food class labels. This problem requires additional knowledge about the recipes besides class names to identify similar recipes.…”

Section: Introductionmentioning

confidence: 99%

Ki-Cook: clustering multimodal cooking representations through knowledge-infused learning

et al. 2023

View full text Add to dashboard Cite

Cross-modal recipe retrieval has gained prominence due to its ability to retrieve a text representation given an image representation and vice versa. Clustering these recipe representations based on similarity is essential to retrieve relevant information about unknown food images. Existing studies cluster similar recipe representations in the latent space based on class names. Due to inter-class similarity and intraclass variation, associating a recipe with a class name does not provide sufficient knowledge about recipes to determine similarity. However, recipe title, ingredients, and cooking actions provide detailed knowledge about recipes and are a better determinant of similar recipes. In this study, we utilized this additional knowledge of recipes, such as ingredients and recipe title, to identify similar recipes, emphasizing attention especially on rare ingredients. To incorporate this knowledge, we propose a knowledge-infused multimodal cooking representation learning network, Ki-Cook, built on the procedural attribute of the cooking process. To the best of our knowledge, this is the first study to adopt a comprehensive recipe similarity determinant to identify and cluster similar recipe representations. The proposed network also incorporates ingredient images to learn multimodal cooking representation. Since the motivation for clustering similar recipes is to retrieve relevant information for an unknown food image, we evaluated the ingredient retrieval task. We performed an empirical analysis to establish that our proposed model improves the Coverage of Ground Truth by 12% and the Intersection Over Union by 10% compared to the baseline models. On average, the representations learned by our model contain an additional 15.33% of rare ingredients compared to the baseline models. Owing to this difference, our qualitative evaluation shows a 39% improvement in clustering similar recipes in the latent space compared to the baseline models, with an inter-annotator agreement of the Fleiss kappa score of 0.35.

show abstract

Double Attention for Multi-Label Image Classification

Cited by 15 publications

References 39 publications

Clothing Design Style Recommendation Using Decision Tree Algorithm Combined with Deep Learning

Clothing Design Style Recommendation Using Decision Tree Algorithm Combined with Deep Learning

Double Attention Based on Graph Attention Network for Image Multi-Label Classification

Ki-Cook: clustering multimodal cooking representations through knowledge-infused learning

Contact Info

Product

Resources

About