Multi-label video classification via coupling attentional multiple instance learning with label relation graph

Li, Xuewei; Wu, Hongjun; Li, Mengzhu; Liu, Hongzhe

doi:10.1016/j.patrec.2022.01.003

Cited by 14 publications

(9 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In graph convolutional networks (GCNs) there are many ways to employ convolutional layer operators. Based on these variations alternative models are proposed such as the graph transformer network 27 , graph isomorphism network 28 , multi-attention label relation learning CNN 29 , causal incremental GCN 30 , principal neighborhood aggregation-based GCN 31 , among other approaches. Since GCNs do not assign different weights to neighboring nodes, graph attention network (GAT) gains ground as it takes different neighboring nodes into account 32 .…”

Section: Related Workmentioning

confidence: 99%

Interference recommendation for the pump sizing process in progressive cavity pumps using graph neural networks

Starke,

Hoppe,

Sartori

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Pump sizing is the process of dimensional matching of an impeller and stator to provide a satisfactory performance test result and good service life during the operation of progressive cavity pumps. In this process, historical data analysis and dimensional monitoring are done manually, consuming a large number of man-hours and requiring a deep knowledge of progressive cavity pump behavior. This paper proposes the use of graph neural networks in the construction of a prototype to recommend interference during the pump sizing process in a progressive cavity pump. For this, data from different applications is used in addition to individual control spreadsheets to build the database used in the prototype. From the pre-processed data, complex network techniques and the betweenness centrality metric are used to calculate the degree of importance of each order confirmation, as well as to calculate the dimensionality of the rotors. Using the proposed method a mean squared error of 0.28 is obtained for the cases where there are recommendations for order confirmations. Based on the results achieved, it is noticeable that there is a similarity of the dimensions defined by the project engineers during the pump sizing process, and this outcome can be used to validate the new design definitions.

show abstract

Section: Related Workmentioning

confidence: 99%

Interference recommendation for the pump sizing process in progressive cavity pumps using graph neural networks

Starke,

Hoppe,

Sartori

et al. 2023

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Multi-label classification methods assign tags from a predefined word set to videos [4,15,36]. Sophisticated methods leverage extra information besides video content, such as tag graph with knowledge [3,9,14,38], query log 1 https://github.com/SCZwangxiao/RADAR-MM2022.git [21], user behavior [2], and user profile [11,33]. Among all the above extra information, tag graph has attracted the most attention.…”

Section: Related Work 21 Video Taggingmentioning

confidence: 99%

“…• Video tagging methods: ML-GCN [3], NeXtVLAD [15,36], CMA [38], and MALL-CNN [14]. We compared them to demonstrate the effectiveness of incorporating extra social networks and tag graph information.…”

Section: Baselinesmentioning

confidence: 99%

See 1 more Smart Citation

Micro-video Tagging via Jointly Modeling Social Influence and Tag Relation

Wang,

Gan,

Wei

et al. 2023

Preprint

View full text Add to dashboard Cite

The last decade has witnessed the proliferation of micro-videos on various user-generated content platforms. According to our statistics, around 85.7% of micro-videos lack annotation. In this paper, we focus on annotating micro-videos with tags. Existing methods mostly focus on analyzing video content, neglecting users' social influence and tag relation. Meanwhile, existing tag relation construction methods suffer from either deficient performance or low tag coverage. To jointly model social influence and tag relation, we formulate micro-video tagging as a link prediction problem in a constructed heterogeneous network. Specifically, the tag relation (represented by tag ontology) is constructed in a semi-supervised manner. Then, we combine tag relation, video-tag annotation, and user follow relation to build the network. Afterward, a better video and tag representation are derived through Behavior Spread modeling and visual and linguistic knowledge aggregation. Finally, the semantic similarity between each micro-video and all candidate tags is calculated in this video-tag network. Extensive experiments on industrial datasets of three verticals verify the superiority of our model compared with several state-of-the-art baselines.

show abstract

“…Treating each label independently [12] ignores their potential correlations as well as increases training times significantly with each additional label. Several works investigate multi-label recognition problems using graph learning approaches to model label correlations and co-occurrences [30,47]. However, such approaches do not learn from label similarities between samples and do not project these similarities to the latent representation space.…”

Section: Learning Representations and Label Relationsmentioning

confidence: 99%

Learning from Label Relationships in Human Affect

Foteinopoulou,

Patras

2022

Preprint

View full text Add to dashboard Cite

Human affect and mental state estimation in an automated manner, face a number of difficulties, including learning from labels with poor or no temporal resolution, learning from few datasets with little data (often due to confidentiality constraints) and, (very) long, in-the-wild videos. For these reasons, deep learning methodologies tend to overfit, that is, arrive at latent representations with poor generalisation performance on the final regression task. To overcome this, in this work, we introduce two complementary contributions. First, we introduce a novel relational loss for multilabel regression and ordinal problems that regularises learning and leads to better generalisation. The proposed loss uses label vector inter-relational information to learn better latent representations by aligning batch label distances to the distances in the latent feature space. Second, we utilise a two-stage attention architecture that estimates a target for each clip by using features from the neighbouring clips as temporal context. We evaluate the proposed methodology on both continuous affect and schizophrenia severity estimation problems, as there are methodological and contextual parallels between the two. Experimental results demonstrate that the proposed methodology outperforms the baselines that are trained using the supervised regression loss, as well as pre-training the network architecture with an unsupervised contrastive loss. In the domain of schizophrenia, the proposed methodology outperforms previous state-of-the-art by a large margin, achieving a PCC of up to 78% performance close to that of human experts (85%) and much higher than previous works (uplift of up to 40%). In the case of affect recognition, we outperform previous vision-based methods in terms of CCC on both the OMG and the AMIGOS datasets. Specifically for AMIGOS, we outperform previous SoTA CCC for both arousal and valence by 9% and 13% respectively, and in the OMG dataset we outperform previous vision works by up to 5% for both arousal and valence.

show abstract

Multi-label video classification via coupling attentional multiple instance learning with label relation graph

Cited by 14 publications

References 7 publications

Interference recommendation for the pump sizing process in progressive cavity pumps using graph neural networks

Interference recommendation for the pump sizing process in progressive cavity pumps using graph neural networks

Micro-video Tagging via Jointly Modeling Social Influence and Tag Relation

Learning from Label Relationships in Human Affect

Contact Info

Product

Resources

About