Two-Stage Label Embedding via Neural Factorization Machine for Multi-Label Classification

Chen, Chen; Wang, Haobo; Liu, Weiwei; Zhao, Xingyuan; Hu, Tianlei; Chen, Gang

doi:10.1609/aaai.v33i01.33013304

Cited by 29 publications

(9 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use three datasets: CIFAR-100 [13,21], ImageNet-Subset [7], and TinyImageNet [35] in our experiments. For a fair comparison with baseline class-incremental learning methods [1,11,17,37,43,49] in the FCIL setting, we follow the same protocols proposed by [37,49] to set incremental tasks, utilize the identical class order generated from iCaRL [37], and employ the same backbone (i.e., ResNet-18 [15]) as the classification model [2]. The SGD optimizer whose learning rate is 2.0 is used to train all models.…”

Section: Implementation Detailsmentioning

confidence: 99%

Federated Class-Incremental Learning

Dong¹,

Wang²,

Fang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Federated learning (FL) has attracted growing attentions via data-private collaborative training on decentralized clients. However, most existing methods unrealistically assume object classes of the overall framework are fixed over time. It makes the global model suffer from significant catastrophic forgetting on old classes in real-world scenarios, where local clients often collect new classes continuously and have very limited storage memory to store old classes. Moreover, new clients with unseen new classes may participate in the FL training, further aggravating the catastrophic forgetting of global model. To address these challenges, we develop a novel Global-Local Forgetting Compensation (GLFC) model, to learn a global classincremental model for alleviating the catastrophic forgetting from both local and global perspectives. Specifically, to address local forgetting caused by class imbalance at the local clients, we design a class-aware gradient compensation loss and a class-semantic relation distillation loss to balance the forgetting of old classes and distill consistent inter-class relations across tasks. To tackle the global forgetting brought by the non-i.i.d class imbalance across clients, we propose a proxy server that selects the best old global model to assist the local relation distillation. Moreover, a prototype gradient-based communication mechanism is developed to protect the privacy. Our model outperforms state-of-the-art methods by 4.4%∼15.1% in terms of average accuracy on representative benchmark datasets. The code is available at https://github. com/conditionWang/FCIL.

show abstract

Section: Implementation Detailsmentioning

confidence: 99%

Federated Class-Incremental Learning

Dong¹,

Wang²,

Fang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In latent embedding learning methods the inputs and outputs are projected into a shared latent space [13,14,15]. An effective recent method is the Multivariate Probit Variational AutoEncoder (MPVAE) [6].…”

Section: Related Workmentioning

confidence: 99%

Multi-relation Message Passing for Multi-label Text Classification

Ozmen¹,

Zhang²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

A well-known challenge associated with the multi-label classification problem is modelling dependencies between labels. Most attempts at modelling label dependencies focus on co-occurrences, ignoring the valuable information that can be extracted by detecting label subsets that rarely occur together. For example, consider customer product reviews; a product probably would not simultaneously be tagged by both "recommended" (i.e., reviewer is happy and recommends the product) and "urgent" (i.e., the review suggests immediate action to remedy an unsatisfactory experience). Aside from the consideration of positive and negative dependencies, the direction of a relationship should also be considered. For a multi-label image classification problem, the "ship" and "sea" labels have an obvious dependency, but the presence of the former implies the latter much more strongly than the other way around. These examples motivate the modelling of multiple types of bi-directional relationships between labels. In this paper, we propose a novel method, entitled Multi-relation Message Passing (MrMP), for the multi-label classification problem. Experiments on benchmark multi-label text classification datasets show that the MrMP module yields similar or superior performance compared to state-of-the-art methods. The approach imposes only minor additional computational and memory overheads.

show abstract

“…The second group deals with latent embedding, in which they learn one shared latent space representing both input features and output labels (Bhatia et al 2015a;Yeh et al 2017;Tang et al 2018;Chen et al 2019a). Most recently, Bai, Kong, and Gomes (2020) propose MPVAE: it learns VAE-based probabilistic latent spaces for both labels and features and aligns the latent representations using the Kullback-Leibler divergence.…”

Section: Related Workmentioning

confidence: 99%

HOT-VAE: Learning High-Order Label Correlation for Multi-Label Classification via Attention-Based Variational Autoencoders

Zhao¹,

Kong²,

Bai³

et al. 2021

Preprint

View full text Add to dashboard Cite

Understanding how environmental characteristics affect biodiversity patterns, from individual species to communities of species, is critical for mitigating effects of global change. A central goal for conservation planning and monitoring is the ability to accurately predict the occurrence of species communities and how these communities change over space and time. This in turn leads to a challenging and long-standing problem in the field of computer science -how to perform accurate multi-label classification with hundreds of labels? The key challenge of this problem is its exponential-sized output space with regards to the number of labels to be predicted. Therefore, it is essential to facilitate the learning process by exploiting correlations (or dependency) among labels. Previous methods mostly focus on modelling the correlation on label pairs; however, complex relations between real-world objects often go beyond second order. In this paper, we propose a novel framework for multi-label classification, Highorder Tie-in Variational Autoencoder (HOT-VAE), which performs adaptive high-order label correlation learning. We experimentally verify that our model outperforms the existing state-of-the-art approaches on a bird distribution dataset on both conventional F1 scores and a variety of ecological metrics. To show our method is general, we also perform empirical analysis on seven other public real-world datasets in several application domains, and Hot-VAE exhibits superior performance to previous methods.

show abstract

Two-Stage Label Embedding via Neural Factorization Machine for Multi-Label Classification

Cited by 29 publications

References 14 publications

Federated Class-Incremental Learning

Federated Class-Incremental Learning

Multi-relation Message Passing for Multi-label Text Classification

HOT-VAE: Learning High-Order Label Correlation for Multi-Label Classification via Attention-Based Variational Autoencoders

Contact Info

Product

Resources

About