Representation Learning With Dual Autoencoder for Multi-Label Classification

Abstract

Multi-label classification aims to deal with the problem that an object may be associated with one or more labels, which is a more difficult task due to the complex nature of multi-label data. The crucial problem of multi-label classification is the more robust and higher-level feature representation learning, which can reduce non-helpful feature attributes from the input space prior to training. In recent years, deep learning methods based on autoencoders have achieved excellent performance in multi-label classification for the advantages of powerful representations learning ability and fast convergence speed. However, most existing autoencoder-based methods only rely on the single autoencoder model, which pose challenges for multi-label feature representations learning and fail to measure similarities between data spaces. To address this problem, in this paper, we propose a novel representation learning method with dual autoencoder for multi-label classification. Compared to the existing autoencoder-based methods, our proposed method can capture different characteristics and more abstract features from data by the serially connection of two different types of autoencoders. More specifically, firstly, the algorithm of Reconstruction Independent Component Analysis (RICA) in sparse autoencoder is trained on patches on all training and test dataset for robust global feature representations learning. Secondly, with the output of RICA, stacked autoencoder with manifold regularization (SAMR) is introduced to ameliorate the quality of multi-label features learning. Comprehensive experiments on several real-world data sets demonstrate the effectiveness of our proposed approach compared with several competing state-of-the-art methods.

Dual Auto-Encoder Based Rating Prediction Recommendation Algorithm

“…Therefore, we apply deep learning to recommendation algorithm to improve the accuracy of rating prediction of recommendation algorithm. Sedhain et al [24] combined auto-encoders with collaborative filtering recommendation and proposed the AutoRec model, which utilizes a single-layer hidden auto-encoder to encode and decode the rating matrix and reconstruct the user's predicted ratings of items, which is the first combination of auto-encoder and recommendation system; Wang et al [25] proposed CDL to extract potential features of items from text information using stacked noise reduction auto-encoder and combined with PMF for recommendation, which solved the problem of low recommendation accuracy of rating prediction algorithm when data is sparse; Subsequently, Wu et al [26] proposed to recommend items to users using RNN and applied it to NetEase with good results; subsequently, Zhang et al [27] proposed an semi auto-encoder based HRSA model that introduces exploit auxiliary information for rating prediction and Top-k recommendation; Zhu et al [28] proposed exploit dual auto-encoders in recommendation algorithms, employing dual auto-encoders for multi-label feature learning, which achieves robust global feature learning by concatenating two different types of auto-encoders to obtain different features from the data; Wu et al [29] proposed the CDAE algorithm to solve the Top-k recommendation problem, which is similar to the model structure of AutoRec, but the key difference is that CDAE introduces a user feature for each user to improve the recommendation effect. Zhuang et al [30] proposed to use dual auto-encoders to learn the potential feature information of users and items separately, and then obtain the prediction values by the inner product of the potential feature vectors of users and items to improve the recommendation effect.…”