Topic Graph Based Non-negative Matrix Factorization for Transfer Learning

Abstract

Abstract. We propose a method called Topic Graph based NMF for Transfer Learning (TNT) based on Non-negative Matrix Factorization (NMF). Since NMF learns feature vectors to approximate the given data, the proposed approach tries to preserve the feature space which is spanned by the feature vectors to realize transfer learning. Based on the learned feature vectors in the source domain, a graph structure called topic graph is constructed, and the graph is utilized as a regularization term in the framework of NMF. We show that the proposed regularization term corresponds to maximizing the similarity between topic graphs in both domains, and that the term corresponds to the graph Laplacian of the topic graph. Furthermore, we propose a learning algorithm with multiplicative update rules and prove its convergence. The proposed method is evaluated over document clustering problem, and the results indicate that the proposed method improves performance via transfer learning.

Non-negative Matrix Factorization with sparse features

“…Since we focus on the preservation of pairwise similarities with respect to the given data space, instead of the pairwise relation among instances [1] or among constructed features [9], the pairwise relation among the attributes is considered in our current approach.…”

“…In order to preserve the pairwise similarities of attributes, we consider a graph structure and employ the graph regularization in NMF [1], [9]. Since we focus on the preservation of pairwise similarities with respect to the given data space, instead of the pairwise relation among instances [1] or among constructed features [9], the pairwise relation among the attributes is considered in our current approach.…”

“…(11) is nonincreasing under the above update rules, and since it is nonnegative, the algorithm converges. The proof based on the auxiliary function [1], [9] is omitted due to space limitation.…”

Topic graph based transfer learning via generalized KL divergence based NMF

“…Based on Non-negative Matrix Factorization (NMF) [8], [12], [5], we proposed a transfer learning method called TNT (Topic graph based NMF for Transfer learning) [10]. Since NMF learns feature vectors to approximate the given data instances, TNT tries to preserve the feature space which is spanned by the vectors in transfer learning.…”

“…The domain in which the knowledge is learned is called source domain, and the other domain to which the knowledge is transfered is called target domain in this paper. Various methods have been proposed to realize transfer learning [9], [13], [11], [10].…”