Supervised tensor learning

Abstract

This paper aims to take general tensors as inputs for supervised learning. A supervised tensor learning (STL) framework is established for convex optimization based learning techniques such as support vector machines (SVM) and minimax probability machines (MPM). Within the STL framework, many conventional learning machines can be generalized to take n th -order tensors as inputs. We also study the applications of tensors to learning machine design and feature extraction by linear discriminant analysis (LDA). Our method for tensor based feature extraction is named the tenor rank-one discriminant analysis (TR1DA). These generalized algorithms have several advantages: 1) reduce the curse of dimension problem in machine learning and data mining; 2) avoid the failure to converge; and 3) achieve better separation between the different categories of samples. As an example, we generalize MPM to its STL version, which is named the tensor MPM (TMPM). TMPM learns a series of tensor projections iteratively. It is then evaluated against the original MPM. Our experiments on a binary classification problem show that TMPM significantly outperforms the original MPM.

Top 10 algorithms in data mining

“…There have been a large number of data mining algorithms rooted in these fields to perform different data analysis tasks. The 10 algorithms identified by the IEEE International Conference on Data Mining (ICDM) and presented in this article are among the most influential algorithms for classification [47,51,77], clustering [11,31,40,[44][45][46], statistical learning [28,76,92], association analysis [2,6,13,50,54,74], and link mining.…”

Local ridge regression for face recognition

“…Many researches have showed that representing the images as tensors of arbitrary order can further improve the performance of algorithms in most cases [18][19][20][21][22][23][24]. Consequently, how to generalize LRR to tensor learning is another interesting topic for future study.…”

Slice Oriented Tensor Decomposition of EEG Data for Feature Extraction in Space, Frequency and Time Domains

“…These methods, such as tensor PCA [8], tensor LDA [9,10], tensor subspace analysis [11][12][13], treat original data as second-or high-order tensors. For supervised feature classification [14], the tensor factorization can lead to structured dimensionality reduction by learning multiple interrelated subspaces. Discriminant analysis using tensor representation [15] can avoid the curse of dimensionality dilemma and overcome the small sample size problem.…”