Multi-Branch Tensor Network Structure for Tensor-Train Discriminant Analysis

Abstract

Higher-order data with high dimensionality arise in a diverse set of application areas such as computer vision, video analytics and medical imaging. Tensors provide a natural tool for representing these types of data. Although there has been a lot of work in the area of tensor decomposition and lowrank tensor approximation, extensions to supervised learning, feature extraction and classification are still limited. Moreover, most of the existing supervised tensor learning approaches are based on the orthogonal Tucker model. However, this model has some limitations for large tensors including high memory and computational costs. In this paper, we introduce a supervised learning approach for tensor classification based on the tensortrain model. In particular, we introduce a multi-branch tensor network structure for efficient implementation of tensor-train discriminant analysis (TTDA). The proposed approach takes advantage of the flexibility of the tensor train structure to implement various computationally efficient versions of TTDA. This approach is then evaluated on image and video classification tasks with respect to computation time, storage cost and classification accuracy and is compared to both vector and tensor based discriminant analysis methods.

Category Contrastive Distillation with Self-Supervised Classification

“…constraints. Model compression is an effective method to optimize models, mainly divided into knowledge distillation ( [1], [2], [3], [4], [5]), pruning ( [6]) and quantization ( [7], [8]), etc. Through model compression, large models are effectively transformed into lightweight counterparts, facilitating their migration to mobile devices.…”