Learning a Discriminative Dictionary with CNN for Image Classification

Yu, Shuai; Zhang, Tao; Ma, Chao; Zhou, Lei; Yang, Jie; He, Xiangjian

doi:10.1007/978-3-319-46672-9_22

Cited by 5 publications

(12 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This would make all subsequent tasks (generative or discriminative) much easier. In this work, we will show how to achieve this with a different fundamental metric, known as the rate reduction, introduced by [13].…”

Section: Learning Generative Models Via Auto-encoding or Ganmentioning

confidence: 99%

“…In this work, we will introduce a more refined 2k-class measure for the k real and k generated classes. In addition, to avoid features for each class collapsing to a singleton [30], instead of cross entropy, we will use the so-called rate-reduction measure that promotes multi-mode and multi-dimension in the learned features [13]. One may view the rate reduction as a metric distance that has closed-form formulae for a mixture of (subspace-like) Gaussians, whereas neither JS-divergence nor W-distance can be computed in closed form (even between two Gaussians).…”

Section: Learning Generative Models Via Auto-encoding or Ganmentioning

confidence: 99%

“…As has been shown by [13], maximizing the difference between the two terms will expand the whole feature set while compressing and linearizing features of each of the k classes. If the mapping f maximizes the rate reduction, it maps the features of different classes into independent (orthogonal) subspaces in R d .…”

Section: Learning Linear Discriminative Representation Via Rate Reduc...mentioning

confidence: 99%

“…One issue with this one-sided LDR learning (4) is that maximizing the above objective (5) tends to expand the dimension of the learned subspace for features in each class (if the dimension of the feature space d is too high, maximizing the rate reduction may overestimate the dimension of each class. Hence, to learn a good representation, one needs to pre-select a proper dimension for the feature space, as achieved in the experiments in [13].…”

Section: Closed-loop Transcription To An Ldr (Ctrl)mentioning

confidence: 99%

“…According to the interpretation of the rate reduction given in [13], the above quantity precisely measures the volume of the space between Z j and Ẑj , illustrated as a pair of black and blue lines in Figure 1. Then, for the "distance" of all, say k, classes, we simply sum the rate reduction for all pairs:…”

Section: Measuring Distances In the Feature Space And Data Spacementioning

confidence: 99%

See 4 more Smart Citations

CTRL: Closed-Loop Transcription to an LDR via Minimaxing Rate Reduction

Dai

Tong

et al. 2022

Entropy

View full text Add to dashboard Cite

This work proposes a new computational framework for learning a structured generative model for real-world datasets. In particular, we propose to learn a Closed-loop Transcriptionbetween a multi-class, multi-dimensional data distribution and a Linear discriminative representation (CTRL) in the feature space that consists of multiple independent multi-dimensional linear subspaces. In particular, we argue that the optimal encoding and decoding mappings sought can be formulated as a two-player minimax game between the encoder and decoderfor the learned representation. A natural utility function for this game is the so-called rate reduction, a simple information-theoretic measure for distances between mixtures of subspace-like Gaussians in the feature space. Our formulation draws inspiration from closed-loop error feedback from control systems and avoids expensive evaluating and minimizing of approximated distances between arbitrary distributions in either the data space or the feature space. To a large extent, this new formulation unifies the concepts and benefits of Auto-Encoding and GAN and naturally extends them to the settings of learning a both discriminative and generative representation for multi-class and multi-dimensional real-world data. Our extensive experiments on many benchmark imagery datasets demonstrate tremendous potential of this new closed-loop formulation: under fair comparison, visual quality of the learned decoder and classification performance of the encoder is competitive and arguably better than existing methods based on GAN, VAE, or a combination of both. Unlike existing generative models, the so-learned features of the multiple classes are structured instead of hidden: different classes are explicitly mapped onto corresponding independent principal subspaces in the feature space, and diverse visual attributes within each class are modeled by the independent principal components within each subspace.

show abstract

Section: Learning Generative Models Via Auto-encoding or Ganmentioning

confidence: 99%

Section: Learning Generative Models Via Auto-encoding or Ganmentioning

confidence: 99%

Section: Learning Linear Discriminative Representation Via Rate Reduc...mentioning

confidence: 99%

Section: Closed-loop Transcription To An Ldr (Ctrl)mentioning

confidence: 99%

Section: Measuring Distances In the Feature Space And Data Spacementioning

confidence: 99%

See 3 more Smart Citations