Generalized Loss-Sensitive Adversarial Learning with Manifold Margins

Edraki, Marzieh; Qi, Guo-Jun

doi:10.1007/978-3-030-01228-1_6

Cited by 25 publications

(11 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For this purpose, the idea of adversarially training a generator and its corresponding encoder has been independently developed in Bidirectional Generative Adversarial Networks (BiGAN) [15] and Adversarially Learned Inference (ALI) [16], respectively. The idea is later integrated into a regularized loss-sensitive GAN model with proved distributional consistency and generalizability to generate real data [70].…”

Section: Gan-based Representationsmentioning

confidence: 99%

Small Data Challenges in Big Data Era: A Survey of Recent Progress on Unsupervised and Semi-Supervised Methods

Qi¹,

Luo

2022

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

182

View full text Add to dashboard Cite

Small data challenges have emerged in many learning problems, since the success of deep neural networks often relies on the availability of a huge amount of labeled data that is expensive to collect. To address it, many efforts have been made on training complex models with small data in an unsupervised and semi-supervised fashion. In this paper, we will review the recent progresses on these two major categories of methods. A wide spectrum of small data models will be categorized in a big picture, where we will show how they interplay with each other to motivate explorations of new ideas. We will review the criteria of learning the transformation equivariant, disentangled, self-supervised and semi-supervised representations, which underpin the foundations of recent developments. Many instantiations of unsupervised and semi-supervised generative models have been developed on the basis of these criteria, greatly expanding the territory of existing autoencoders, generative adversarial nets (GANs) and other deep networks by exploring the distribution of unlabeled data for more powerful representations. While we focus on the unsupervised and semi-supervised methods, we will also provide a broader review of other emerging topics, from unsupervised and semi-supervised domain adaptation to the fundamental roles of transformation equivariance and invariance in training a wide spectrum of deep networks. It is impossible for us to write an exclusive encyclopedia to include all related works. Instead, we aim at exploring the main ideas, principles and methods in this area to reveal where we are heading on the journey towards addressing the small data challenges in this big data era.

show abstract

Section: Gan-based Representationsmentioning

confidence: 99%

Small Data Challenges in Big Data Era: A Survey of Recent Progress on Unsupervised and Semi-Supervised Methods

Qi¹,

Luo

2022

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

182

View full text Add to dashboard Cite

show abstract

“…III. THE PROPOSED METHOD As we all know, the original data contains much useful information, such as the given label information and the underlying geometric information residing in the data [35][36][37][38]. If we can appropriately utilize these information to guide the projection learning, then a better classification performance will be obtained.…”

Section: Discriminatively Regularized Least-squaresmentioning

confidence: 99%

Adaptive Locality Preserving Regression

Wen

Zhong

Zhang

et al. 2020

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

This paper proposes a novel discriminative regression method, called adaptive locality preserving regression (ALPR) for classification. In particular, ALPR aims to learn a more flexible and discriminative projection that not only preserves the intrinsic structure of data, but also possesses the properties of feature selection and interpretability. To this end, we introduce a target learning technique to adaptively learn a more discriminative and flexible target matrix rather than the pre-defined strict zero-one label matrix for regression. Then a locality preserving constraint regularized by the adaptive learned weights is further introduced to guide the projection learning, which is beneficial to learn a more discriminative projection and avoid overfitting. Moreover, we replace the conventional 'Frobenius norm' with the special l2,1 norm to constrain the projection, which enables the method to adaptively select the most important features from the original high-dimensional data for feature extraction. In this way, the negative influence of the redundant features and noises residing in the original data can be greatly eliminated. Besides, the proposed method has good interpretability for features owning to the row-sparsity property of the l2,1 norm. Extensive experiments conducted on the synthetic database with manifold structure and many realworld databases prove the effectiveness of the proposed method.

show abstract

“…There have been many efforts [38,39] to employ manifold properties to stabilize GAN training process and improve the quality of generated samples but none of them benefit from smart samples selection to expedite the training as suggested in the first column of Fig. 1.…”

Section: Training Ganmentioning

confidence: 99%

Select to better learn: Fast and accurate deep learning using data selection from nonlinear manifolds

Joneidi¹,

Vahidian²,

Esmaeili³

et al. 2020

Preprint

View full text Add to dashboard Cite

Finding a small subset of data whose linear combination spans other data points, also called column subset selection problem (CSSP), is an important open problem in computer science with many applications in computer vision and deep learning. There are some studies that solve CSSP in a polynomial time complexity w.r.t. the size of the original dataset. A simple and efficient selection algorithm with a linear complexity order, referred to as spectrum pursuit (SP), is proposed that pursuits spectral components of the dataset using available sample points. The proposed non-greedy algorithm aims to iteratively find K data samples whose span is close to that of the first K spectral components of entire data. SP has no parameter to be fine tuned and this desirable property makes it problem-independent. The simplicity of SP enables us to extend the underlying linear model to more complex models such as nonlinear manifolds and graph-based models. The nonlinear extension of SP is introduced as kernel-SP (KSP). The superiority of the proposed algorithms is demonstrated in a wide range of applications.

show abstract

Generalized Loss-Sensitive Adversarial Learning with Manifold Margins

Cited by 25 publications

References 7 publications

Small Data Challenges in Big Data Era: A Survey of Recent Progress on Unsupervised and Semi-Supervised Methods

Small Data Challenges in Big Data Era: A Survey of Recent Progress on Unsupervised and Semi-Supervised Methods

Adaptive Locality Preserving Regression

Select to better learn: Fast and accurate deep learning using data selection from nonlinear manifolds

Contact Info

Product

Resources

About