Principled Hybrids of Generative and Discriminative Models

Lasserre, Julia; Bishop, Christopher M.; Minka, Tom

doi:10.1109/cvpr.2006.227

Cited by 232 publications

(191 citation statements)

References 5 publications

Supporting

Mentioning

184

Contrasting

Unclassified

Order By: Relevance

“…Some rights reserved. studies performed poorly, and this empirical evidence has been bolstered by theoretical arguments (Lasserre et al, 2006).…”

Section: Introductionmentioning

confidence: 99%

An improved hierarchical Bayesian model of language for document classification

Allison

2008

Proceedings of the 22nd International Conference on Computational Linguistics - COLING '08

View full text Add to dashboard Cite

This paper addresses the fundamental problem of document classification, and we focus attention on classification problems where the classes are mutually exclusive. In the course of the paper we advocate an approximate sampling distribution for word counts in documents, and demonstrate the model's capacity to outperform both the simple multinomial and more recently proposed extensions on the classification task. We also compare the classifiers to a linear SVM, and show that provided certain conditions are met, the new model allows performance which exceeds that of the SVM and attains amongst the very best published results on the Newsgroups classification task.

show abstract

“…Some rights reserved. studies performed poorly, and this empirical evidence has been bolstered by theoretical arguments (Lasserre et al, 2006).…”

Section: Introductionmentioning

confidence: 99%

An improved hierarchical Bayesian model of language for document classification

Allison

2008

Proceedings of the 22nd International Conference on Computational Linguistics - COLING '08

View full text Add to dashboard Cite

show abstract

“…We propose a joint development of both the schemes into a coherent optimization framework. The framework presented in this paper extends the idea of joint learning of generative/discriminative models [1,12,13,25].…”

Section: Introductionmentioning

confidence: 99%

Non-negative matrix factorization as a feature selection tool for maximum margin classifiers

Gupta

Xiao

2011

CVPR 2011

View full text Add to dashboard Cite

Non-negative matrix factorization (NMF) has previously been shown to be a useful decomposition tool for multivariate data. Non-negative bases allow strictly additive combinations which have been shown to be part-based as well as relatively sparse. We pursue a discriminative decomposition by coupling NMF objective with a maximum margin classifier, specifically a support vector machine (SVM). Conversely, we propose an NMF based regularizer for SVM. We formulate the joint update equations and propose a new method which identifies the decomposition as well as the classification parameters. We present classification results on synthetic as well as real datasets.

show abstract

“…Hegerath et al [13] present a Gaussian mixture density classifier for patch-based object recognition which, in principle, is a generative model but which is refined by discriminatively changing the cluster-weights. The discriminative refinement of a generative model can in some cases be shown to be identical to directly training a discriminative model [17,24] if done properly. The model presented in [17] which also resembles a mixture, thus is a much cleaner way to achieve a similar goal.…”

Section: Introductionmentioning

confidence: 99%

“…A direct approach to joining the two principles is proposed by Minka [24] and used in an object recognition framework by Lasserre et al [17] which allows to seamlessly blend from a fully discriminative model to a fully generative model. Grabner et al [12] modify a discriminative, boosted model to account for reconstruction in addition to the discriminatory performance and a clear performance boost for noisy data was observed.…”

Section: Introductionmentioning

confidence: 99%

Object classification by fusing SVMs and Gaussian mixtures

Deselaers

Heigold

Ney

2010

Pattern Recognition

View full text Add to dashboard Cite

We present a new technique that employs support vector machines (SVMs) and Gaussian mixture densities (GMDs) to create a generative/discriminative object classification technique using local image features. In the past, several approaches to fuse the advantages of generative and discriminative approaches were presented, often leading to improved robustness and recognition accuracy. Support vector machines are a well known discriminative classification framework but, similar to other discriminative approaches, suffer from a lack of robustness with respect to noise and overfitting. Gaussian mixtures, on the contrary, are a widely used generative technique. We present a method to directly fuse both approaches, effectively allowing to fully exploit the advantages of both. The fusion of SVMs and GMDs is done by representing SVMs in the framework of GMDs without changing the training and without changing the decision boundary. The new classifier is evaluated on the PASCAL VOC 2006 data. Additionally, we perform experiments on the USPS dataset and on four tasks from the UCI machine learning repository to obtain additional insights into the properties of the proposed approach. It is shown that for the relatively rare cases where SVMs have problems, the combined method outperforms both individual ones.

show abstract

Principled Hybrids of Generative and Discriminative Models

Cited by 232 publications

References 5 publications

An improved hierarchical Bayesian model of language for document classification

An improved hierarchical Bayesian model of language for document classification

Non-negative matrix factorization as a feature selection tool for maximum margin classifiers

Object classification by fusing SVMs and Gaussian mixtures

Contact Info

Product

Resources

About