Some Notes on Alternating Optimization

Bezdek, James C.; Hathaway, Richard J.

doi:10.1007/3-540-45631-7_39

Cited by 340 publications

(308 citation statements)

References 12 publications

Supporting

Mentioning

308

Contrasting

Order By: Relevance

“…This allows one to use well-defined functions of measures for a specific problem in order to improve performance. Additionally, the techniques of variable splitting [Boyd et al 2011] and alternating minimization procedure [Bezdek and Hathaway 2002] are invoked to provide a more scalable solution.…”

Section: Related Workmentioning

confidence: 99%

“…Even if it converges, it might not converge to the locally optimal solution. Some authors [Cheney and Goldstein 1959;Zangwill 1969;Wu 1982;Bezdek and Hathaway 2003] have shown that the convergence guarantee of alternating optimization can be analyzed using the topological properties of the objective and the space over which it is optimized. Others have used information geometry [Csiszár and Tusnády 1984;Wang and Schuurmans 2003b;Subramanya and Bilmes 2011] to analyze the convergence, as well as a combination of both information geometry and topological properties of the objective [Gunawardana and Byrne 2005].…”

Section: Convergence Analysis Of Oacmentioning

confidence: 99%

“…To analyze the same, we use some formulations that were derived in Bezdek and Hathaway [2003] to characterize the local convergence rate of alternating minimization type of algorithms in general. In this section, we first explain the tools and then show that the analysis applies to the objective function J seamlessly.…”

Section: Analysis Of Rate Of Convergence For Oacmentioning

confidence: 99%

“…Before presenting the main theorem from Bezdek and Hathaway [2003], the formal definition of q-linear rate of convergence is provided here. The "q" in this definition stands for quotient.…”

Section: Tools For Analyzing Local Rate Of Convergencementioning

confidence: 99%

See 3 more Smart Citations

An Optimization Framework for Combining Ensembles of Classifiers and Clusterers with Applications to Nontransductive Semisupervised Learning and Transfer Learning

Acharya

Hruschka

Ghosh

et al. 2014

ACM Trans. Knowl. Discov. Data

View full text Add to dashboard Cite

An optimization framework for combining ensembles of classifiers and clusterers with applications to nontransductive semisupervised learning and transfer learning. Data, New York, v.9, n.1, p Unsupervised models can provide supplementary soft constraints to help classify new "target" data because similar instances in the target set are more likely to share the same class label. Such models can also help detect possible differences between training and target distributions, which is useful in applications where concept drift may take place, as in transfer learning settings. This article describes a general optimization framework that takes as input class membership estimates from existing classifiers learned on previously encountered "source" (or training) data, as well as a similarity matrix from a cluster ensemble operating solely on the target (or test) data to be classified, and yields a consensus labeling of the target data. More precisely, the application settings considered are nontransductive semisupervised and transfer learning scenarios where the training data are used only to build an ensemble of classifiers and are subsequently discarded before classifying the target data. The framework admits a wide range of loss functions and classification/clustering methods. It exploits properties of Bregman divergences in conjunction with Legendre duality to yield a principled and scalable approach. A variety of experiments show that the proposed framework can yield results substantially superior to those provided by naïvely applying classifiers learned on the original task to the target data. In addition, we show that the proposed approach, even not being conceptually transductive, can provide better results compared to some popular transductive learning techniques. ACM Transactions on Knowledge Discovery from

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Convergence Analysis Of Oacmentioning

confidence: 99%

Section: Analysis Of Rate Of Convergence For Oacmentioning

confidence: 99%

“…Before presenting the main theorem from Bezdek and Hathaway [2003], the formal definition of q-linear rate of convergence is provided here. The "q" in this definition stands for quotient.…”

Section: Tools For Analyzing Local Rate Of Convergencementioning

confidence: 99%

See 2 more Smart Citations

An Optimization Framework for Combining Ensembles of Classifiers and Clusterers with Applications to Nontransductive Semisupervised Learning and Transfer Learning

Acharya

Hruschka

Ghosh

et al. 2014

ACM Trans. Knowl. Discov. Data

View full text Add to dashboard Cite

show abstract

“…In particular, this strategy is utilized in the variational Bayesian techniques which decouple the difficult problem of describing the posterior probability density of the model parameters into many smaller tractable problems [1,2,3,4,5,6,7,8,9]. Theoretical justification for the method as an optimization algorithm and some different applications can be found in [10].…”

Section: Introductionmentioning

confidence: 99%

Speeding up cyclic update schemes by pattern searches

Honkela¹

Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02.

View full text Add to dashboard Cite

A popular strategy for dealing with large parameter estimation problems is to split the problem into manageable subproblems and solve them cyclically one by one until convergence. We address a well-known problem with this strategy, namely slow convergence under low noise. We propose using so called pattern searches which consist of a parameter-wise update phase followed by a line search. The search direction of the line search is computed by combining the individual updates of all subproblems. The approach can be used to accelerate learning of several methods proposed in the literature without the need for large algorithmic modifications such as evaluation of global gradients. The proposed modification is shown to reduce the convergence time in a realistic independent component analysis (ICA) problem by more than 85 %.

show abstract

Kernel Fisher's Discriminant with Heterogeneous Kernels

Dundar

Fung²

2009

Kernel Methods for Remote Sensing Data Analysis

View full text Add to dashboard Cite

In hyperspectral data analysis, materials of practical interest, such as agricultural crops, forest plantations, natural vegetation, minerals, and fields of interest in urban areas exist in a variety of states and are usually observed in a number of conditions of illumination. That is, most land-cover types do not have a single spectral response. For example a crop type will show different spectral characteristics at different times of the day and year. Similarly, roof tops are usually made of a variety of different materials including concrete, tile, bricks, glass etc. all of which have different spectral responses. The number of such examples can easily be augmented.One possible way to deal with this problem is to model each class distribution data using Finite Mixture models (McLachlan and Peel (2004)). Finite Mixture Models usually leads to competitive performance when there is enough labeled data to reveal the underlying structure of the class distributions. However, in most real world settings, this may not be the case. The price one must pay for labeled data is usually prohibitively expensive, as acquiring labeled data requires a tedious and time consuming process of human labeling.

show abstract

Some Notes on Alternating Optimization

Cited by 340 publications

References 12 publications

An Optimization Framework for Combining Ensembles of Classifiers and Clusterers with Applications to Nontransductive Semisupervised Learning and Transfer Learning

An Optimization Framework for Combining Ensembles of Classifiers and Clusterers with Applications to Nontransductive Semisupervised Learning and Transfer Learning

Speeding up cyclic update schemes by pattern searches

Kernel Fisher's Discriminant with Heterogeneous Kernels

Contact Info

Product

Resources

About