Concerning the differentiability of the energy function in vector quantization algorithms

Abstract

The adaptation rule for Vector Quantization algorithms, and consequently the convergence of the generated sequence, depends on the existence and properties of a function called the energy function, defined on a topological manifold. Our aim is to investigate the conditions of existence of such a function for a class of algorithms examplified by the initial "K-means" (Mac-Queen, 1967) and Kohonen algorithms (Kohonen, 1982;Kohonen, 1988). The results presented here supplement previous studies, including (Tolat, 1990), (Erwin et al., 1992), (Cottrell et al., 1994), (Pagès, 1993) and (Cottrell et al., 1998). Our work shows that the energy function is not always a potential but at least the uniform limit of a series of potential functions which we call a pseudo-potential. It also shows that a large number of existing vector quantization algorithms developed by the Artificial Neural Networks community fall into this category. The framework we define opens the way to study the convergence of all the corresponding adaptation rules at once, and a theorem gives promising insights in that direction. We also demonstrate that the "K-means" energy function is a pseudo-potential but not a potential in general. Consequently, the energy function associated to the "Neural-Gas" is not a potential in general.

A Sequential Algorithm for Training the SOM Prototypes Based on Higher-Order Recursive Equations

“…The minimization of QE is the goal of vector quantization, while the SOM can be regarded as a computational intelligence algorithm that aims to minimize the distortion measure DM(Δ, σ) for some σ > 0. It is known that the SOM only minimizes DM(Δ, σ) approximately, whereas the effective minimization of DM(Δ, σ) is extremely heavy numerically [3,20,21].…”