Information and Entropy in Cluster Analysis

Convexity-based clustering criteria: theory, algorithms, and applications in statistics

“…Convexity-based clustering criteria have first been introduced and investigated by Bock (1983Bock ( , 1992Bock ( , 1994 in the context of the well-known χ 2 goodness-of-fit test for testing the hypothesis H 0 : Q = P 0 that the distribution Q of a I R p -valued random vector X (with a corresponding sample x 1 , ..., x n ∈ I R p ) is given by a fully specified distribution P 0 (e.g., a p-dimensional standard normal distribution N q (0, I p )). The χ 2 test proceeds by first subdividing the domain I R p of X into a given (suitable) number m of 'classes' B 1 , ..., B m ⊂ I R p (discretization, without using the data), then counting the number N i of samples in each class B i and finally rejecting the…”

“…Numerical results have been obtained by Bock (1992Bock ( , 1994 for the case of normal distributions P 0= N p (0, I p ) and P 1= N p (µ, βI p ) (with µ = 0 and β > 1). Here the optimum classes will be parallel linear layers or concentric ellipsoids in I R p .…”

“…Obviously, the 'discrete' maximization problem (2) is returned by (3) if we use P = P n , the empirical distribution P n of the n data points x 1 , ..., x n , define the classes of objects by C i := {k ∈ O | x k ∈ B i } for i = 1, ..., m and recall that in this setting we have E[X|X ∈ B i ] = x Ci . Criteria of this type have been considered by Bock (1983Bock ( , 1992Bock ( , 1994 and Pötzelberger and Strasser (2001), typical examples are provided by φ(x) = ||x||, = ||x|| q q (with the Minkowski norm for some q ≥ 1), = x · log x or = − log x for p = 1, etc. (other cases are listed in section 4.1).…”

“…Whereas we will concentrate here on the basic ideas and some applications, a thorough mathematical analysis of the maximization problem (3) is given by Pötzelberger and Strasser (2001). They provide a p-variate generalization of the seminal paper by Bock (1983) where the MSP algorithm has been proposed in the one-dimensional case (see section 5.1 and also Bock 1992Bock , 1994. A general theory of quantization problems is provided, e.g., in the monograph by Graf and Luschgy (2000).…”

Data Compression and Statistical Inference

“…The principle is of a decision theoretic nature and has already been considered in the context of classification by Bock, [5] and [6].…”