Variable Screening for Cluster Analysis

Abstract

Inclusion of irrelevant variables in a cluster analysis adversely affects subgroup recovery. This paper examines using moment-based statistics to screen variables; only variables which pass the screening are then used in clustering. Normal mixtures are analytically shown often to possess negative kurtosis. Two related measures, m and coefficient of bimodality b, are also 'examined.A Monte Carlo study compared the screening measures to no selection, De Soete's (1988) ultrametric weights, and Fowlkes, Gnanadesikan, and Kettenring's (1988) forward selection procedure. Screening based on kurtosis degraded recovery and is not recommended.In contrast, screening on m or on b improved recovery over both no selection and forward selection, and screening performed as well as ultrametric weights. Combining screening with ultrametric weights performed extremely well. All methods were found to be somewhat sensitive to other types of error.Screening variables appears a viable alternative to both ultrametric weights and forward selection. The potential advantages and disadvantages of screening are considered.

A Preliminary Study of the Effects of Within‐group Covariance Structure on Recovery in Cluster Analysis

“…Johnson (1967) and Milligan (1979) demonstrated the relationship between the ultrametric inequality and many commonly used hierarchical clustering algorithms, and Milligan and Isaac (1980) give simulation results which provide support for the utility of the conceptualization. Two studies (Milligan, 1989b, Donoghue, 1994a have examined De Soete's algorithm, and found that it greatly improved cluster recovery when the data contained one, two, or three irrelevant dimensions. Further studies to evaluate this method would clearly be useful.…”

Transsexual Groups in Veale et al. (2008) are “Autogynephilic” and “Even More Autogynephilic”

“…Experts advise that, in performing cluster analysis, ''variables should only be included if there is good reason to think that they will define the clusters'' (Everitt, Landau, & Leese, 2001, p. 179). Variables that are not theoretically relevant should be omitted, because ''irrelevant variables are likely to 'swamp' the genuine differences of interest'' (Gordon, 1981, p. 30; see also Donoghue, 1994). If Veale et al sought to partition their participants into autogynephilic and nonautogynephilic clusters, they should have conducted an analysis using only Blanchard's Core Autogynephilia and Autogynephilic Interpersonal Fantasy scales, and omitted the irrelevant variables Attraction to Feminine Males and Attraction to Transgender Fiction.…”

A Preliminary Comparison of the Effectiveness of Cluster Analysis Weighting Procedures for Within‐group Covariance Structure

“…The relationship of the weighting methods identified here to the closely allied problem of variable selection (Milligan, 1989b;Donoghue, 1994a) remains to be addressed. Both of these papers found that the ultrametric weights were useful in ameliorating the degradation of cluster recovery caused by including irrelevant variables in the cluster analysis.…”

“…Johnson (1967) and Milligan (1979) demonstrated the relationship between the ultrametric inequality and many commonly used hierarchical clustering algorithms, and Milligan and Isaac (1980) give simulation results which provide support for the utility of the conceptualization. Two studies (Milligan, 1989b, Donoghue, 1994a have examined De Soete's algorithm, and found that it greatly improved cluster recovery when the data contained "error" dimensions, i.e, dimensions which contained no information about subgroup membership. However, it is not known whether this method helps with the difficulties of within-group covariance structure.…”

“…To guide these choices, numerous studies have examined various aspects of the clustering process, including comparison of clustering algorithms (e.g., Belbin, Faith, & Milligan, 1992;Blashfield, 1976;Donoghue, 1994bDonoghue, , 1995Milligan, 1979Milligan, , 1989aScheibler & Schneider, 1985), the effect of various types of "error" in the data to be clustered (Milligan, 1980), variable standardization (Milligan & Cooper, 1988;Barton, 1993), selection/weighting of irrelevant variables (De Soete, 1986, 1988Milligan, 1989b;Donoghue, 1994a), procedures to determine the number of clusters (e.g., Milligan & Cooper, 1985), and procedures to compare clustering solutions (Milligan & Cooper, 1986).…”