Document Clustering Evaluation: Divergence from a Random Baseline

Abstract

Divergence from a random baseline is a technique for the evaluation of document clustering. It ensures cluster quality measures are performing work that prevents ineffective clusterings from giving high scores to clusterings that provide no useful result. These concepts are defined and analysed using intrinsic and extrinsic approaches to the evaluation of document cluster quality. This includes the classical clusters to categories approach and a novel approach that uses ad hoc information retrieval. The divergence from a random baseline approach is able to differentiate ineffective clusterings encountered in the INEX XML Mining track. It also appears to perform a normalisation similar to the Normalised Mutual Information (NMI) measure but it can be applied to any measure of cluster quality. When it is applied to the intrinsic measure of distortion as measured by RMSE, subtraction from a random baseline provides a clear optimum that is not apparent otherwise. This approach can be applied to any clustering evaluation. This paper describes its use in the context of document clustering evaluation.

Automatic Classification of Functional Gait Disorders

“…in case of PCA-based representations). We computed the accuracy of LDA and reported the divergence from a random baseline [30] to quantify to which degree an input parameter or input representation is able to separate the underlying classes. The random baseline was estimated by the zero rule (always choosing the most frequent class in the dataset).…”

“…Since in different experiments the random baseline varies, the absolute values of accuracy are of limited expressiveness. To enable a fair comparison, we employ the divergence from a random baseline approach [30] and thus provide for each experiment the difference between the random baseline and the absolute classification accuracy.…”