Reduction of Dimension and Size of Data Set by Parallel Fast Simulated Annealing

Abstract

Abstract.A universal method of dimension and sample size reduction, designed for exploratory data analysis procedures, constitutes the subject of this paper. The dimension is reduced by applying linear transformation, with the requirement that it has the least possible influence on the respective locations of sample elements. For this purpose an original version of the heuristic Parallel Fast Simulated Annealing method was used. In addition, those elements which change the location significantly as a result of the transformation, may be eliminated or assigned smaller weights for further analysis. As well as reducing the sample size, this also improves the quality of the applied methodology of knowledge extraction. Experimental research confirmed the usefulness of the procedure worked out in a broad range of problems of exploratory data analysis such as clustering, classification, identification of outliers and others.

An algorithm for reducing the dimension and size of a sample for data exploration procedures

“…The effectiveness of the presented method will be verified for fundamental procedures in exploratory data analysis: clustering, classification and detection of atypical elements (outliers). Many aspects considered in this paper were initially proposed by Łukasik and Kulczycki (2011) as well as Kulczycki and Łukasik (2014) in their basic form.…”