A novel feature selection approach is presented in this paper. Sammon’s Stress Function transforms the high dimension data to a lower dimension data set. A data set is divided into small partitions. The features are assigned randomly to these partitions. Using GA with Sammon Error as fitness value, a small, desired number of features are selected from every partition. The combination of the reduced subsets of the features from these partitions is again divided into small partitions. After a certain number of iterating the process, a desired small number of features is obtained. For experimental validation, the proposed method has been tested on 11 standard datasets with three classifiers namely, Decision Tree, MLP and KNN. The classification accuracies obtained by the proposed method is highest on most of the considered datasets against the results reported in literature. Moreover, the proposed method selects comparatively less number of features in comparison to considered methods. The optimistic results obtained from the proposed method justify its strength.