The process which was used for grouping the similar elements or occurring closely is called cluster. Nowadays cluster analysis is one of the major data analysis techniques. On the other hand many important problems involve clustering for large datasets. KSOM and k-means is one of the most popular partitioning clustering algorithms that are widely used. The original k-means algorithm is computationally expensive and the number of clusters K, to be specified before the algorithm is applied. The other thing is, it is quite sensitive to initial centroids. When more number of dimensions is added then K-Means fails to give optimum result. For this "Curse of High Dimensionality" problem is occurred. Here we propose that Kohonen Self Organizing Map (KSOM) is used to define number of clusters and then load based initial centroid K-Means algorithm (KSOMKM) is used to find out the more accurate number of cluster for High Dimensional Dataset. Finally the Kohonen Self Organizing Map (KSOM) with Load based K-Means algorithm (KSOMKM) is tested on different datasets. There are an IRIS data set, Diabetes dataset, Thyroid, Blood pressure dataset. Its performance is compared with other clustering algorithm for number of iteration, quantization errors and topographic errors.  Index Terms-curse of dimensionality, data mining, high-dimensional datasets and Kohonen Self Organizing Map (KSOM).