Clustering is the process of grouping similar data into a set of clusters. Cluster analysis is one of the major data analysis techniques and k-means one of the most popular partitioning clustering algorithm that is widely used. But the original k-means algorithm is computationally expensive and the resulting set of clusters strongly depends on the selection of initial centroids. Several methods have been proposed to improve the performance of k-means clustering algorithm. In this paper we propose a heuristic method to find better initial centroids as well as more accurate clusters with less computational time. Experimental results show that the proposed algorithm generates clusters with better accuracy thus improve the performance of k-means clustering algorithm.