Supervised classification techniques, broadly depend on the availability of labeled data. However, collecting this labeled data is always a tedious and costly process. To reduce these efforts and improve the performance of classification process, this paper proposes a new framework, which combines a most basic classification technique with the semi-supervised process of clustering. Semi-supervised clustering algorithms, aim to increase the accuracy of clustering process by effectively exploring available supervision from a limited amount of labeled data and help to label the unlabeled data. In our paper, a semi-supervised clustering is integrated with naive bayes classification technique which helps to better train the classifier. To evaluate the performance of the proposed technique, we conduct experiments on several real world benchmark datasets. The experimental results show that the proposed approach surpasses the competing approaches in both accuracy and efficiency.