Distance metric learning is a powerful approach to deal with the clustering problem with side information. For semisupervised clustering, usually a set of pairwise similarity and dissimilarity constraints is provided as supervisory information. Although some of the existing methods can use both equivalence (similarity) and inequivalence (dissimilarity) constraints, they are usually limited to learning a global Mahalanobis metric (i.e., finding a linear transformation). Moreover, they find metrics only according to the data points appearing in constraints, and cannot utilize information of other data points. In this paper, we propose a probabilistic metric learning algorithm which uses information of unconstrained data points (data points which do not appear in neither positive nor negative constraints) along with both positive and negative constraints. We also kernelize our metric learning method based on the kernel trick which provides a non-linear version of the learned metric. Experimental results on synthetic and real-world data sets demonstrate the effectiveness of the proposed metric learning algorithm.