The number of labeled samples has a great impact on the classification results of very high-resolution (VHR) remote sensing image. However, the acquisition of available labeled samples is difficult and time-consuming. Faced with the limited labeled samples on high-resolution remote sensing image, semisupervised method becomes an effective way. In semi-supervised learning, accurate similarity prediction between unlabeled and labeled samples is very important. However, reliable similarity prediction between high-dimensional features is difficult. For more reliable similarity prediction for high-dimensional feature, a novel semi-supervised classification framework via improved metric learning (IML) with convolutional neural network (CNN) is proposed. In the proposed method, a novel trainable metric learning network is designed to accurately evaluate the similarity between high-dimension features. The vector distance parameter solving problem is transformed into a neural network design problem, which can automatically calculate parameters by BP algorithm. Finally, the pixel constraint mechanism is introduced to select the unlabeled samples. Experimental results conducted on three VHR remote sensing images, including Aerial, Xi'an, and Pavia University, and the results present that the proposed method performs better than the compared state-of-the-art methods.