With the popularity of smart devices and the Internet, the volume of multimedia data is growing rapidly, and content-based image retrieval (CBIR) can search for similar images from large-scale images to realize the utilization of the data. For data owners, outsourcing the management and maintenance of image data to cloud service providers can effectively reduce costs, but there is a privacy leakage problem. In this paper, we focus on image feature extraction, index design, and image similarity recognition methods under a dual server model with content-based image security similarity recognition as the research topic, the work done such as proposing a BOVW (Bag of Visual Word) feature-based image security similarity recognition scheme. The scheme combines SIFT (scale-invariant feature transform) feature secure extraction and locally sensitive hashing algorithm to achieve secure extraction of BOVW features of images. To protect the BOVW features of images, an inverted index based on word frequency division is designed, the index is stored in chunks, and an image secure similarity recognition scheme based on CNN (convolutional neural networks) features is proposed. The scalable hash index based on dimensional division is designed based on the image CNN features secure extraction algorithm. The security and performance of the proposed scheme are theoretically analyzed and experimentally verified. Based on different image datasets, the impact of different parameters on the performance of the scheme is tested, and optimized parameters are given. The experimental results show that the proposed scheme in this paper can effectively improve the efficiency of analyzing the similarity of plant botanical art images compared to the existing schemes.