This paper develops a retrieval scheme for encrypted JPEG images based on a Markov process. In our scheme, the stream cipher and permutation encryption are combined to encrypt discrete cosine transform (DCT) coefficients for protecting JPEG image content's confidentiality. And thus, it is easy for the content owner to achieve the encrypted JPEG images uploaded to a database server. In the image retrieval stage, although the server does not know the plaintext content of a given encrypted query image, he can still extract image feature calculated from the transition probability matrices related to DCT coefficients, which indicate the intra-block, inter-block, and inter-component dependencies among DCT coefficients. And these three types of dependencies are modeled by the Markov process. After that, with the multi-class support vector machine (SVM), the feature of the encrypted query image can be converted into a vector with low dimensionality determined by the number of image categories. The encrypted database images are conducted similarly. After low-dimensional vector representation, the similarity between the encrypted query image and database image may be evaluated by calculating the distance of their corresponding feature vectors. At the client side, the returned encrypted images similar to the query image can be decrypted to the plaintext images with the help of the encryption key.