Fashion image retrieval is an important branch of image retrieval technology.With the rapid development of online shopping, fashion image retrieval technology has made a breakthrough from text-based to content-based. But there is still not a proper deep learning method used for fashion image retrieval. This article proposes a fashion image retrieval framework based on dilated convolutional residual network which consists of two major parts, image feature extraction and feature distance measurement. For image feature extraction, we first extract the shallow features of the input image by a multi-scale convolutional network, and then develop a novel dilated convolutional residual network to obtain the deep features of the image. Finally, the extracted features are transformed into high-dimensional features vector by a binary retrieval vector module. For feature distance measurement, we first use PCA to reduce the dimension of the extracted high-dimensional vectors. Then we propose a mixed distance measurement algorithm combined with cosine distance and Mahalanobis distance to calculate the spatial distance of the feature vectors for similarity ranking, which solves the problems of poor robustness in complex background fashion image retrieval and the inefficiency calculation of Mahalanobis distance. The experimental results show the superiority of our fashion image retrieval framework over existing state-of-the-art methods.