A bearing fault is one of the major causes of rotating machinery faults. However, in real industrial scenarios, the harsh and complex environment makes it very difficult to collect sufficient fault data. Due to this limitation, most of the current methods cannot accurately identify the fault type in cases with limited data, so timely maintenance cannot be conducted. In order to solve this problem, a bearing fault diagnosis method based on the fractional order Siamese deep residual shrinkage network (FO-SDRSN) is proposed in this paper. After data collection, all kinds of vibration data are first converted into two-dimensional time series feature maps, and these feature maps are divided into the same or different types of fault sample pairs. Then, a Siamese network based on the deep residual shrinkage network (DRSN) is used to extract the features of the fault sample pairs, and the fault type is determined according to the features. After that, the contrastive loss function and diagnostic loss function of the sample pairs are combined, and the network parameters are continuously optimized using the fractional order momentum gradient descent method to reduce the loss function. This improves the accuracy of fault diagnosis with a small sample training dataset. Finally, four small sample datasets are used to verify the effectiveness of the proposed method. The results show that the FO-SDRSN method is superior to other advanced methods in terms of training accuracy and stability under small sample conditions.