The doctor determines whether there are lesions in the human body through the diagnosis of medical images, and classifies and identifies the lesions. Therefore, the automatic classification and recognition of medical images has received extensive attention. Since the inflammatory phenomenon of vascular endothelial cells is closely related to the varicose veins of the lower extremities, in order to realize the automatic classification and recognition of varicose veins of the lower extremities, this paper proposes a varicose vein recognition algorithm based on vascular endothelial cell inflammation images and multi-scale deep learning, called MSDCNN. First, we obtained images of vascular endothelial cells in patients with varicose veins of the lower extremities and normal subjects. Second, multiple convolutional layers extract multi-scale features of vascular endothelial cell images. Then, the MFM activation function is used instead of the ReLU activation function to introduce a competitive mechanism that extracts more features that are compact and reduces network layer parameters. Finally, the network uses a 3 × 3 convolution kernel to improve the network feature extraction capability and use the 1 × 1 convolution kernel for dimensionality reduction to further streamline network parameters. The experimental results tell us that the network has the advantages of high recognition accuracy, fast running speed, few network parameters, and is suitable for small-embedded devices. INDEX TERMS Vascular endothelial cells, inflammation, multi-scale deep learning, varicose veins of the lower extremities.