Mastitis is one of the most predominant diseases with a negative impact on ranch products worldwide. It reduces milk production, damages milk quality, increases treatment costs, and even leads to the premature elimination of animals. In addition, failure to take effective measures in time will lead to widespread disease. The key to reducing the losses caused by mastitis lies in the early detection of the disease. The application of deep learning with powerful feature extraction capability in the medical field is receiving increasing attention. The main purpose of this study was to establish a deep learning network for buffalo quarter-level mastitis detection based on 3054 ultrasound images of udders from 271 buffaloes. Two data sets were generated with thresholds of somatic cell count (SCC) set as 2 × 105 cells/mL and 4 × 105 cells/mL, respectively. The udders with SCCs less than the threshold value were defined as healthy udders, and otherwise as mastitis-stricken udders. A total of 3054 udder ultrasound images were randomly divided into a training set (70%), a validation set (15%), and a test set (15%). We used the EfficientNet_b3 model with powerful learning capabilities in combination with the convolutional block attention module (CBAM) to train the mastitis detection model. To solve the problem of sample category imbalance, the PolyLoss module was used as the loss function. The training set and validation set were used to develop the mastitis detection model, and the test set was used to evaluate the network’s performance. The results showed that, when the SCC threshold was 2 × 105 cells/mL, our established network exhibited an accuracy of 70.02%, a specificity of 77.93%, a sensitivity of 63.11%, and an area under the receiver operating characteristics curve (AUC) of 0.77 on the test set. The classification effect of the model was better when the SCC threshold was 4 × 105 cells/mL than when the SCC threshold was 2 × 105 cells/mL. Therefore, when SCC ≥ 4 × 105 cells/mL was defined as mastitis, our established deep neural network was determined as the most suitable model for farm on-site mastitis detection, and this network model exhibited an accuracy of 75.93%, a specificity of 80.23%, a sensitivity of 70.35%, and AUC 0.83 on the test set. This study established a 1/4 level mastitis detection model which provides a theoretical basis for mastitis detection in buffaloes mostly raised by small farmers lacking mastitis diagnostic conditions in developing countries.