Meat consumption is increasing globally. The safety and quality of meat are considered important issues for human health. During evaluations of meat quality and freshness, microbiological parameters are often analyzed. Counts of indicator cells can provide important references for meat quality. In order to eliminate the error of manual operation and improve detection efficiency, this paper proposed a Convolutional Neural Network (CNN) with a backbone called Detect-Cells-Rapidly-Net (DCRNet), which can identify and count stained cells automatically. The DCRNet replaces the single channel of residual blocks with the aggregated residual blocks to learn more features with fewer parameters. The DCRNet combines the deformable convolution network to fit flexible shapes of stained animal cells. The proposed CNN with DCRNet is self-adaptive to different resolutions of images. The experimental results indicate that the proposed CNN with DCRNet achieves an Average Precision of 81.2% and is better than traditional neural networks for this task. The difference between the results of the proposed method and manual counting is less than 0.5% of the total number of cells. The results indicate that DCRNet is a promising solution for cell detection and can be equipped in future meat quality monitoring systems.