Accurate counting is difficult in the case of large numbers of overlapping and adhering fry. In this study, we propose a lightweight target detection counting method based on deep learning methods that can meet the deployment requirements of edge computing device for automatic fry counting while obtaining a high counting accuracy. We improve the structure of YOLOv4-tiny by embedding different attention mechanisms in the CSP blocks of the backbone network to enhance the feature extraction performance. In addition, the low efficiency of feature fusion in the original model is also addressed by adding different attention mechanisms to the neck network structure to promote the effective fusion of deep feature information with shallow feature information and improve the counting accuracy. The experimental results showed that the six models proposed in this study improved the model accuracy and recall to varying degrees compared with the original YOLOv4-tiny model, while retaining the advantages of the YOLOv4-tiny model in terms of its small number of parameters and fast inference rate. It was also shown that the CBAM(n)-YOLOv4-tiny model obtained by adding the CBAM to the neck network showed the most significant improvement, with an mean average precision (mAP) of 94.45% and a recall of 93.93%.