Fish object detection and counting in pelagic fisheries face many challenges in complex environments. Sonar imaging technology offers a solution because it generates high-resolution images underwater. In this paper, we propose a sonar-based fish object detection and counting method using an improved YOLOv8 combined with BoT-SORT to address issues such as missed detection, false detection, and low accuracy caused by complex factors such as equipment motion, light changes, and background noise in pelagic environments. The algorithm utilizes the techniques of lightweight upsampling operator CARAFE, generalized feature pyramid network GFPN, and partial convolution. It integrates with the BoT-SORT tracking algorithm to propose a new region detection method that detects and tracks the schools of fish, providing stable real-time fish counts in the designated area. The experimental results indicate that while focusing on maintaining a lightweight design, the improved algorithm achieved a 3.8% increase in recall and a 2.4% increase in compared to the original algorithm. This significantly impacts scientific and rational fishery planning, marine resource protection, and improved productivity. At the same time, it provides important data support for marine ecological monitoring, environmental protection, and fishery management, contributing to sustainable fishery development and marine ecology preservation.