The bees’ in-and-out activity is an essential indicator of bee behavior and can be useful to determine the colony condition. The traditional counting of bees’ in-and-out activity mainly relies on manual work, which is time-consuming and inefficient. Therefore, there is a need to devise some more efficient alternative methods. For this purpose, the present study was conducted. This study proposed a bee in-and-out activity counting method based on video detection, including object detection, multiple object tracking (MOT), and an in-and-out activity counting algorithm. Images and video stream data were captured using a camera (Huiboshi model X20, Shenzhen Huiboshi Technology, Shenzhen, China) in a smart beehive. Two bee in-and-out counting methods, single-line and box methods, were designed. After three levels of evaluation, the results showed that the best counting model consists of YOLOv8m, OC-SORT, and the box method, which achieved F1in of 91.49% and F1out of 89.08%. The model can track and count honey bees in a complex environment and can be used to analyze their activity and colony condition.