The task of tracking a moving target by the UAV has two major challenges: one is that the multi-angle and multi-scale changes caused by the target’s movement make the target detection more difficult, the other is that the stability and accurate of target tracking are difficult to guarantee. In order to solve these problems, using monocular vision to get the image information of the target and YOLOv4 is used to detect multi-angle and multi-scale moving targets. Then based on the image visual servo control method, the PID controller is designed, and the deviation between the target pixel and the ideal pixel is used as the input, the output of the controller is the desired pixel speed of the UAV in image coordinate system. the tracking speed of the UAV under the machine system is solved according to the image Jacobian matrix, and a tracking strategy is designed to improve the tracking accurate and stability. In the process of target tracking, the difference between the x-axis and Y-axis coordinates of the target center point and the image center point is less than 30 pixels. The results show that the proposed algorithm can keep the target in the center of the image.