Machine learning only uses single-channel grayscale features to model the target, and the filter solution process is relatively simple. When the target has a large change relative to the initial frame, the tracking effect is poor. When there is the same kind of target interference in the target search area, the tracking results will be poor. The tracking algorithm based on the fully convolutional Siamese network can solve these problems. By learning the similarity measurement function, the similarity between the template and the target search area is evaluated, and the target area is found according to the similarity. It adopts offline pre-training and does not update online for tracking, which has a faster tracking speed. According to this study, (1) considering the accuracy and speed, the target tracking algorithm based on correlation filtering performs well. A sample adaptive update model is introduced to eliminate unreliable samples, which effectively enhances the reliability of training samples. With simultaneous changes in illumination and scale, fast motion and in-plane rotation IPR can still be maintained. (2) Determined by calculating the Hessian matrix, in the Struck function, Bike3 parameter adjustment can achieve fast tracking, and Boat5 ensures that the system stability is maintained in the presence of interference factors. The position of the highest scoring point in the fine similarity score map of the same size as the search image is obtained by bicubic interpolation as the target position. (3) The parallax discontinuity caused by the object boundary cannot be directly processed as a smooth continuous parallax. The MeanShift vector obtained by calculating the target template feature and the feature to be searched can increase the accuracy by 53.1%, reduce the robustness by 31.8%, and reduce the error by 28.6% in the SiamVGG algorithm.