This paper proposes a target tracking algorithm based on mean shift and template matching. The algorithm is divided into three stages:prediction, template matching, target positioning, and template updating. In the prediction stage, combined with the target position obtained from the previous frame tracking, the target position is predicted using the mean shift method, and the template matching search gate is defined with the predicted position as the center and the corresponding size as the coverage area. At the template matching stage, using fast template matching algorithm, the target template and search gate are quickly matched from coarse to fine, and the matching degree between matching result and target template is calculated. If the matching degree is greater than the given threshold, the fast template matching will be performed and the result will be used as the tracking result of the current frame image. Otherwise, the target position predicted by the mean shift algorithm is used as the tracking results of the current frame image. Finally, the template updating process is controlled by the tracking results of the current frame to update the target template, and the stable tracking of the target is finally completed. At the same time, the algorithm improves the robust of tracking by combining the advantages of color and edge features to the insensitivity of rotation and deformation. The method has fast calculation speed and high accuracy, it can meet real-time requirements.