Moving target tracking technology is an important topic in the field of time domain sensors, which is widely used in medical diagnosis, intelligent robots, human-computer interaction, education, and entertainment. This article is based on the time domain vision sensor to explore the methods of sports target tracking and speed measurement. The article introduces the regional gray-scale correlation matching method in the vision sensor algorithm, the noise model of the time domain vision sensor, and the method of eliminating noise. At the same time, the tracking effects of SIFT algorithm, Mean Shift algorithm, and MSO algorithm on moving targets are studied, analyzing the ANF algorithm, TSML algorithm, and IHTLS based on the time domain visual sensor to compare the TUD-Crossing test set, the ETHBAHNHOF test set, and the SMOT data set; then the image matching was purified and analyzed based on the time domain visual sensor. The research results show that, in the method of exploring moving target tracking based on time domain vision sensor, when the improved SIFT algorithm uses feature adaptive fusion to describe the target, it can automatically allocate the proportion of color features and texture features, which improves the accuracy and reliability of moving target tracking. Among them, the DNP algorithm can control the final accuracy rate above 80%, and its recall rate is basically greater than 80%. In the case that the accuracy of rough matching is less than 50%, the final matching accuracy after processing by multiple purification algorithms is above 98%.