The key object tracking in sports video scenarios poses a pivotal challenge in the analysis of sports techniques and tactics. In table tennis, due to the small size and rapid motion of the ball, identifying and tracking the table tennis ball through video is a particularly arduous task, where the majority of existing detection and tracking algorithms struggle to meet the practical application requirements in real-world scenarios. To address this issue, this paper proposes a combined technical approach integrating detection and discrimination, tailored to the unique motion characteristics of table tennis. For the detector, we utilize and refine a common video differential detector. As for the discriminator, we introduce GMP (a Graph Max-message Pass Neural Network), which is designed specifically for tracking table tennis balls or similar objects. Furthermore, we enhance an existing dataset for table tennis tracking problems by enriching its scenarios. The results demonstrate that our proposed technical solution performs impressively on both the dataset and the intended real-world environments, showcasing the good scalability of our algorithms and models as well as their potential for application in other scenarios.