Online Multiple Athlete Tracking with Pose-Based Long-Term Temporal Dependencies

Kong, Longteng; Zhu, Mengxiao; Ran, Nan; Liu, Qingjie; He, Rui

doi:10.3390/s21010197

Cited by 7 publications

(5 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The matching of new detections to existing tracks in the approach of Hurault et al [ 16 ] is based on a spatial appearance criteria learned in a self-supervised way with a triplet loss. The approach of Kong et al [ 87 ] is based on the players’ poses modeled by a LSTM network. For Hockey, Vats et al [ 81 ] resorted to an offline tracker [ 41 ] and two ResNet-18 networks [ 65 ] to identify teams and jersey numbers.…”

Section: Previous Workmentioning

confidence: 99%

Individual Locating of Soccer Players from a Single Moving View

Maglo,

Orcesi,

Denize

et al. 2023

Sensors

View full text Add to dashboard Cite

Positional data in team sports is key in evaluating the players’ individual and collective performances. When the sole source of data is a broadcast-like video of the game, an efficient video tracking method is required to generate this data. This article describes a framework that extracts individual soccer player positions on the field. It is based on two main components. As in broadcast-like videos of team sport games, the camera view moves to follow the action and a sport field registration method estimates the homography between the pitch and the frame space. Our method estimates the positions of key points sampled on the pitch thanks to an encoder–decoder architecture. The attention mechanisms of the encoder, based on a vision transformer, captures characteristic pitch features globally in the frames. A multiple person tracker generates tracklets in the frame space by associating, with bipartite matching, the player detections between the current and the previous frames thanks to Intersection-Over-Union and distance criteria. Tracklets are then iteratively merged with appearance criteria thanks to a re-identification model. This model is fine-tuned in a self-supervised way on the player thumbnails of the video sample to specifically recognize the fine identification details of each player. The player positions in the frames projected by the homographies allow the obtaining of the real position of the players on the pitch at every moment of the video. We experimentally evaluate our sport field registration method and our 2D player tracker on public datasets. We demonstrate that they both outperform previous works for most metrics. Our 2D player tracker was also awarded first place at the SoccerNet tracking challenge in 2022 and 2023.

show abstract

Section: Previous Workmentioning

confidence: 99%

Individual Locating of Soccer Players from a Single Moving View

Maglo,

Orcesi,

Denize

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Hurault et al [24] use a single network with a Faster R-CNN backbone [40] to detect small soccer players and extract re-ID features. Kong et al [28] mix player appearance, posture and motion criteria to match new detections with existing tracks. Vats et al [47] use a Faster R-CNN network [40] to detect hockey players and a batch method for tracking [5].…”

Section: Tracking With Re-identificationmentioning

confidence: 99%

Efficient tracking of team sport players with few game-specific annotations

Maglo

Orcesi

Pham

2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

One of the requirements for team sports analysis is to track and recognize players. Many tracking and reidentification methods have been proposed in the context of video surveillance. They show very convincing results when tested on public datasets such as the MOT challenge. However, the performance of these methods are not as satisfactory when applied to player tracking. Indeed, in addition to moving very quickly and often being occluded, the players wear the same jersey, which makes the task of reidentification very complex. Some recent tracking methods have been developed more specifically for the team sport context. Due to the lack of public data, these methods use private datasets that make impossible a comparison with them. In this paper, we propose a new generic method to track team sport players during a full game thanks to few human annotations collected via a semi-interactive system. Non-ambiguous tracklets and their appearance features are automatically generated with a detection and a reidentification network both pre-trained on public datasets. Then an incremental learning mechanism trains a Transformer to classify identities using few game-specific human annotations. Finally, tracklets are linked by an association algorithm. We demonstrate the efficiency of our approach on a challenging rugby sevens dataset. To overcome the lack of public sports tracking dataset, we publicly release this dataset at https://kalisteo.cea.fr/ index.php/free-resources/. We also show that our method is able to track rugby sevens players during a full match, if they are observable at a minimal resolution, with the annotation of only 6 few seconds length tracklets per player.

show abstract

“…For the pose estimation step, the authors chose a combined heatmap, offset, and regression approach, using heatmaps and offset losses only during training. Kong et al [90] proposed a framework consisting of the Pose-based Triple Stream Network (PTSN) and an online multi-state matching algorithm. PTSN is responsible for computing the similarity values between the historical tracklets and the candidate detection in the current frame.…”

Section: Pose Trackingmentioning

confidence: 99%

3D Pose Estimation and Tracking in Handball Actions Using a Monocular Camera

Sajina

Ivašić-Kos

2022

J. Imaging

View full text Add to dashboard Cite

Player pose estimation is particularly important for sports because it provides more accurate monitoring of athlete movements and performance, recognition of player actions, analysis of techniques, and evaluation of action execution accuracy. All of these tasks are extremely demanding and challenging in sports that involve rapid movements of athletes with inconsistent speed and position changes, at varying distances from the camera with frequent occlusions, especially in team sports when there are more players on the field. A prerequisite for recognizing the player’s actions on the video footage and comparing their poses during the execution of an action is the detection of the player’s pose in each element of an action or technique. First, a 2D pose of the player is determined in each video frame, and converted into a 3D pose, then using the tracking method all the player poses are grouped into a sequence to construct a series of elements of a particular action. Considering that action recognition and comparison depend significantly on the accuracy of the methods used to estimate and track player pose in real-world conditions, the paper provides an overview and analysis of the methods that can be used for player pose estimation and tracking using a monocular camera, along with evaluation metrics on the example of handball scenarios. We have evaluated the applicability and robustness of 12 selected 2-stage deep learning methods for 3D pose estimation on a public and a custom dataset of handball jump shots for which they have not been trained and where never-before-seen poses may occur. Furthermore, this paper proposes methods for retargeting and smoothing the 3D sequence of poses that have experimentally shown a performance improvement for all tested models. Additionally, we evaluated the applicability and robustness of five state-of-the-art tracking methods on a public and a custom dataset of a handball training recorded with a monocular camera. The paper ends with a discussion apostrophizing the shortcomings of the pose estimation and tracking methods, reflected in the problems of locating key skeletal points and generating poses that do not follow possible human structures, which consequently reduces the overall accuracy of action recognition.

show abstract

Online Multiple Athlete Tracking with Pose-Based Long-Term Temporal Dependencies

Cited by 7 publications

References 38 publications

Individual Locating of Soccer Players from a Single Moving View

Individual Locating of Soccer Players from a Single Moving View

Efficient tracking of team sport players with few game-specific annotations

3D Pose Estimation and Tracking in Handball Actions Using a Monocular Camera

Contact Info

Product

Resources

About