Athlete Pose & Tracking from Match Footage
Player tracking with persistent identities, full-body pose keypoints and ball tracking - per-frame quality that sports models can actually learn temporal patterns from.
The challenge
Sports models train on temporal consistency: one identity switch between two similar-looking players, or a few sloppy keyframes of pose, teaches the model that bodies teleport and joints jitter. The value of sports data is per-frame discipline sustained across thousands of frames of fast, occluded motion.
Broadcast footage - the most abundant source - fights back with camera cuts, zooms, replays and on-screen graphics. Training data assembled naively from broadcasts is contaminated by all four.
Pose adds its own consistency problem: the same skeleton definition must be applied identically across players, camera angles and motion blur, or the keypoint data encodes annotator disagreement instead of biomechanics.
Why this data matters
Sports analytics has moved from box scores to biomechanics: performance science wants movement load from pose, tactics wants formations and spacing from tracks, broadcasters want augmented replays, and scouting wants queryable movement profiles. All of it sits on the same substrate - identity-stable tracks and consistent pose.
The same schema transfers across sports - tennis, football, cricket, basketball - with sport-specific event vocabularies layered on top. Building the schema once and reusing it is what makes multi-sport datasets economical.
Fixed-camera capture (practice courts, training facilities) yields cleaner, longer tracks than broadcast; the two sources complement each other and Payana's pipeline handles both.
How we did it
- 1
Sequence segmentation
Usable play sequences are separated from replays, cuts and graphics, so training data contains only real, continuous motion. Replay and graphics frames are flagged, not silently deleted - the record shows what was excluded and why.
- 2
Identity-stable tracking
Players keep persistent IDs within each sequence, verified through occlusions and crossings. Jersey numbers and team assignments ride along as attributes where visible.
- 3
Pose keypoints
Full-body skeletons on a consistent keypoint definition are annotated on keyframes and tracked across the sequence, with human review concentrated on the frames where motion is fastest and occlusion heaviest.
- 4
Ball tracking
The ball - small, fast, and frequently invisible - is tracked with interpolation through occlusions and reviewer confirmation at every reappearance.
- 5
Event annotation
Time-stamped events (serves, shots, passes, fouls) are layered on top of the geometry, giving models the grammar of the match alongside the motion.
- 6
Review at speed
Reviewers scrub annotated playback at frame accuracy; identity switches and pose jitter are the two defects the review pass is explicitly designed to catch.
Inside the dataset
- Player tracks with persistent identities per sequence
- Full-body pose keypoints on a consistent skeleton definition
- Ball trajectory with occlusion-aware interpolation
- Team / jersey attributes where visible
- Time-stamped event annotations aligned to the video clock
- Replay and graphics frames flagged out of the training set
Quality control
The review pass is built around the two defects that ruin sports data: identity switches and keypoint jitter. Sequences are replayed with tracks and skeletons overlaid; any teleporting ID or vibrating joint is corrected before export, and reviewer decisions are tracked.
Versions ship with track-length distributions, keypoint visibility statistics and event counts, so an analytics team can see the dataset's temporal quality before building on it.
What was delivered
- Player tracks with persistent identities per sequence
- Full-body pose keypoints on a consistent skeleton
- Ball tracking and time-stamped event annotations
- Replay/graphics frames flagged out of the training set
Delivered as COCO (keypoint variant), YOLO pose, and JSONL event streams with fixed train/valid/test splits.
What the data enables
- Player-tracking and re-identification models
- Pose-based performance science (movement load, technique analysis)
- Tactical analytics: formations, spacing, pressing patterns from tracks
- Broadcast enhancement and automated highlight detection
Want a dataset like this?
Send us your images or video - we annotate, review, and deliver a training-ready dataset in your format.
Book an intro call