Traffic Scene Segmentation & Multi-Object Tracking
Vehicles, pedestrians and road furniture labeled with instance segmentation and persistent track IDs - the identity-disciplined data AV perception models train on.
The challenge
Traffic perception is where annotation quality is most visibly a safety issue. A perception model that misses a far-field pedestrian - because its training data never labeled people at 20-pixel sizes - fails exactly where the stakes are highest.
Tracking data has its own discipline problem: if the same car changes identity mid-sequence in the training data, the model learns that identities can teleport. Multi-object tracking datasets live or die on identity consistency through occlusions, crossings and re-entries.
And traffic has one of the longest tails in vision: auto-rickshaws, construction vehicles, temporary signage, hand carts. Datasets that dump the tail into 'other' produce models that hesitate precisely when they meet something unusual.
Why this data matters
Autonomous-vehicle and ADAS stacks consume several distinct supervision types at once: 2D boxes for detection, instance masks for fine boundaries, lane and free-space geometry for planning, sign and signal states for rules of the road, and tracked sequences for motion prediction. Each has its own failure modes in annotation.
Smart-city and traffic-analytics teams need much of the same data from fixed cameras - counting, flow, incident detection - usually at lower cost per frame but at massive volume.
In both cases the deciding factor is the same: the dataset must reflect the deployment distribution - the pixel sizes, weather, lighting and clutter the production model will actually see - or metrics measured on it are fiction.
How we did it
- 1
Locked taxonomy
Vehicle classes, pedestrians, cyclists, lanes, signs and signals are defined in an explicit taxonomy with visual decision rules before scale-up, so classes stay consistent across annotators instead of drifting into 'other'.
- 2
Instance segmentation
AI smart polygons trace exact object boundaries from a click or rough box; annotators refine, and every mask passes human review. Boxes alone are delivered where the training stack only needs detection.
- 3
Identity discipline
For video, each object keeps one track ID across its whole visible life - through occlusions and re-entries - propagated with AI tracking assistance and verified by reviewers before export.
- 4
Small-object passes
Zoomed review passes cover the far field, so pedestrians and vehicles at distance are labeled at the pixel sizes they actually appear at and small-object recall doesn't silently collapse.
- 5
Static scene layer
Lanes, signs and signals are annotated as their own layer - the static grammar of the scene - so planners and rule-of-road models get supervision alongside the dynamic objects.
- 6
Long-tail mining
Unusual classes are actively mined from the footage and explicitly represented in splits, instead of appearing three times in a million frames and never being learned.
Inside the dataset
- Instance masks and boxes per object with taxonomy class labels
- Persistent track IDs per object across each video sequence
- Lane, sign and signal annotations as a static scene layer
- Far-field objects labeled at true pixel sizes
- Long-tail classes explicitly represented in the splits
Quality control
Safety-critical data gets layered review: machine pre-labels are corrected by annotators, sequences get an identity-consistency pass (does every track survive its occlusions?), and a zoomed far-field pass checks small objects. Reviewer overrides are tracked per frame group.
Every version ships with quality metrics - class distributions, small-object counts by pixel-size bucket, and track-length statistics - so a perception team can judge dataset fitness before a single training run.
What was delivered
- Instance-segmentation masks per object with class labels
- Persistent track IDs for multi-object tracking sequences
- Lane / sign / signal static-scene annotations
- Human review on every frame group, with quality metrics
Exported as COCO JSON, YOLO, Pascal VOC and PNG masks; tracking sequences carry per-frame IDs for MOT training and evaluation.
What the data enables
- Detection and instance-segmentation training for AV / ADAS perception
- Multi-object tracking and motion-prediction datasets
- Traffic analytics: counting, flow, incident detection from fixed cameras
- Evaluation sets that reflect the true deployment distribution
Want a dataset like this?
Send us your images or video - we annotate, review, and deliver a training-ready dataset in your format.
Book an intro call