Traffic video is where annotation quality is most visibly a safety issue. Perception models for autonomous vehicles, advanced driver-assistance systems (ADAS), and smart-city analytics are only as reliable as the labeled data behind them - and traffic scenes are among the hardest to label well: dozens of objects per frame, constant occlusion as vehicles pass one another, tiny distant objects that grow into large near ones, motion blur, night scenes, and lens flare. Payana's traffic annotation service was built around exactly these conditions, and a full traffic dataset has already been produced end-to-end on the platform.
The economics of traffic annotation changed when segmentation models learned to propose boundaries. Hand-tracing a single busy intersection frame - fifteen vehicles, six pedestrians, two rickshaws - takes a trained annotator ten to fifteen minutes. The same frame on Payana takes under a minute: one click per object, each click returning a pixel-accurate polygon, followed by a human pass to correct the handful of boundaries the model got wrong. Multiply that difference across the tens of thousands of frames a perception dataset needs and it is the difference between a quarter and a week.
What we annotate in traffic scenes
A production traffic dataset is rarely one label type. It combines several, and the platform supports all of them natively in the same project workspace:
| Label type | Traffic use | Typical classes |
|---|---|---|
| Bounding boxes | Vehicle & pedestrian detection | car, bus, truck, motorcycle, bicycle, auto-rickshaw, pedestrian, animal |
| Instance segmentation | Pixel-accurate vehicle shape, free-space estimation | per-vehicle polygon masks |
| Semantic segmentation | Road-scene understanding | road, lane marking, sidewalk, vegetation, sky, construction zone |
| Video object tracks | Multi-object tracking (MOT), trajectory prediction | persistent per-vehicle identity across frames |
| Classification tags | Scene-level attributes | day/night, weather, congestion level, intersection type |
Regional realism matters more than most teams expect. Detectors trained only on Western freeway footage fail on dense, mixed traffic - two-wheelers weaving between lanes, auto-rickshaws, hand carts, pedestrians crossing mid-block, unmarked lanes, and vehicles that ignore lane discipline entirely. Because Baaz operates from India, Payana labels the real distribution of South-Asian road scenes as fluently as structured highway footage - precisely the data gap most global autonomous-driving datasets leave open. If your product will ever see an Indian, Southeast-Asian, or African road, that distribution needs to be in your training data, correctly labeled.
How AI-assisted traffic labeling works on Payana
Manually tracing a polygon around every car in every frame is the old way. On Payana, an annotator clicks once on a vehicle and the platform's GPU-backed segmentation engine returns a clean, tight polygon around it in about a second. For video, the annotator then triggers tracking, and the platform propagates that object through the entire clip, producing a mask on every single frame rather than sparse keyframes joined by interpolation. The distinction matters for traffic specifically: interpolated labels drift on turning, braking, or accelerating vehicles, while per-frame masks stay locked to the object through the whole maneuver.
Occlusion handling is built into the tracker rather than bolted on. When a tracked car disappears behind a bus, the track is automatically marked outside for those frames instead of hallucinating a box over the bus; the moment the car re-emerges, the same track resumes with a fresh mask and the same identity. Annotators queue multiple vehicles for tracking in one session - label object after object and the GPU works through the queue - so a 60-second intersection clip with fifteen tracked vehicles is a matter of minutes of human effort, not hours of tracing.
Motion-adaptive temporal smoothing removes the boundary flicker that raw frame-by-frame model output produces, without lagging genuinely fast motion: a vehicle braking hard or a pedestrian sprinting keeps a crisp, aligned boundary on every frame. The visual result is annotation with the stability of research-grade benchmark datasets, produced at service speed.
Dataset formats and delivery for AV and analytics teams
Traffic datasets export directly into the formats perception teams actually train with: COCO JSON for detection and instance segmentation, YOLO text labels with a generated data.yaml for real-time detectors, Pascal VOC XML for legacy pipelines, and PNG masks for semantic segmentation. Video tracks are unrolled into annotated frames with the tracking already applied, so a tracking project can train a frame-level detector with zero extra tooling on your side. Every export is a versioned snapshot with a reproducible train/valid/test split - request version 3 again six months later and you get byte-identical data, which is what makes model comparisons across time honest.
Quality control on safety-critical data
Every AI-proposed label passes through a human reviewer inside the same editor: boundaries are nudged vertex-by-vertex, misclassified vehicles re-labeled, missed objects added with another click. For safety-critical deliveries we run a second-pass review on a sampled subset and report class balance, objects-per-frame distribution, and small-object counts alongside the dataset - so your team knows what the model will actually see before training begins, not after the first confusing validation run.
Designing a traffic taxonomy that survives production
Most traffic datasets fail in the schema, not the drawing. A label set that looks complete on day one ("car, truck, bike, person") collapses the first time the model meets an ambulance, a tractor towing a trailer, or a person pushing a bicycle - is that a cyclist or a pedestrian? Before annotation starts we agree a written labeling guide with the client: the class list, the decision rules for every known ambiguity, and the attribute set. Attributes are where traffic schemas earn their keep: per-object flags for truncated (cut by the frame edge), occluded (partially hidden), and direction of travel let one dataset serve detection, tracking, and behavior-prediction models without re-annotation. The guide travels with the dataset, so a second annotation batch a year later - or a different vendor auditing our work - applies exactly the same rules.
Class granularity is a modeling decision disguised as a labeling decision, and we push clients to make it consciously. A parking-analytics product needs "car / two-wheeler / other"; an insurance-telematics model may need make-level granularity; an emergency-vehicle detection feature needs "ambulance" split out from "van" even though they look nearly identical from behind. Finer classes cost more per label and need more examples per class to train - our class-balance reports make that trade-off visible while there is still budget to adjust.
Small objects, far fields, and the long tail
The hardest labels in any traffic dataset are the smallest ones: a pedestrian thirty meters ahead of a dashcam is a few dozen pixels tall, and a motorcycle at the far side of a wide intersection may be smaller than the compression artifacts around it. These labels also matter most - perception systems need the most warning for the objects farthest away. Payana's editor supports deep zoom with crisp vector overlays, so annotators label far-field objects at pixel precision rather than skipping them, and our small-object counts per class are reported with every delivery so you know how much far-field signal the dataset actually contains. The long tail gets the same discipline: animals on the road, hand carts, construction equipment, overloaded vehicles - the rare classes that cause disproportionate real-world failures are labeled exhaustively when they appear, never sampled away.
Multi-object tracking datasets: identity discipline
Detection datasets ask "what is in this frame?"; MOT datasets add "and which one is it?" - and identity errors are the expensive kind, because a single identity swap corrupts two trajectories at once. The platform enforces identity discipline structurally: each tracked object is one persistent entity with per-frame geometry, occlusion gaps are explicit outside intervals rather than silent absences, and re-acquisition after occlusion continues the same entity rather than spawning a new one. Reviewers audit tracks by scrubbing the timeline - every track renders as a continuous band with its gaps visible - which makes identity swaps easy to spot and cheap to fix before export.
Who uses traffic annotation, and for what
Beyond AV perception teams, the same labeled data powers a wide product surface: smart-city analytics (congestion measurement, junction optimization, pedestrian-flow studies), violation detection (helmet detection on two-wheelers, wrong-way driving, signal jumping), tolling and parking (vehicle classification, occupancy counting), road-safety research (near-miss detection, conflict analysis), and insurance telematics (dashcam event classification). Each application weights the schema differently - a tolling classifier cares about axle-visible truck subclasses; a safety study cares about pedestrian posture and crossing behavior - and the pilot phase tunes the taxonomy to the product before the bulk of the budget is spent.
Lanes, signs, and signals: the static scene
Vehicles are only half of a road scene. The static infrastructure - lane markings, traffic signs, signals, curbs, crosswalks - carries the rules the dynamic objects obey, and models need both. Payana annotates lane markings as polygon regions per marking type (solid, dashed, double, zebra), traffic signs as detection classes with condition attributes (occluded, faded, vandalized - the states that break sign-recognition models in the field), and signal heads with per-frame state tags (red/amber/green) on video, which yields signal-phase datasets for intersection analytics without any extra tooling. Because static and dynamic labels live in one project, cross-cutting questions - "did vehicles stop at the stop line during the red phase?" - are answerable directly from the export, which is exactly the query structure violation-detection and intersection-safety products are built on.
What a traffic engagement looks like
A typical project runs in three moves. First, a schema pilot: you share a representative hour of footage across your camera types and conditions; we return a proposed taxonomy and a small annotated sample within days, and your ML team confirms the schema trains the way they expect. Second, bulk annotation against the frozen schema, with rolling deliveries - you start training on the first tranche while the second is being labeled, so model feedback can still steer the later batches (more night scenes, more two-wheelers, more rain). Third, versioned delivery with the class-balance, small-object, and per-condition reports described above. Follow-on batches - new cities, new camera positions, new seasons - append as new dataset versions, keeping every previously trained model reproducible while the data asset grows. For teams searching for a traffic video annotation service, an autonomous driving dataset partner, or vehicle detection and tracking labeling with South-Asian road realism, this is the engagement shape Payana runs every week.
Yes. The pipeline is viewpoint-agnostic: dashcam (ego-view), fixed CCTV at intersections, and aerial drone footage are all supported, including mixed datasets that combine all three.
The tracker marks the object outside for the frames it is not visible and automatically resumes the same identity when it reappears - no duplicate identities, no ghost boxes drifting over other vehicles.
Tracking runs in segments of up to roughly 3,000 frames (about 100 seconds at 30 fps) per pass, and longer videos are handled across passes. Every tracked frame receives its own mask - no interpolated in-between frames.
Yes - and we recommend including them deliberately. Scene-level tags (day/night, weather, congestion) are part of the schema, and class-balance reporting shows their distribution so the dataset covers the conditions your model must survive in production.