Robotics & Manipulation

Egocentric (head- or body-mounted) video is the closest human analogue to what a robot's own camera sees: the workspace, the objects, and the hands manipulating them, all from the actor's point of view. Payana captures this footage with wearable cameras while workers simply do their jobs - no scripting, no staging, no lab setup.

Each recording flows through an automated episode pipeline: a vision-language model narrates the action, detection and open-vocabulary models find every object and tool, segmentation produces pixel-accurate masks, speech becomes transcripts, and hand-pose estimation extracts a 7-dimensional end-effector state per frame - position, orientation, and gripper aperture, the same action space robot policies are trained in.

The result is demonstration data that pairs observation with action at every timestep - what the person saw and what their hands did - which is precisely the supervision imitation learning and behavior-cloning policies need. Long recordings of repeated work are automatically segmented into individual task cycles, so one shift of footage becomes hundreds of training episodes.

Every episode is human-reviewed before delivery, and off-task segments (breaks, interruptions, camera fumbles) are flagged by automatic curation so they never contaminate a policy's training set.