Whole-body data for whole-body control.
Humanoid and legged programs need labels that follow the entire body — joints, contacts, balance, and terrain — and evaluation that knows a stumble on gravel is not the same as a stumble on tile.
The challenge
Legged locomotion data is dense in ways image datasets never were: dozens of joints, foot contacts that flicker on and off, terrain that changes underfoot, and failure that arrives in a fraction of a second. Annotating it requires keypoints and events on a shared timeline with proprioception, and evaluating it requires slicing by surface, slope, gait, and payload. Without that structure, teams see an aggregate success rate and no idea which conditions are dragging it down.
Built for this data
Whole-body keypoints
Body and hand keypoints across multi-camera rigs with occlusion handling and proprioceptive cross-checks.
Contact and balance events
Foot-strike, lift-off, slip, and recovery events proposed from force and IMU signals and confirmed by annotators.
Terrain segmentation
Surface class, slope, and obstacle masks fused from cameras and depth so locomotion policies learn what they stepped on.
Gait-aware evaluation
Performance sliced by gait, speed, surface, slope, and payload, with sim-to-real gaps per slice.
Failure clustering
Falls and near-falls grouped by cause — slip, trip, controller saturation, perception miss.
Safety-critical governance
Audit trails on every label and decision for programs operating near people.
Data and labels
- Multi-camera video with synchronized joint encoders, IMU, and foot force sensors
- Whole-body and hand keypoints with visibility flags
- Contact, slip, and recovery event markers
- Terrain and obstacle segmentation masks
- Motion-capture reference where available
Typical workflow
Ingest trials
Locomotion and manipulation trials with full proprioception and video.
Propose events
Agents detect contact and balance events from sensor signals for confirmation.
Label body and terrain
Keypoints and terrain masks annotated with cross-view consistency checks.
Evaluate by condition
Policies scored per surface, slope, gait, and payload with failures traced to trials.
Outcomes teams work toward
Common questions
Do you integrate with motion-capture systems?
Yes. Mocap can be imported as reference for keypoint validation and evaluation.
Can contact events be derived automatically?
Agents propose them from force and IMU data; a person confirms before they enter the dataset.
Is hand manipulation covered?
Yes, with dexterous hand keypoints and grasp outcome labels alongside locomotion.
Show us an episode. We'll show you the gaps.
Start with a walkthrough on your own data.