Candy — physics verification on real footage

The gate refused. Detection is perfect. The declared plane is wrong.

Same gripper-release gate as ALOHA, over a different real-footage dataset (12 candy clips). Detection catches all injected violations at 1.000 recall. But clean clips false-flag at 1.000 rate because the declared support plane does not match where the candy actually rests. The calibration sweep (35 cells) passes in 0/35 cells — no plausible plane/scale rescues this run. The failed assumption: the support plane is declared from the scored clip rather than measured from independent calibration clips — an input-property gap, not a detector bug. A real deployment's metadata (declared object size, calibrated support plane, IMU, depth) would supply what a public dataset cannot.

← Back to Lab index

What we tested

Does the gripper-release gate generalise to a second real-footage dataset?

12/12

injected violations caught · 1.000 · target ≥ 0.950

6/6

clean clips false-flagged · 1.000 · target ≤ 0.020

[0.610, 1.000]

Wilson 95% CI on false positives

Charts

Flag rate by violation class (Candy)

flag rate

Methodology

Gripper-release gate on candy clips. Same gripper-channel Otsu gate as ALOHA. Only score free frames after measured release.
Calibration sweep. Sweep floor margin and metres-per-pixel across declared ranges. No cell passes — the assumption is invariant.
Picker funnel. Object found in all 12 clips. Not a tracking failure.

Seeds: 12 clips, single camera, tracked

FP ≤ 0.02
detection ≥ 0.95
coverage complete

Notes

Real tracked footage, gripper-release gated. Dataset: lerobot/aloha_static_candy (v1.0). Hardware: ALOHA stationary bimanual (2× ViperX 300), 3× RealSense D435i @ 640×480, 30 FPS. Object: Candy wrapper, ~5 g, metallized plastic film. Task: Pick, transfer, place in container. 50 episodes. Object diameter assumed; support plane declared. 0/35 calibration cells pass. Notebook: haga-candy-verification. HuggingFace: https://huggingface.co/datasets/lerobot/aloha_static_candy

Sample size n=12 clean clips. Single camera. Object diameter assumed (not declared). Support plane declared from scored clip (not measured from independent calibration clips). Calibration sweep: 35 cells (floor margin 0.75–0.95 × metres-per-pixel ±2×), 0/35 pass — the support plane assumption is invariant. This run does not prove the detector works on candy-like objects; it proves the gate refuses when calibration is missing.

Run it yourself

The notebook below is the production pipeline, inlined and public. It pulls its own data and attaches no private dataset, so you can fork it and run it end to end on free compute. Thresholds are declared before the first episode runs, and a FAIL is published as measured.

Candy — physics verification on real footageNotebook runlerobot/aloha_static_candy

Real robot footage, gripper-release gated. Detection 1.000, FP 1.000. Calibration sweep (35 cells) passes 0/35 — support plane assumption invariant.

Simulation for the stress runs; real tracked robot footage for the physics verification runs. Neither has run on hardware.