Cross-task stress comparison

One perturbation budget, four manipulation primitives — they do not degrade the same way.

Same gate tier (success-ratio-min) applied to all four tasks. Lift and PickPlaceCan hold to moderate then collapse; Stack degrades earlier; Door shows no detectable response. The gate is relative to each task's own nominal, so low-nominal tasks are not mis-gated. Run-to-run reproducibility check shows Door's tiers can shift by up to 0.16.

← Back to Lab index

What we tested

How does the same mass/friction budget bite differently across Lift, Stack, PickPlaceCan, and Door?

1.00 → 0.26

Lift: nominal → severe

0.96 → 0.20

Stack: nominal → severe

1.00 → 0.24

PickPlaceCan: nominal → severe

0.68 → 0.62

Door: nominal → severe

Charts

Success rate across tasks and severity

rate

Methodology

Same protocol, four tasks. Run the same tiered mass/friction protocol on Lift, Stack, PickPlaceCan, and Door with paired seeds across conditions.
Gate per task. Each task is gated against its own nominal success (success_ratio_min × nominal). This prevents mis-gating low-nominal tasks.
Reproducibility check. Re-run Door on identical seeds to measure run-to-run spread. Documented spread up to 0.00 tiers.

Seeds: 0..49 paired across conditions (50 episodes × 4 conditions × 4 tasks)

gate: success_ratio_min × nominal per task
Wilson 95% CIs on all rates

Notes

Sim-only cross-task comparison. The gate is relative to each task's own nominal; an absolute threshold would mis-gate Door.

Run it yourself

The notebook below is the production pipeline, inlined and public. It pulls its own data and attaches no private dataset, so you can fork it and run it end to end on free compute. Thresholds are declared before the first episode runs, and a FAIL is published as measured.

Cross-task stress comparisonNotebook run

Same tiered protocol on Lift, Stack, PickPlaceCan, Door with paired seeds; gate relative to each task's nominal.

Simulation for the stress runs; real tracked robot footage for the physics verification runs. Neither has run on hardware.