Independent verification for physical AI

We check your policy and your world model, and we publish the runs where the check failed.

Haga applies one adversarial method to the two artifacts that decide what a robot does: the policy, and the world it is trained in. Every result ships as a number, an interval, and a limitation.

Request an evaluation Read the lab

Every number on this site is read from the same Kaggle artifact files the Lab runs write — read from the artifact, not retyped.

Problem

A world that breaks physics teaches a policy to break physics.

World models are trained on synthetic frames and policies are trained inside simulated worlds. Both are graded by the team that built them. A policy that looks robust in a world with wrong physics will not survive contact with a world with right physics. The two failures are the same failure, and nothing in the current workflow checks it.

Policy evaluation is a leaderboard, slow and public, and usually reached only after the release decision.
World-model evaluation is a demo reel. A model that freezes an object mid-air is shown, then shipped.
Neither artifact is measured against a criterion that was frozen before the run.

Method

One pipeline, two instruments.

Haga scores a submitted artifact with a fixed pipeline. The pipeline is the same for every run, so a number from one run is comparable with a number from another.

Policy stress: A policy is run under tiered mass and friction perturbations, in simulation, with paired seeds. The gate is relative to the policy's own nominal rate, so the comparison is fair across tasks of different difficulty.
World-model physics: Tracked trajectories from a generated world are checked for physical violations against calibrated detectors, with thresholds frozen before the held-out cohort is scored.

Flagship run

One run, end to end.

A scripted OSC_POSE position controller in robosuite, stressed by tiered mass and friction and scored against a pre-registered relative gate. Simulation only: no learned policy, no real robot. The per-task ladder below is the run's own: every rate is read from the same result file the rest of the page is read from.

PASSP1-E1 policy stress (robosuite, SAP v2) · gate is relative to each task's own nominal rate

Success rate at each stress tier, per task. A task whose rate never changed held its nominal success through every tier — that measures the harness's reproducibility, not its sensitivity.
Task	nominal	mild	moderate	severe	Moved
Lift	1.000	1.000	1.000	1.000	no — held nominal
Stack	0.900	0.840	0.840	0.820	yes
PickPlaceCan	1.000	1.000	1.000	1.000	no — held nominal
Door	0.800	0.760	0.660	0.760	yes

Read the full run →

Published

Three runs, published as measured.

Each line below is a link to the full run: the method, the table, the interval, and what the run does not show.

Open the lab →Read a sample report

Evaluation

Submit a policy or a world model.

Tell us what should be scored. We acknowledge every submission, agree the criterion and the thresholds before anything runs, and report what we find — including the failures.

Or write directly: haga@mushoodhanif.com