Robot Policy Evaluation: Test Before Real Hardware

Robot policy evaluation is the evidence-gathering stage that decides whether a trained policy is safe and reliable enough to run on hardware. It answers more than “did the robot complete the task?” by measuring repeatability, failure modes, recovery, tracking error and the amount of human intervention required.

A useful evaluation starts before a policy reaches the robot: verify the file contract, run held-out simulation scenarios and preserve the exact policy version and test configuration. Hardware time is expensive and potentially unsafe, so it should test a policy that has already survived cheaper gates.

What should you measure in a robot policy evaluation?

Measure task performance and safety separately, then report both per scenario. One aggregate score hides whether a policy is consistently mediocre, occasionally excellent or unsafe in a known condition.

Define acceptance thresholds for the task before seeing the final results. A narrow walking-policy experiment and a long-running research demo have different tolerances, so universal numbers would create false confidence.

What is the policy evaluation ladder?

Use a staged ladder from cheap, reversible checks to supervised physical trials. Stop at the first stage that fails; a stronger result later does not cancel an earlier safety failure.

Gate 1: Validate the policy contract

Check observation shape, action shape, model identifier, API version, normalizer and file provenance. An ONNX file that loads is not necessarily the policy that was trained.

Gate 2: Replay known simulation scenarios

Run the exact training task and a small set of held-out seeds. Confirm that the exported policy performs like the checkpoint it came from. This catches export and preprocessing errors before they reach hardware.

Gate 3: Stress the simulation

Change terrain, friction, mass, command timing and disturbances one category at a time. Preserve the scenario list so two policies can be compared on the same evaluation set.

Gate 4: Test a realistic engine or CPU path

Run the controller with the deployment runtime where possible. A 50 Hz policy, for example, should be exercised with the same observation layout, timing assumptions and action conversion used onboard.

Gate 5: Bench the hardware

Put the robot on a stand or restraint. Check startup, policy load, joint direction, limit handling, stop behaviour and deadman or emergency controls before free-floor movement.

Gate 6: Run supervised floor trials

Start with short episodes, a clear area and an operator able to stop the robot. Increase duration and scenario diversity only after repeated clean runs.

How many robot trials are enough?

There is no universal trial count. The right sample depends on failure cost, policy variance and the size of the operating envelope. A useful screening rule is to preselect scenarios, run enough episodes to observe repeated failures rather than one fluke, and report confidence limits alongside success rates.

Never replace the scenario suite with a single long run. A ten-minute success can hide a failure that occurs only from a particular starting orientation or surface.

How do you evaluate sim-to-real transfer?

Keep separate scores for simulation, realistic simulation, bench tests and physical tests. Report the delta rather than claiming one overall transfer score.

• Simulation: compare the exported policy with the training checkpoint.
• Held-out simulation: test unseen seeds and randomized conditions.
• Bench: verify safe response without free movement.
• Physical floor: measure success and safety under supervision.
• Real-to-sim replay: reconstruct a physical failure in simulation and identify the missing or incorrect model term.

The sim-to-real robotics guide explains the modelling and randomization side. Evaluation closes the loop by proving whether those choices improved real behaviour.

What should a policy evaluation report contain?

A compact report should let another engineer reproduce the result:

1. Policy checkpoint and exported-file hash.
2. Robot model, runtime version and configuration.
3. Simulator version, scene and evaluation seed set.
4. Physical surface, battery state and safety setup.
5. Per-episode result and failure category.
6. Aggregate success, fall and intervention rates.
7. The exact stop rule and any threshold breaches.
8. A replay or log reference for every critical failure.

Avoid combining policies with different normalizers, action mappings or hardware settings into one chart. Those are different experiments, even if the task description is the same.

How does Microduck support policy evaluation?

Microduck exports policies to ONNX and runs them on a 50 Hz onboard loop. The official manifest records the robot model, API version, observation/action widths and whether a policy is episodic or perpetual. The runtime rejects incompatible shapes before replacing the active policy.

The policy guide covers manifest fields, simulation-first testing and reset commands. The training guide covers evaluation and export, while the software architecture explains where safety sits relative to the motor bus.

No evaluation result should be inferred from the existence of a policy file. The policy must be replayed at the correct shape and control rate, then tested through the hardware ladder.

Robot policy evaluation FAQ

What is robot policy evaluation? It is the process of measuring whether a trained controller completes its task safely, repeatably and under stated operating conditions.

Is simulation evaluation enough? No. Simulation is the first gate. Hardware transfer needs bench and supervised physical validation.

What metrics matter most? Task success, safety stops, falls, recovery, intervention rate and repeatability. Tracking error matters when the task includes velocity, pose or trajectory commands.

How many episodes should I run? Use a predefined scenario suite and enough episodes to expose repeated failures and estimate uncertainty. The required count depends on risk and task variance.

How do I compare two policies fairly? Use the same seeds, scenes, starting states, hardware setup and measurement code. Report per-scenario outcomes as well as aggregates.

What should happen after a physical failure? Stop the experiment, preserve logs, categorise the failure and reproduce it in simulation or in a safer bench setup before changing the policy.

Key takeaways

• Evaluate safety and task performance separately.
• Validate the exported contract before running any hardware trial.
• Use a fixed held-out scenario suite for policy comparisons.
• Record per-episode failures, not only average reward or success.
• Stage hardware tests through bench and supervised floor trials.
• Keep enough evidence for another engineer to reproduce the result.

Where to go next

• Sim-to-real robotics - modelling and transfer strategy
• How to train Microduck - train and export a policy
• Microduck policies - manifest, ONNX and safe loading
• Microduck training compute - compute and validation planning
• Biped robot reinforcement learning - locomotion-specific failure modes