Robotics & Embodied AI
Robot reward model trained on successful and failed runs generalized to six unseen scenes

A robot reward model trained on both successful and failed demonstrations generalized to six scenes it had not seen in training, a 2026 paper reports. The model, Robometer, also improved robot learning in downstream tasks, the abstract says. A reward model scores how well a robot's actions move a task toward success. The work appeared in Robotics: Science and Systems.
Why did the researchers compare whole trajectories?
Earlier robot reward models mostly rely on frame-level progress labels from expert demonstrations. That approach scales poorly when suboptimal or failed data is common. Robometer combines frame-level progress supervision with comparisons between full trajectories, so it can learn from both successful and failed runs.
What was Robometer trained on?
The researchers used a dataset called RBM-1M with more than one million trajectories across 21 robot embodiments, meaning 21 robot bodies or platforms. The dataset included expert trajectories with reward labels and failed trajectories without reward labels. Robometer had two training goals: predict frame-by-frame task progress and learn trajectory-level preferences from pairwise comparisons. The researchers also trained it to predict per-frame task success.
How well did it generalize?
The researchers evaluated the reward model in six out-of-distribution scenes collected at three institutions. Out-of-distribution scenes are scenes unlike the training examples. The abstract reports that Robometer learned reward functions that generalized well across benchmarks and real-world evaluations.
What downstream reinforcement learning tasks did it support?
The paper says the learned rewards support offline and online reinforcement learning. The authors also describe uses in filtering and retrieving imitation learning data and in automated failure detection.
What are common questions about this work?
What is the training dataset?
RBM-1M contains more than one million trajectories across 21 robot embodiments. It pairs expert trajectories that have reward labels with failed trajectories that do not.
Does Robometer detect failures?
The paper lists automated failure detection as one downstream use of the learned reward functions.
What did the researchers release?
The abstract says code, model weights, and videos are available.
Limits and what's next
The abstract describes evaluations in six out-of-distribution scenes collected at three institutions. The evaluation setting and the RBM-1M dataset tie the findings to those conditions. RBM-1M spans more than one million trajectories across 21 robot embodiments. This is one study, and the abstract reports no independent replication. The paper makes code, model weights, and videos available, the abstract says.
Source: Anthony Liang et al., “Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisons,” Robotics: Science and Systems, 2026.