RoboLab-120

RoboLab (RoboLab-120)

How to read this picture

Simulation benchmark of 120 tabletop manipulation tasks for generalist robot policies trained on real-world data.1

Sources
Last checked 10 Oct 2026Basic entry18 of 23 facts checked at the sourceNext check 8 Apr 2027
Runs in
Simulation2
Checked against real robots
Checked
Skill
Handling objects
Robot
One arm3
Used by
314
citations
Licence
Apache-2.05

Comparisons with real robots

Checked3

Details

Authors compared RoboLab-120 success rates with existing RoboArena (real-robot) Elo scores for 4 policies (pi0.5, pi0-FAST, pi0, PaliGemma): Spearman rho = 1.00, Pearson r = 0.68. No new real-robot runs in the paper; correlation computed by the RoboLab authors on RoboArena's numbers. Weak evidence: n = 4 policies, no task-level pairing.

Known problems 1

  1. Two supported simulator stacks with different physx builds may give non-comparable results

    Checked 2026-10-10.2

Details

About

What it is
Benchmark Inferred1
More

Classified by the Atlas from how the authors describe and distribute it.

Built by
Authors: Jenai Xuning Yang, Rishit Dagli, Alex Zook, Hugo Hadfield, Ankit Goyal, Stan Birchfield, Fabio Ramos, Jonathan Tremblay. All NVIDIA; Dagli also University of Toronto; Ramos also The University of Sydney.6
More

Checked 2026-10-10.

Released
2026-04 (arXiv v1 2026-04-10; GitHub repo created 2026-04-08)1
More

Checked 2026-10-10.

Version
RoboLab-120 (initial task set); code v0.3.1; paper arXiv v42
More

Checked 2026-10-10.

Last update
Code tag v0.3.1 published 2026-08-12 (CHANGELOG 0.3.1 dated 2026-08-11: Kinova Gen3 support, floor-standing robots, per-scene ground-height lock). Paper v4 2026-08-14. Latest default-branch commit 2026-09-12 (Cosmos3 policy client update). README news 2026/08: RoboVoLo task library released as an extension.7
More

Release list at https://github.com/NVlabs/RoboLab/releases; commit date from GitHub API. Checked 2026-10-10.

Setup

Runs in
Simulation2
More

README: task-based evaluation benchmark built on NVIDIA Isaac Lab. Checked 2026-10-10.

Robot
One arm3
More

Paper experiments: Franka Panda 7-DoF + Robotiq 2F-85, external ZED 2i and wrist ZED mini cameras (DROID-style setup). Repo says tasks are not tied to one robot; v0.3.1 adds Kinova Gen3 (Robotiq 2F-85). Checked 2026-10-10.

Robot model
Franka Panda + Robotiq 2F-85 (paper); Kinova Gen3 (code v0.3.1)7
More

Checked 2026-10-10.

Setting
Tabletop8
More

Blog: 120 human-curated tabletop pick-and-place tasks. Checked 2026-10-10.

Scoring and access

Scored by
Success rate, Progress score3
More

Headline: overall success rate (fully completed episodes); also a normalized graded score with partial credit per subtask. Checked 2026-10-10.

Score
Paper protocol: 10 episodes per task; overall success rate across 120 tasks plus graded score. Leaderboard shows SR% and an undefined 'Score' column; most entries are N = 1,200 episodes (10 per task x 120 tasks, inferred), Lakeside 3,000.9
More

Checked 2026-10-10.

Who runs it
Unknown
More

Checked 2026-10-10.

Leaderboard
Official leaderboard9
More

Checked 2026-10-10.

Code licence
Apache-2.05
More

Read the LICENSE file. Checked 2026-10-10.

Data licence
No demonstration dataset is distributed (policies fine-tuned on DROID). Bundled 3D object assets carry their own licences: handal, hope, basic, fruits_veggies, objaverse = CC BY-NC-SA 4.0; hot3d = CC BY-SA 4.0 plus HOT3D Dataset License non-sale restriction; ycb = MIT.10
More

Checked 2026-10-10.

Commercial use
unclear: code Apache-2.0 but several bundled asset folders are non-commercial (CC BY-NC-SA 4.0) Inferred10
More

Not legal advice. Checked 2026-10-10.

Published at
Robotics: Science and Systems XXII (RSS 2026), Sydney (arXiv journal-ref)1
More

Checked 2026-10-10.

Sources 10

  1. 1RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist PoliciesPaper · Apr 2026 · checked 10 Oct 2026
  2. 2NVlabs/RoboLab on GitHub (repository)Repository · checked 10 Oct 2026
  3. 3RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies (full text)Paper · Apr 2026 · checked 10 Oct 2026
  4. 4Semantic Scholar API recordIndex · checked 10 Oct 2026
  5. 5NVlabs/RoboLab on GitHub (blob)Repository · checked 10 Oct 2026
  6. 6RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist PoliciesOfficial site · checked 10 Oct 2026
  7. 7NVlabs/RoboLab on GitHub (file CHANGELOG.md)Repository · checked 10 Oct 2026
  8. 8How to Evaluate General-Purpose Robot Policies for Real-World Deployment | NVIDIA Technical BlogOfficial blog · checked 10 Oct 2026
  9. 9RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist PoliciesLeaderboard · checked 10 Oct 2026
  10. 10NVlabs/RoboLab on GitHub (file THIRD_PARTY_NOTICES.md)Repository · checked 10 Oct 2026

Change history

  1. Created as a basic entry: identity facts checked at primary sources (phase 1 re-verification).