Genie Sim
Genie Sim Benchmark (AgiBot)
AgiBot's simulation benchmark scoring robot policies on instruction, spatial, robustness and manipulation boards, with a VLM judge.1
Comparisons with real robots
Checked2
Details
Paper Fig. 7: 16 model configurations, R^2 = 0.931, slope ~1.023 between sim and real scores; 50 trials per configuration in each. All 16 are pi0.5 fine-tuned on 4 tasks with 4 data mixes, measured by the AgiBot authors. It tests one policy family, not a ranking of different policies. RoboColiseum site claims a sim-to-real gap under 10% without showing a statistic.
Details
About
- What it is
- Benchmark Inferred5
More
Classified by the Atlas from how the authors describe and distribute it.
- Built by
- AgiBot (README: 'Genie Sim is the simulation platform from AgiBot'); 19 authors listed on arXiv, affiliations not shown on the abs page1
More
Checked 2026-10-10.
- Released
- 2025-04: repo created 2025-04-21 (earliest commits April 2025). Release log starts at v2.1 2025-06-25 (10 more manipulation tasks for AgiBot World Challenge 2025) and v2.2 2025-07-14 (evaluation metrics for all challenge tasks). Genie Sim 3.0: arXiv v1 2026-01-05, CES announcement 2026-01-06. Inferred1
More
First public date taken from repo creation (GitHub API); no v1.0 entry in the release log. Press release: https://agibot.com/article/231/detail/29.html Checked 2026-10-10.
- Version
- v3.2.0 (VERSION file)1
More
Checked 2026-10-10.
- Last update
- v3.2.0: release-log entry dated 2026-06-25 opens RoboColiseum and adds the 'spatial' board; GitHub release tag v3.2.0 published 2026-08-04; arXiv v4 2026-08-14; last push 2026-09-07.1
More
Checked 2026-10-10.
Setup
- Runs in
- Simulation2
More
Runs in NVIDIA Isaac Sim; README also lists Newton backends. Checked 2026-10-10.
- Robot
- Humanoid, Two arms Inferred2
More
Paper: data generated on AgiBot G1 and G2; title says 'for Humanoid Robot'. README: Genie G2 family is Tier 1; reference URDFs for Franka, UR5, Aloha, ARX, Agilex. Form factor of G1/G2 not checked at a primary source today. Checked 2026-10-10.
- Robot model
- AgiBot G1, AgiBot G2 (paper); reference URDFs for Franka, UR5, Aloha, ARX, Agilex (README)1
More
Checked 2026-10-10.
- Setting
- Retail or logistics, Industrial, Whole home, Office or lab, Kitchen1
More
README: assets cover retail, industry, catering, home and office. 'catering' mapped to kitchen. Checked 2026-10-10.
Scoring and access
- Scored by
- Success rate, Progress score, Automatic judge2
More
Board scores are 0-1 per task. A VLM judges whether task requirements are met. v2.2 release log: evaluation script records the score of all steps. Exact aggregation formula not given. Checked 2026-10-10.
- Trials
- 50 trials per configuration in the Sim2Real study; board trial counts not stated2
More
Checked 2026-10-10.
- Who runs it
- organiser-run on RoboColiseum: users submit a model and the pipeline runs automatically; README: debug locally, then launch an official evaluation against your inference server6
More
Checked 2026-10-10.
- Leaderboard
- Official leaderboard1
More
Baseline numbers are publisher-run. Checked 2026-10-10.
- Code licence
- MPL-2.0 for source/geniesim* and source/data_collection; source/scene_reconstruction has multiple licences4
More
Read the LICENSE file. GitHub API reports NOASSERTION. Checked 2026-10-10.
- Data licence
- CC BY-NC-SA 4.0 stated in the ModelScope dataset card README and in the gated Hugging Face asset card (AgiBot World Community License prompt). ModelScope metadata field says 'Apache License 2.0'.7
More
Conflict between card text and metadata field; card text recorded as primary. Asset card: https://huggingface.co/datasets/agibot-world/GenieSimAssets Checked 2026-10-10.
- Access
- Download after registering8
More
Checked 2026-10-10.
- Commercial use
- Not allowed Inferred7
More
From CC BY-NC-SA 4.0 on data/assets; code MPL-2.0 allows commercial use. Not legal advice. Checked 2026-10-10.
- Published at
- arXiv preprint only (no comments or journal-ref on abs page)5
More
Checked 2026-10-10.
Sources 8
- 1AgibotTech/genie_sim on GitHub (repository)Repository · checked 10 Oct 2026
- 2Genie Sim 3.0 : A High-Fidelity Comprehensive Simulation Platform for Humanoid Robot (full text)Paper · Jan 2026 · checked 10 Oct 2026
- 3Semantic Scholar API recordIndex · checked 10 Oct 2026
- 4AgibotTech/genie_sim on GitHub (blob)Repository · checked 10 Oct 2026
- 5Genie Sim 3.0 : A High-Fidelity Comprehensive Simulation Platform for Humanoid RobotPaper · Jan 2026 · checked 10 Oct 2026
- 6RoboColiseumLeaderboard · checked 10 Oct 2026
- 7GenieSim3.0-Dataset - ModelScope 魔搭社区Official site · checked 10 Oct 2026
- 8agibot-world/GenieSimAssets on Hugging Face (dataset)Repository · checked 10 Oct 2026
Change history
- Created as a basic entry: identity facts checked at primary sources (phase 1 re-verification).