ManiSkill3

The paper's conclusion calls ManiSkill3 a 'framework/benchmark'; the GitHub description says 'an open source GPU parallelized robotics simulator and benchmark'. We file it as a simulator that hosts many tasks because it has no single fixed task set or score.

23 authors; Hao Su is last author. 19 list UC San Diego, and 7 of those also list Hillbot. Hillbot and the Qualcomm Embodied AI Fund are thanked for support.

UC San Diego · Hao Su lab; 19 of 23 authors¹

Hillbot · Robotics company; second affiliation of 7 authors including Stone Tao, Fanbo Xiang and Hao Su¹

Carnegie Mellon University · Chen Bao¹

TU Dresden · Roberto Calandra¹

Tsinghua University · Rui Chen¹

King's College London · Shan Luo¹

First 3.x package on PyPI on 2024-03-09 (3.0.0.dev0); first GitHub release 2024-03-28; arXiv v1 2024-10-01; RSS 2025.

arXiv v2 (2025-05-30) is the RSS 2025 demonstration-track text with a new title. The paper was also an oral at the ICLR 2025 Robot Learning Workshop.

3.0.1, released 2026-04-21, is the first release without the 'beta' label. Betas ran from 3.0.0.b2 (2024-05-02) to 3.0.0b22 (2025-12-05).

There is no 3.0.0 release: PyPI goes from 3.0.0b22 to 3.0.1. A tag v3.0.0b23 exists without a release. The roadmap plans to replace the PhysX backend with Newton/MuJoCo Warp, which would change the physics behind task scores.

3.0.0b11 (2024-10) · Release notes for b12 say an RNG bug in b11 sometimes stopped environments from randomizing on reset, and ask users to upgrade.²⁵

3.0.0b10 (2024-10-01) · Added the four SIMPLER Bridge real-to-sim twins and aligned evaluation metrics and baseline reporting.²⁶

Planned backend change · Roadmap: switch the PhysX backend to Newton/MuJoCo Warp.⁴¹

Last commit on main 2026-08-02. Last release 3.0.1 on 2026-04-21.

GitHub reports the last push to any branch on 2026-08-04.

Commits on main through 2026-08-02 and a release on 2026-04-21, both within six months. The roadmap plans API clean-up and a physics backend switch.

Rendering uses SAPIEN's parallel rasterizer; ray tracing is available without parallelization. GPU simulation is supported on Linux with NVIDIA GPUs only (README system table).

Docs have 35 robot pages (our count), including Franka Panda, xArm 6/7, UR10e, WidowX 250S, WidowX AI, SO100, Koch v1.1, Fetch, Google Robot, Unitree G1, H1 and Go2, ANYmal C, Allegro, D'Claw, Inspire hands and TriFinger. The paper says 20+ robots.

Counted from docs/source/robots/*/index.md on 2026-10-10. Some pages are variants of one robot (wrist camera, simplified legs).

RoboCasa kitchens, ReplicaCAD and AI2-THOR scenes load in ManiSkill3, but the docs say none of these large scene datasets has trainable tasks with success/fail conditions yet; RoboCasaKitchen-v1 has no success condition. So 'kitchen' is not tagged.

51 task entries in 9 documentation categories (our count). The paper groups tasks into 12 categories.

Counted 2026-10-10 from the generated task tables and pages: table-top 21, control 9, humanoid 4, digital twins 4, quadruped 3, mobile manipulation 3, dexterous 3 task families (some with difficulty levels 0 to 4), drawing 3, external 1 (ManiSkill-HAB). Of the 44 entries in task tables, 35 have success/fail conditions and 17 have demonstrations.

3 room-scale scene datasets: ReplicaCAD, AI2-THOR-family scenes (for example ArchitecTHOR) and RoboCasa kitchens

Used for exploration and data generation. Only ManiSkill-HAB defines scored tasks in ReplicaCAD.

Official demonstrations for 16 tasks on Hugging Face, about 0.47 GB of compressed files. Motion-planning sets hold 1,000 trajectories per task (two tasks checked). The paper says 'millions of demonstration frames'.

Task count and size summed by us from the Hugging Face file listing (16 zip files, 473,816,098 bytes). Episode counts read from trajectory.json for PickCube-v1 and PegInsertionSide-v1. Files store states and actions only; observations are regenerated with the replay tool.

Sources · Motion planning, RL policies and teleoperation, labelled in each file's metadata (for example 10 teleoperated PickCube-v1 demos)⁴⁵⁴⁶

Per-backend files · File names record the controller and the simulation backend (physx_cpu or physx_cuda)²⁴⁴⁵

Start states are randomized every episode. Some tasks also change object geometry (for example peg shapes in PegInsertionSide). No held-out object split is documented.

The docs' evaluation setup re-randomizes every episode (reconfiguration_freq=1), which 'randomizes object geometries if the task has object randomization'. We found no statement that evaluation objects are unseen in training.

Tasks define success/fail conditions and, for most, dense rewards for RL. There is no benchmark-wide headline score.

Each episode records success_once (succeeded at any step), success_at_end (succeeded at the last step), the matching fail flags and the return. The docs say learning-from-demonstration work typically reports success_once. No benchmark-wide average.

The two success metrics can differ for the same rollouts, so a paper must say which one it reports.

The docs give an evaluation script (no early resets, re-randomize every episode, 64 parallel environments in the example) but no required number of episodes.

ManiSkill3 paper, imitation baselines · Best success rate obtained during training (Tables II and III); PerAct evaluated over 100 episodes¹

RDT-1B five-task protocol · 250 trials per task (10 seeds x 25), mean and standard deviation¹⁰

RL baselines · Evaluation curves shared as Weights & Biases reports; RL standard benchmark small set listed in the docs²⁹

No organiser runs submissions. Each paper runs its own evaluation.

The paper's RL curves show 95% confidence intervals over 5 seeds and the Koch sim2real curves show 95% intervals; its imitation tables give single numbers. RDT reports mean and standard deviation over 10 seeds; E0 gives single numbers.

No leaderboard in the README, docs or paper. The authors share baseline training runs as Weights & Biases reports, which are not a results table for outside methods.

LICENSE is Apache-2.0. Since 2025-09-14 a LICENSE-3RD-PARTY file adds the BSD licence of copied PyTorch3D code, after issue #1208 flagged a file under LGPL-3.0. The README says rigid-body environments use 'fully permissive licenses (e.g., Apache-2.0)'.

The card covers haosulab/ManiSkill_Demonstrations. The README's asset licence (CC BY-NC 4.0) may also apply to rendered observations of those assets; the card does not say.

README: 'The assets are licensed under CC BY-NC 4.0'. Individual sources carry other labels (items).

CONFLICT between the blanket README statement and per-source labels. Terms for PartNet-Mobility assets (downloaded from UCSD storage) could not be checked: the SAPIEN site did not respond on 2026-10-10.

Robot models · Own LICENSE files: Allegro (BSD-style, SimLab), D'Claw (Apache-2.0, ROBEL), Unitree G1 (BSD-3-Clause), Koch (Apache-2.0, Hugging Face), SO100 (Apache-2.0)²⁰

Scene copies on Hugging Face · haosulab/ReplicaCADRearrange and haosulab/AI2THOR cards say cc-by-4.0; haosulab/RoboCasa says mit, while RoboCasa's own README says CC BY 4.0²¹

pip install mani_skill; assets and demonstrations download from public Hugging Face and UCSD links without registration. Since a 2026-06-07 link update, the docs point to mani-skill/ManiSkill_Demonstrations, which returns HTTP 401; the download tool still uses the working haosulab repository.

Code (Apache-2.0) allows commercial use. The README licenses the bundled assets under CC BY-NC 4.0, which bars commercial use, even though some individual assets carry permissive labels. A company could use the simulator with its own assets. Not legal advice.