UMI-Bench
UMI-Bench 1.0
Real-robot benchmark of 10 tabletop tasks for policies trained on handheld UMI-style gripper data.1
Details
About
- What it is
- Benchmark Inferred1
More
Classified by the Atlas from how the authors describe and distribute it.
- Built by
- Soochow University; Lumos Robotics; Fudan University; Shanghai Jiao Tong University; Shanghai TeleAI; Shanghai AI Laboratory; INSAIT; Xi'an Jiaotong-Liverpool University. 19 authors.4
More
Affiliations from project page; arXiv abs page lists authors only. Checked 2026-10-10.
- Released
- 2026-06 (arXiv v1 2026-06-09; Hugging Face dataset created 2026-06-03)1
More
Checked 2026-10-10.
- Version
- 1.04
More
Checked 2026-10-10.
- Last update
- Hugging Face dataset last modified 2026-09-28; no newer paper version5
More
Hugging Face API lastModified. Checked 2026-10-10.
Setup
- Runs in
- Real robots1
More
Checked 2026-10-10.
- Robot
- One arm, Two arms2
More
Checked 2026-10-10.
- Robot model
- FastTouch tabletop arm (6-DoF URDF and Python SDK released); data collected with FastUMI Pro handheld device; wrist RGB cameras2
More
Checked 2026-10-10.
- Setting
- Tabletop2
More
1.2 m x 1.0 m x 0.75 m table with a 5 cm grid board. Checked 2026-10-10.
Scoring and access
- Scored by
- Success rate, Progress score, Human rating2
More
Full Success Rate (binary) and Progress Score (0-100, task rubrics); human evaluators annotate recorded rollouts. 'Overall Score' aggregates 50 rollouts per task-model pair; exact formula not given. Checked 2026-10-10.
- Trials
- 50 real-world rollouts per task (500 total per model)2
More
Checked 2026-10-10.
- Who runs it
- self-reported: local-first protocol run by each lab; paper results run by the authors Inferred1
More
Checked 2026-10-10.
- Leaderboard
- None4
More
Checked 2026-10-10.
- Code licence
- Unknown
More
Also searched GitHub for UMI-Bench repos; found only unaffiliated personal repos. Checked 2026-10-10.
- Data licence
- CC-BY-4.0 (Hugging Face dataset card)5
More
Checked 2026-10-10.
- Commercial use
- Allowed Inferred5
More
CC BY 4.0. Not legal advice. Checked 2026-10-10.
- Published at
- arXiv preprint (no comments or journal-ref)1
More
Checked 2026-10-10.
Sources 5
- 1UMI-Bench 1.0: An Open and Reproducible Real-World Benchmark for Tabletop Robotic Manipulation with UMI DataPaper · Jun 2026 · checked 10 Oct 2026
- 2UMI-Bench 1.0: An Open and Reproducible Real-World Benchmark for Tabletop Robotic Manipulation with UMI Data (full text)Paper · Jun 2026 · checked 10 Oct 2026
- 3Semantic Scholar API recordIndex · checked 10 Oct 2026
- 4UMI-Bench 1.0Official site · checked 10 Oct 2026
- 5UMIbenchmark/UMI-Benchmark-v1 on Hugging Face (dataset)Repository · checked 10 Oct 2026
Change history
- Created as a basic entry: identity facts checked at primary sources (phase 1 re-verification).