Gemini Robotics evals

Gemini Robotics evaluations (1.0, 1.5, 2)

Google DeepMind's in-house real-robot tests of its Gemini Robotics models; task suites are described but not released.¹

Last checked 10 Oct 2026Basic entry12 of 19 facts checked at the sourceNext check 8 Apr 2027

Runs in

Real robots¹

Checked against real robots

Real robots

Skill

Handling objects

Robot

Two arms, Humanoid, Robot hand, Many types²

Licence

not released¹

Details

What it is: In-house test Inferred³
More
Classified by the Atlas from how the authors describe and distribute it.
Built by: Google DeepMind (Gemini Robotics Team)³
More
GR 1.5 author line: 'Gemini Robotics Team' with 171 other contributors.
Released: 2025-03 (Gemini Robotics tech report arXiv 2503.20020 v1 2025-03-25; blog 2025-03-12)⁴
More
Blog date from https://deepmind.google/blog/gemini-robotics-brings-ai-into-the-physical-world/.
Version: Gemini Robotics 2 evaluation (blog charts); methodology last documented in Gemini Robotics 1.5 report v3 Inferred²
More
Versions follow model releases: GR 1.0 (2025-03), GR 1.5 (2025-10, v3 2025-11), GR 2 (2026-07).
Last update: 2026-07: Gemini Robotics 2 blog (2026-07-30) publishes new success rates per skill category; Gemini Robotics 2 safety report dated 2026-07-29²
More
No GR 2 VLA technical paper found. Last full methodology: GR 1.5 report, arXiv 2510.03342 v3 (2025-11-28).

Runs in: Real robots¹
More
GR 1.5 Section 2.3: 'we perform A/B/n testing on real robots' for all comparisons in the report. Over 90% of development-time evaluation episodes ran in MuJoCo simulation, but reported comparisons are real.
Robot: Two arms, Humanoid, Robot hand, Many types²
More
GR 1.5 embodiments: ALOHA, Bi-arm Franka, Apollo humanoid (https://arxiv.org/html/2510.03342). GR 2: Apollo 2 with SharpaWave hands, Apollo 2 with Inspire hands, Franka Duo. On-Device 2 adaptation: SO101, Dexmate (https://deepmind.google/models/model-cards/gemini-robotics-on-device-2/).
Setting: Tabletop, Mixed Inferred⁵
More
Tasks span tabletop manipulation and settings such as a laundry room (GR 1.0); long-horizon tasks include trash sorting, desk organization, suitcase packing (GR 1.5); GR 2 shows a garage and a cluttered room.
Size: GR 1.0: generalization benchmark 85 tasks; dexterity benchmark 20 tasks; 20 trials per task for specialist long-horizon tasks (12 for the spelling game). GR 1.5: full benchmark 230 tasks across all embodiments; ALOHA generalization = 68 prior tasks + 5 new action tasks + 12 new task-generalization tasks; long-horizon benchmarks of 4 tasks on ALOHA and 4 on Bi-arm Franka.¹
More
GR 1.0 numbers from https://arxiv.org/html/2503.20020. GR 2 blog gives no task or trial counts.

Scored by: Progress score, Success rate, Composite index¹
More
GR 1.0: models are run back-to-back in random order on the same cell (A/B testing) and compared with a pairwise t-test (https://arxiv.org/html/2503.20020). Long-horizon progress is the sum of subtask points. Organiser-run, self-reported. From chart alt text. Each bar averages several tasks in a skill category (multi-finger bars are single tasks). Error bars shown without values. Trial counts and sim/real not stated on the blog or the VLA page (https://deepmind.google/models/gemini-robotics/vla/). Composite index; VQA answers graded by Gemini 2.5 Flash. See the separate ERQA record.
Leaderboard: None. Scores are only in papers.¹
More
Results only in reports and blogs.
Code licence: not released¹
More
No evaluation code release found in the reports or blogs.
Data licence: not released¹
Access: Closed¹
More
Task suites, rubrics are described in appendices, but robots, scenes and evaluation code are not released.

10 Oct 2026Created as a basic entry: identity facts checked at primary sources (phase 1 re-verification).