Gemini Robotics 2.5 vs GR00T 2: one shows a factory pilot, the other shows its method

Google DeepMind and NVIDIA announced new robot foundation models a day apart, and the obvious question is which one is better. The published evidence cannot answer it. The two companies tested different tasks on different robots against different baselines, counted success in different units, and released none of their headline numbers in a form a third party can rerun. What can be compared is the quality of the evidence, the data behind each model, who can use it and when, and what safety tooling comes with it. On those criteria the two releases differ sharply.

NVIDIA introduced GR00T 2 on 9 November at the Conference on Robot Learning (CoRL) in Austin, Texas, with a 135-page whitepaper that states trial counts, names its baselines and discloses where its own numbers are flattering. Google DeepMind introduced Gemini Robotics 2.5 on 10 November with a blog post, a factory pilot run with Apptronik and Mercedes-Benz, and an open jailbreak benchmark, but no linked technical report. Gemini Robotics 2.5 is going to partners. GR00T 2 is described as open, with code promised for December and no published licence.

The Frontier Wire has not run either model. This comparison is built from the two announcements, the GR00T 2 whitepaper and research page, DeepMind’s model pages, the ASIMOV dataset card on Hugging Face, the October Gemini 4 Argon safety report and two trade-press accounts. Every result below is reported by the company that built the model unless stated otherwise.

TL;DR

• No figure in either announcement can be ranked against a figure in the other. The tasks, robots, baselines and success units differ.
• GR00T 2’s whitepaper gives trial counts, blinded comparisons and its own caveats, including that its 95% five-demonstration result is “high partly by construction.”
• Gemini Robotics 2.5’s 96% factory-kitting figure comes from a real production line but is published without a denominator or a definition of a success.
• Neither model is downloadable today. DeepMind is rolling out to partners; NVIDIA says code arrives in December and has not published weight licence terms.
• DeepMind’s open ASIMOV-Jailbreak benchmark is the most reusable safety artifact in either release, but its 99.5% headline belongs to Gemini 4 Argon, not Gemini Robotics 2.5.

The criteria before the verdict

A comparison publishes its criteria first so a reader can disagree with the weighting. These five criteria follow what a robotics team would need to decide whether to invest time in either model. Capability is deliberately not a row, because the evidence for capability is not comparable across the two releases.

The table records what each company published. It does not say which model performs better on a robot, which is the question neither company’s materials let anyone settle.

A reconstructed timeline

Release, paper, event and publication dates differ for both models. The sequence matters because it shows how much of each stack was already public before this week and what is still only promised.

Two different kinds of model

Gemini Robotics 2.5 is a vision-language-action (VLA) model: it takes camera images and a language instruction and outputs robot actions. DeepMind pairs it with a separate embodied reasoning model, Gemini Robotics ER, which plans and tracks a task and hands motor execution to the VLA. In the Mercedes-Benz pilot, according to DeepMind, ER kept track of the workflow while Gemini Robotics 2.5 controlled the hands and arms. DeepMind says the model works across robot types, naming Apptronik’s Apollo 2 humanoid with the five-finger, 22-degree-of-freedom SharpaWave hand, and the Franka Duo platform with two-fingered parallel grippers.

GR00T 2 is what NVIDIA calls a world-action model (WAM). According to the GEAR research page, one diffusion transformer denoises future video and robot actions together: the model reads recent video, the instruction and the robot state, generates a short predicted future and an action chunk, executes the chunk and repeats. Small per-robot heads turn the plan into motor commands. NVIDIA’s argument is that predicting how the scene should change lets the model learn physics from human video that contains no robot actions at all. GR00T 2 is built on NVIDIA’s Cosmos-Predict2.5-2B video model, a 2-billion-parameter transformer. A faster variant, GR00T 2 Turbo, encodes video with a more compressed representation so it can run on the Unitree G1 humanoid’s onboard Jetson Thor, where NVIDIA reports 124 ms per prediction. NVIDIA’s two documents describe Turbo differently: the whitepaper says it keeps the same 2B transformer, while the research page says it uses a smaller one.

The architectures share a higher-level pattern. Both companies put a reasoning model above the action model. In the Mercedes-Benz pilot DeepMind used Gemini Robotics ER to track the workflow; NVIDIA’s whitepaper pairs GR00T 2 with “a frontier agent” that steers it through what NVIDIA calls a WAM API. That shared pattern is one reason the comparison should look at the whole stack, not only the action model.

The Frontier Wire’s survey of world models, video generators and simulators explains why a model that predicts video is not automatically a physics simulator. The same caution applies here: GR00T 2’s predicted video is a planning signal, not evidence that the predicted contact forces are correct.

How DeepMind evaluated Gemini Robotics 2.5

DeepMind’s post says it evaluated the model across 200 distinct tasks, giving examples such as serving food with tongs, tying a bow on a gift box and pipetting a liquid. An earlier paragraph in the same post says “over 200.” Compared with “previous baselines,” DeepMind reports a 30% to 100% higher success rate depending on the type of task, and task completion 20% to 30% faster.

Those are the only aggregate capability figures in the announcement. The post’s charts add some detail. They compare Gemini Robotics 2.5 with Gemini Robotics 2 in four task groups (tabletop pick-and-place, multi-step and contact-rich work, precision kitting and insertion, and dexterous manipulation), and the chart descriptions give per-task success rates for a few tasks: closing the lid of a toolbox rose from 0% to 93%, unhanging a mug from 29% to 75%, packing pastries with tongs from 5% to 100%, and folding pants from 0% to 40%. Several conditions are still missing:

• The aggregate baseline is not named. The text says “previous baselines.” The charts use Gemini Robotics 2, but the post does not say whether the 30% to 100% range and the speed figure use the same comparison.
• The form of the gain is not stated. A 100% higher success rate could mean 30% rising to 60%, or 45% rising to 90%. Several chart tasks start from 0%, where no relative percentage applies, so the range must be calculated in some other way, perhaps per task group or in percentage points. The post does not say, and it gives no absolute success rate across the 200 tasks.
• Task types are not defined. The range of 30% to 100% is “depending on the type of task,” with no list of types or the task count in each.
• Trial counts, robots per task and success criteria are not stated. Nor is whether evaluators knew which model was running.

None of this means the results are wrong. It means a reader cannot tell how large the improvement is in absolute terms across the 200 tasks, or how much uncertainty surrounds the handful of per-task rates that are shown. DeepMind’s Gemini Robotics 2 announcement in July linked a separate safety technical report; the 2.5 post links none for the model itself.

How NVIDIA evaluated GR00T 2

The GR00T 2 whitepaper is unusually explicit about method, including where its numbers overstate. The headline few-shot figures in NVIDIA’s blog, 71% average success across 50 tasks with one demonstration per task and 95% with five, come from a benchmark the whitepaper calls Dexter-50. Its exact conditions, as NVIDIA reports them:

• Robot: Dexter, a bimanual robot (a Tianji robot with two 22-degree-of-freedom Sharpa hands).
• Tasks: 50 tasks that never appear in Dexter’s training data but do appear in YAM parallel-gripper teleoperation data, in human video, or both.
• Zero-shot: the pretrained model receives only the instruction. It succeeds in 40% of trials and completes 40 of the 50 tasks at least once.
• Few-shot: a rank-32 LoRA adapter (a small set of trainable weights added to a frozen model) is trained on 1 or 5 teleoperated Dexter episodes per task. Five episodes amount to under three minutes of robot data. Each run takes about 30 minutes on one RTX 5090, with the same hyperparameters for every task.
• Results: 71% average success with one episode and 95% with five, with 37 of the 50 tasks succeeding in every trial at five episodes.

Then comes the disclosure that matters most. In NVIDIA’s words: “The 50 tasks were chosen among those the model learns with five episodes (at least 80% success), so the five-episode rate is high partly by construction.” The task set was filtered on the outcome being reported. The whitepaper sections reviewed here do not say how many candidate tasks were screened out to reach 50, so the share of tasks GR00T 2 can learn from five demonstrations is unknown. The NVIDIA blog post repeats the 71% and 95% figures without this condition. The Robot Report’s account of the launch likewise attributes the figures and leaderboard rankings to NVIDIA without describing the method.

Elsewhere the whitepaper’s design is stronger. On its YAM long-horizon post-training benchmark, each result is based on up to 20 trials, against named baselines: π0.5, GR00T 1.7, FastWAM, MolmoAct2 and FLUX 3 Action. On those six YAM tasks NVIDIA reports 71.7% average success for GR00T 2 against at most 12.5% for the baselines. On a 71-task Dexter zero-shot suite, with three trials per task, the four models ran interleaved in the same sessions and the operator labelling success was not told which model was running.

NVIDIA’s checkpoint ranking used what it calls GearArena: blinded A/B trials in which two to four checkpoints run in randomized order on the same scene, by the same operator, within minutes. By the snapshot in the whitepaper it held 195 campaigns, 16,044 graded robot episodes and 11,767 pairwise comparisons across 10 YAM stations and more than 20 operators, collected between July and November 2026. The whitepaper’s limitations section adds that ties are 30% to 35% of matches, that independent replication comes from distinct tasks rather than trials (16 tasks on one suite, 38 on the current one), and that ratings from different suites are not comparable.

The 96% factory figure, read carefully

DeepMind’s strongest piece of evidence is the one NVIDIA has no counterpart for: a run on active production lines. With Apptronik and Mercedes-Benz, Apollo 2 humanoids performed automotive kitting at plants in Kecskemét, Hungary, and Sindelfingen, Germany. Kitting means sorting individual parts from bulk bins into trays in the exact order needed for each vehicle. DeepMind describes the robot reaching into supply bins, picking up metal automotive brackets and placing them into sequencing containers, “all while correcting for any gaps,” and reports a 96% success rate. It calls the work a proof-of-concept and a pilot.

A production line is a harder and more meaningful setting than a lab bench, and that deserves weight. But the published figure leaves out what a reader would need to interpret it:

• The unit of success. Is 96% the share of individual picks, of completed trays, or of shifts without intervention?
• The denominator. How many attempts, over what period, at which of the two plants?
• Intervention policy. Did a person reset failed picks, and do recoveries count as successes?
• The comparison. What is the success rate of the existing process, human or automated?

Apptronik’s chief executive, Jeff Cardenas, is quoted in DeepMind’s post saying “we saw the models improve dramatically in just a few weeks of post-training and carry over to Mercedes-Benz production with very little drop-off.” That tells us the system was post-trained for this task, which is normal, but the post gives no figures for the drop-off.

A worked example with hypothetical inputs

The two missing conditions, unit and denominator, change what 96% means. The inputs below are hypothetical and chosen only to show the arithmetic; DeepMind has not published the real values.

The first three rows show that a small pilot is consistent with a wide range of real reliability. The last two show why the unit matters: if 96% is per pick, a tray that needs ten correct picks would be finished cleanly about two times in three, assuming independent errors. If 96% is per tray, the per-pick rate is much higher. The two readings describe very different machines for a plant manager.

NVIDIA’s own whitepaper makes the general point: distinguishing 95% from 99% success takes about 285 trials per policy, and 99% from 99.9% more than 1,000, under a two-sided test at the 0.05 level with 80% power. The Frontier Wire’s analysis of benchmark sizes covers the same problem for AI agent benchmarks.

An evidence worksheet: why the numbers do not line up

The table below takes each headline number from both announcements and records its conditions. It is the reason this article does not rank the models.

Placing 96% next to 95% would compare a factory pick-and-place pilot on a humanoid with a filtered set of lab tasks on a bimanual robot after five demonstrations. They are not measurements of the same thing. The simulation leaderboard results are the closest thing to a shared yardstick in either release, and only NVIDIA reports them. DeepMind reports no public leaderboard result for Gemini Robotics 2.5, so they cannot be used to compare the two either.

The leaderboard claim also needs its own conditions. NVIDIA’s blog says GR00T 2 “ranks among the top three” on RoboCasa365, RoboLab, RoboDojo and MolmoSpaces. The whitepaper is more specific. Among vision-language-action and world-action-model entries on the public leaderboards, it reports the highest success rate on RoboCasa365 (67.2%) and RoboDojo (35.5%), third place on RoboLab (47.4%) and second on MolmoSpaces (45.1%), the last two after post-training only on real DROID data. Baseline results are taken from the official leaderboards as of 6 November. With reinforcement learning and agentic execution added, NVIDIA reports 88.0% on RoboCasa365 and 75.8% on RoboDojo, but the RoboDojo figure was evaluated on one of that benchmark’s three official layout banks.

Data: one release counts hours, the other does not

NVIDIA’s whitepaper states GR00T 2’s pretraining mixture: 402,730 hours of egocentric video of people at work, 10,184 hours of demonstrations with a handheld UMI gripper across 2,897 tasks, and 23,500 hours of robot teleoperation. The robot data spans parallel-gripper arms (YAM and Franka), dexterous hands (Sharpa and Wuji) and the Unitree G1 humanoid. The human video comes from an internal corpus NVIDIA calls Human-1M, which the whitepaper describes as about 1 million hours before filtering; the 402,730-hour inventory used for GR00T 2 spans more than 51,500 collectors and 123,000 scenes. Of the corpus, 100,000 hours are published as PhysicalAI-Human-1M-100K, which Hugging Face lists under CC BY 4.0.

NVIDIA also ran an ablation that supports its data story. It pretrained four models on none, 1%, 10% and 100% of the human video and evaluated each on 72 Dexter tasks. Task progress rose at every step, from 64% to 77% in video rollouts and from 50% to 70% on the real robot, and real-robot success rose from 18% to 41%. The real-robot figures rest on three rollouts per task. The whitepaper is candid about what this does not show: there is no known “exchange rate” between an hour of human video, an hour of UMI data and an hour of teleoperation.

DeepMind’s announcement does not state Gemini Robotics 2.5’s training data volumes or sources. It says Gemini 4 Argon acted “as a simulation engine,” generating thousands of varied simulated practice runs from a real camera scene and building digital twins, which combined with real demonstrations helped the robot generalize to new scenes. The October Gemini 4 Argon for Robotics post gives one concrete example: from a single photo of a tabletop, Argon built a MuJoCo simulation twin and generated more than 4,500 episodes, which Google says raised a real robot’s success rate on its evaluation from 56% to 75% and cut completion time from 35 to 24 seconds. That example concerns a bimanual Franka setup in Google’s post, and Google does not present it as a measurement of Gemini Robotics 2.5.

Access: what “open” and “partners” mean this week

DeepMind’s position is consistent with July, when its VLA models went to early-access partners while the reasoning model went to Google AI Studio. The Gemini Robotics ER 2 page lists public preview in Google AI Studio and the Gemini API, and private preview on Gemini Enterprise Agent Platform. The acknowledgements of the 2.5 post thank Agile, Apptronik and Mercedes-Benz.

NVIDIA’s position is a promise. Humanoids Daily reports that NVIDIA’s director of robotics, Jim Fan, said at CoRL that GR00T 2 and GR00T 2 Turbo weights will be released as open source in December, and that NVIDIA has not yet published licence terms. The Frontier Wire did not attend the talk. A search of Hugging Face’s public model index on 12 November found no GR00T 2 checkpoint under NVIDIA’s account; the newest GR00T entries there are GR00T N1.7 variants.

The predecessor gives a guide but not a guarantee. The Isaac-GR00T repository says GR00T N1.7 code is released under Apache 2.0 and its weights under the NVIDIA Open Model License, and calls N1.7 “fully commercially usable.” Whether GR00T 2 follows the same terms is not stated anywhere reviewed for this article. The Frontier Wire’s licensing map explains why the weight licence, not the word “open,” decides what a company can ship.

Safety tooling sits at different layers

Safety in robotics happens at several layers, and the two releases put their effort in different ones. The taxonomy below sorts what each company published.

DeepMind’s benchmark and its scope

ASIMOV-Jailbreak is the most reusable safety artifact in either release. The dataset card describes two suites: paired harmful and benign bimanual tabletop tasks, and a multi-turn adversarial suite of 845 episodes across 15 strategy groups, each up to 10 steps with simulated read-only inspection and calibration tools. Scoring combines deterministic checks on action tool calls with a language-model judge that reports an attack success rate, a benign over-refusal rate and a 41-criterion Physical Safety Rubric. Measuring over-refusal matters, because a model that refuses everything would otherwise look perfectly safe.

Three details limit what the benchmark proves today. First, the 99.5% jailbreak resistance in DeepMind’s post is for Gemini 4 Argon, against 82% for Gemini Robotics ER 2. The post gives no ASIMOV-Jailbreak score for Gemini Robotics 2.5. Second, the benchmark’s default judge is Gemini 4 Argon, and the post does not say which judge produced the 99.5%. A model graded by its own model family is a known weakness, which the harness partly addresses by allowing several judges at once. Third, access is gated: the repository is public but users must log in and accept conditions to get the files. Its licence is also inconsistent, with Hugging Face metadata listing CC BY 4.0 and the README stating the MIT License.

DeepMind’s reported steering results are harder to interpret. The post says inference-time reasoning guidance “boosted safe task completion around obstacles by 33% and around nearby people by 43%,” without saying whether those are percentage points or relative gains, or what the starting rates were.

The October Gemini 4 Argon safety report sets the boundary itself. It notes that safety behavior on one of its evaluations “is highly sensitive to the actual harness used,” and states that its agentic robotics evaluations “only cover high-level semantic safety reasoning and orchestration capabilities.” Validating certified hardware, redundancy and real-time guarantees, it says, “is necessary for safe physical deployments of robots.”

NVIDIA’s open problem and its separate stack

The GR00T 2 whitepaper lists “safe physical action” among its open problems: “Safety training of language models constrains what they say, and it is not known to carry over to what a robot does. A policy that agents can steer needs its own checks on the physical consequences of each action, at the rate at which actions execute.” The CoRL post announces no safety benchmark or safety result for GR00T 2.

NVIDIA’s safety work sits in a different product. A 21 September NVIDIA post describes Halos, which for robotics spans the IGX Thor module with a dedicated functional-safety island designed to support systems built to IEC 61508 and ISO 13849, Halos Core software, the Holoscan Sensor Bridge and an open-source Outside-In Safety Blueprint that uses external cameras. NVIDIA says TÜV Rheinland is inspecting IGX Thor, Halos OS and Holoscan Sensor Bridge for functional-safety certification readiness. That is a readiness inspection, not a certification, and the post does not connect Halos to GR00T 2.

Put side by side, DeepMind has published a reusable test for the language-and-planning layer, while NVIDIA has a hardware and systems safety path that sits beside the model. Neither has published evidence about the layer both companies’ own documents say is unsolved: the physical consequences of each action at control rate.

What remains unknown

Several facts would change this comparison, and none is public yet:

1. Gemini Robotics 2.5’s absolute success rates across all 200 tasks, its trial counts and the baseline behind the aggregate figures. A technical report with these would turn the 30% to 100% range into numbers a reader can check.
2. The unit and denominator of the 96% kitting figure, and how long the pilot ran. A per-tray rate over several hundred trays would be strong deployment evidence; a per-pick rate over a few dozen would not.
3. How many tasks were screened out of Dexter-50. If NVIDIA started from 52 candidates, the 95% figure is close to a general claim. If it started from 200, it is not.
4. GR00T 2’s weight licence and release date. A permissive licence in December would make GR00T 2 the only one of the two that outside researchers can test independently.
5. Any independent evaluation. Neither model has been tested by a third party. Trade coverage of both launches so far has repeated company figures.
6. ASIMOV-Jailbreak results for action models. The benchmark targets agents that call robot tools; a score for Gemini Robotics 2.5 itself, or for GR00T 2 under an agent, would show whether refusal at the planning layer survives at the action layer.

A proposed matched test

The following is a proposed protocol, not something either company or this publication has run. Once GR00T 2 weights are public and Gemini Robotics 2.5 is available to a partner willing to publish, a fair comparison would fix the robot (Franka hardware is the nearest overlap, since DeepMind names the Franka Duo and GR00T 2’s pretraining includes single-arm Franka data, though neither company has shown both models on the same setup), fix a task list chosen before any model is run, and fix the adaptation budget (for example, five teleoperated demonstrations per task and a stated amount of compute). It would run at least 50 trials per task per model in interleaved, blinded sessions, report per-task success with confidence intervals, and score both models on the harmful and benign suites of ASIMOV-Jailbreak with more than one judge. Without the shared robot and the fixed task list, any result would repeat the problem this article describes.

The verdict, by reader

For a robotics researcher deciding what to build on, GR00T 2 has the better-documented evidence and the more open route, conditional on a licence that has not been published and code that has not shipped. Its whitepaper discloses its own selection effects, trial counts and limits, which is what an auditor needs. The Frontier Wire’s guide to reading a model card like an auditor sets out the same checks applied here.

For a manufacturer evaluating humanoids, Gemini Robotics 2.5 has the only production-line evidence in either release, tied to a named robot and two named plants. That is meaningful, but it is a partner-only system with an undefined success unit, and its value can only be judged by asking DeepMind or Apptronik for the denominator.

For a safety team, ASIMOV-Jailbreak is usable now by anyone who accepts its terms, and both companies’ documents agree that no published evaluation yet covers action-level physical safety. Whoever publishes that evidence first will have the stronger safety case. On the evidence available on 12 November, neither has.