The Gemini 4 Argon ARC-AGI-3 result is really two results. It’s the same model and the same test, yet the two scores sit nearly 48 points apart. Only one of them made most of the headlines.
ARC Prize verified Google’s model this week. However, its results page is dated October 13, and the public announcement landed on October 15. With what ARC Prize calls the provider adapter harness, Argon scored 97.5% on ARC-AGI-3. On the standard harness, though, it scored 49.8%. It also hit 91.3% on the older ARC-AGI-2.
So which number counts? Both, honestly. They just answer different questions. And neither one says “AGI.”
What the Gemini 4 Argon ARC-AGI-3 result actually says
Here’s the verified picture from ARC Prize’s Gemini 4 Argon results page:
- Provider adapter harness: 97.5% at low reasoning, 97.3% at medium and high.
- Standard harness: 49.8% at low reasoning and 46.3% at medium. ARC Prize didn’t publish a high-reasoning score.
Notice something? However, the adapter score barely moves when you turn the reasoning dial. The standard score does.
Neither run was cheap. OfficeChai reports a cost of about $4.8K for the adapter run and about $5.8K for the standard run. Also, the same outlet notes that ARC-AGI-3 scores sat in the single digits just months ago.
The result makes Google the second lab, after OpenAI, to cross 95% on this benchmark. However, only GPT-6 Astra sits higher. That’s a fast climb. Still, a fast climb on one test tells you about that test, and little else yet.
Why one model got two scores
A harness is the plumbing between the model and the test. It decides what the model sees, what it remembers between turns and how it sends moves back.
ARC Prize runs two of them:
- Standard harness. A minimal, provider-neutral setup. The model carries forward only the notes it chooses to keep.
- Provider adapter harness. This one keeps the model’s hidden reasoning state between requests and compacts long conversations. So the model can reuse earlier work instead of starting cold.
Think of an exam where one student keeps their scratch paper between questions. Meanwhile, the other has to rewrite their notes from memory every time. Same brain, different desk.
ARC Prize frames this as “two harnesses, two questions.” For example, the standard run gives an apples-to-apples comparison across labs. The adapter run asks how well a model does with the tooling its own maker built for it. So both Gemini 4 Argon ARC-AGI-3 numbers are legitimate. Still, they simply measure different things.
This fight has history, too. As The Decoder recounts, back in July GPT-5.6 Sol scored 7.8% in the standard environment. However, through OpenAI’s API, with retained reasoning and compaction, it scored 38.3%. OpenAI argued that benchmarks always test the setup too. In the end, Chollet then allowed shared, publicly available API settings.
The 49.8% has its own asterisk
Plenty of recaps treat 49.8% as the “real” score. It’s messier than that.
ARC Prize says API issues stopped it from completing the standard evaluation. Runs finished on 42 of 55 tasks at low reasoning, 46 at medium and just 19 at high. The reported scores still cover all 55 tasks, with partial credit for unfinished runs.
So 49.8% is a low-reasoning, incomplete result. Medium reasoning scoring below low looks strange. ARC Prize doesn’t explain it, though, and we won’t guess beyond “incomplete runs.”
What ARC-AGI-3 measures, and what it doesn’t
ARC-AGI-3 works more like a game console than an exam paper. It drops an agent into small, abstract, turn-based games with no instructions. The agent has to explore, work out the goal, build a model of the rules and then plan its moves.
ARC Prize says humans can solve every environment. In fact, its human baseline came from about 500 members of the public, chosen without any puzzle-skill screening. For each level, the baseline is the median number of actions players needed.
Here’s the smart part. ARC-AGI-3 scores action efficiency, meaning how much experience a model needs to crack a game. However, most benchmarks only track compute cost. This ties back to ARC Prize’s own definition of AGI: acquiring any skill a human can, as efficiently as a human can.
On that front, Argon looked strong. According to The Decoder’s reading of ARC Prize’s chart, it used 62% fewer actions than the human baseline across the 25 public games. It also matched or beat the speedrun record on three of them.
Gemini 4 Argon vs GPT-6 Astra on ARC-AGI-3
Argon arrives a few weeks after OpenAI’s GPT-6 Astra, and the pattern repeats:
| Model | Provider adapter | Standard harness |
|---|---|---|
| GPT-6 Astra | 99.9% (high reasoning) | 62.7% (max reasoning, complete run) |
| Gemini 4 Argon | 97.5% (low reasoning) | 49.8% (low reasoning, incomplete run) |
The adapter gap shows up for both models. So the harness effect looks like something that happens across labs, and Google just happens to be the latest example.
But don’t read the standard column as a clean head-to-head. Astra’s 62.7% came at max reasoning on a finished run. Argon’s came at low reasoning on an unfinished one.
François Chollet, who created ARC, reportedly posted on X that Argon has “a similar performance profile on ARC-AGI-3 as Astra” and that “Google is back at the frontier.” Broader rankings back the near-tie. On Epoch’s Capabilities Index, Argon scores 165, Astra 166 and Opus 5.5 167. That’s a photo finish.
Argon isn’t ahead everywhere, either. Epoch’s early estimates place it eighth on math and sixth on software engineering. For how the big labs have traded places this year, see our 2026 frontier model comparison.
Does a 97.5% Gemini 4 Argon ARC-AGI-3 score mean AGI?
No. And the people who built the benchmark say so.
When Astra hit 99.9%, ARC Prize said it was “not claiming that it is AGI.” It added that saturating ARC-AGI-3 wouldn’t be “proof of achieving AGI,” though it does show real progress toward generalization. The same logic applies to Argon.
Four reasons the score falls short:
- The worlds are closed. OfficeChai notes that ARC Prize describes these as closed, deterministic puzzle worlds. Real life is open-ended and noisy.
- The strict bar is the standard harness. ARC Prize says a future AGI should solve ARC-AGI-3 under those neutral conditions. Argon’s best published standard score sits under 50%.
- Tools muddy the human comparison. ARC Prize points out that some harness results reflect a model and its tools combined. Its human testers had no code interpreter or scratch pad.
- One domain isn’t general. Middling math and coding rankings suggest Argon’s ARC strength doesn’t carry across the board.
Does anyone even agree on what AGI is? Not really, as our piece on what AGI means shows. A benchmark measures one narrow skill under set conditions. That’s useful, but it settles far less than the headlines suggest.
What would change our mind
Here’s what we’d want to see before taking the AGI talk more seriously:
- A complete standard-harness run, including high reasoning, where only 19 of 55 tasks finished.
- Independent replication of the adapter result.
- Transfer outside Google’s stack. Dataist asks exactly this: how much of Argon’s lead survives without provider-specific reasoning and context features?
- Strong results on open-ended, non-deterministic tasks, beyond tidy puzzle games.
ARC Prize also publishes per-game replays and scorecards, so you can inspect the failures yourself. Under the standard harness at low reasoning, some public games scored near zero. G50T landed at 0.0% and BP35 at 1.0%, while several others hit 100%. Bear in mind that the headline figures cover the 55-task semi-private set, which is larger than those 25 public games.
Why Argon’s score still matters
Strip away the AGI talk and something real remains. Google has a model that plays unfamiliar games about as well as OpenAI’s best. It also needs fewer moves than a typical human to do it. For anyone building agents, that’s genuine news.
But the honest headline reads more like this: “Argon scores 97.5% when its own memory tricks are allowed, and under 50% on an unfinished neutral run.” Next time a benchmark number goes viral, ask which harness produced it. Then ask whether the run actually finished.
The bigger question is whether agents that plan across many steps already count as something close to general intelligence. We dig into that in Are Long-Horizon AI Agents Functionally AGI in 2026? So read it with the Gemini 4 Argon ARC-AGI-3 numbers in mind. You might find the answer shifts depending on which score you believe.
