Credit: this is my take on So I’ve been testing Gemini 4…, a 38-minute video by Theo (t3.gg). Prices and benchmark numbers are checked against Google’s launch post, the Gemini API pricing page and Artificial Analysis. The code-audit numbers are Theo’s own tests, as he reported them. The opinions are mine.
The short version#
Google made Gemini 4 Argon available to everyone on October 13. It’s the first Gemini above the Flash tier in more than seven months, and on paper it’s a comeback. It ranks first on the Vals Index, ties GPT-6 Astra at 53 on the Artificial Analysis Intelligence Index, and has the lowest hallucination rate of any leading model on Artificial Analysis’s knowledge test.
Theo got early access and has been running it on his own codebases. His verdict: “one of the worst releases of the year.” In three of his codebase audits, most of what Argon claimed about the code was false.
Both of those can be true at once. If you’re about to wire Argon into an agent, you need to know why.
What the benchmark measures#
Artificial Analysis gives Argon a 15% hallucination rate on AA-Omniscience, against 51% for GPT-6 Astra and 54% for GPT-6.1 Sol. That sounds like a model that doesn’t make things up.
Read the definition, though. The hallucination rate is wrong answers divided by all non-correct answers, including “I don’t know.” A model that declines a lot scores well on it. Argon’s accuracy on the same test is 50%, 13 points below Astra’s 63%. Its combined score, 42, is level with Astra (43) and Sol (42). So the honest summary is: Argon knows less trivia than its rivals, and it guesses less when it doesn’t know.
That’s a test of factual recall. It says very little about what a model does after an hour inside your repo.
What Theo found in real code#
Theo’s method: have each model audit a real codebase, check every claim it makes, then have a panel of judge models score the results.
- His TypeScript-compiler-in-Rust project: Argon scored 4.38. That’s less than half the 8.84 for GPT-6.1 Sol, which made 38 claims and got none wrong. 24 of Argon’s 36 claims were false.
- An audit of his cloud product, Lakebed: Argon scored 2.25 out of 10. 56 of its 78 findings were wrong, about 72%.
- The T3 Code architecture: Argon scored 3.95. It made 31 true claims, 14 partly true and 47 false. Meta’s Muse Wonder Full, on the same task, had 29 true, 8 partly true and 4 false.
Sources: Artificial Analysis, Sep 30, 2026 (left). Theo (t3.gg), Oct 14, 2026, his own tests (right).
Look at that last one again. Argon made more false claims than Muse Wonder Full made true ones. In a fourth round, reviewing a real bug, Theo says it got 14 of its 40 claims wrong. Better, but still far worse than GPT-6.1 Sol, which got none wrong on that one.
Then he gave it a real bug: SQLite connection problems in T3 Code. Argon found the right mechanism, then fixed the wrong half of it. It took about 30 minutes, opened a pull request of roughly 1,200 lines, missed two existing PRs that already fixed the problem, ignored the repo’s PR-writing rules, and failed CI. Theo closed it. Per Theo, the fix that actually got merged was 21 lines.
He now plans to auto-close any pull request to T3 Code written with Argon.
Where it does shine#
To be fair to it, this isn’t a model with nothing going for it. It got the first perfect score on Theo’s private SkateBench, which tests naming skateboard tricks, a task that has stumped most models. Claude Opus 5.5 on max gets 79%. He thinks its front-end design work is good, though not as good as Claude Fable 5.1. Google’s own numbers are strong too: inside Google, Argon in Antigravity raised agentic code submissions by more than 35% and cut rollbacks by nearly a third. Theo isn’t impressed, since that’s measured against Google’s previous tools.
His most useful line is about the Google employee who described Argon to him not as a revolution, but as “I find myself reaching for Opus 5 less.” Opus 5, not 5.5.
The price will double#
Argon is $2 per million input tokens and $10 per million output tokens through November 30. From December 1 it’s $4 and $20. That’s exactly what Claude Opus 5.5 costs today (Anthropic pricing). Theo puts Opus at $5 and $25 in the video, but that’s the old Opus 5 rate.
The per-token price also flatters it. On Artificial Analysis’s tests, Argon used about 62,000 output tokens per task against about 27,000 for GPT-6 Astra. Its cost per task is $1.99 at the launch price and $3.98 after, about 1.2x Astra’s.
Caching is the other catch. Cached input is 95% off, but Argon only supports implicit caching, so you can’t tell it what to cache. Theo saw cache hit rates of 70 to 80% on Argon, where he usually sees 99%+ on other models. He admits that part might be a bug in Antigravity or in his own integration. If it isn’t, you pay full input price far more often than the sticker suggests.
What I’d do with this#
I build and host AI agents for small businesses, and an agent that confidently reports something false is worse than one that says nothing. It sends your team chasing problems that don’t exist. So here’s how I read this:
- Don’t let it review code or audit systems unchecked. If you try Argon there, run its findings through a second model before anyone acts on them. Theo says the same.
- Try it where it’s strong. Vision tasks, documents, first-draft landing pages, and anything that needs Google data. Theo says Gemini is still one of the best ways to work with YouTube transcripts.
- Budget at the December price. Anything you build on the $2/$10 launch rate will cost twice as much in under seven weeks.
- Run your own eval before switching. One model can top a leaderboard and still flunk your codebase. Twenty real tasks from your own workload will tell you more than any index.
Benchmarks tell you what a model can do. They don’t tell you how often it’ll be confidently wrong about your code. Only testing on your own work does that.
Source video: Theo (t3.gg), So I’ve been testing Gemini 4…