Gemini 3, Grok 4.1, & 'Water Lies'

TLDR;

Welcome back! This week, you probably know what we’re going to talk about: Gemini 3 Pro came out and stole the show. But we’ll also talk about another big release, Grok 4.1, the lies being told about data center water consumption, and several other product and market news items of interest.

OpenAI releasedz GPT-5.1 Codex Max just as I was sending this newsletter, so we’ll cover that on Friday’s post.

Enjoy!

FRONTIER

Gemini 3.0 Pro Released. It’s Impressive

Based on preliminary results (you can’t jump to conclusions with benchmarks), we’ve got ourselves what’s probably the most considerable step-function improvement we’ve had in years, thanks to Google’s Gemini 3.0.

One of the most hyped model releases, probably ever, many are swearing by this model, arguing that it feels like a significant improvement and the unequivocal SOTA (state-of-the-art) across most areas.

Out of all the benchmarks above, I want to highlight four:

Humanity’s Last Exam: The hardest (by far) knowledge-based benchmark on the planet. They offer a considerable improvement, even relative to GPT-5.1, which got released last week.

It must be said that Grok 4 Heavy, an amalgam of Grok 4 models in a multi-agent setting, scores higher, at 50.7%, but it’s not a fair comparison.

ARC-AGI 2: Probably the hardest benchmark of them all, which tests models in puzzle-like tasks with little room for memorization and that require data-efficient adaptation (finding the pattern to solve an IQ-test-like pattern with just three examples). As we saw on Sunday, neural networks struggle with sparse patterns, so almost doubling the second-best model in GPT-5.1 is beyond impressive. Interestingly, it’s also a performance-per-dollar improvement, as you can see in the graph below, with the exact cost compared to models like the Claude 4.5 Sonnet while delivering more than double the performance.

Tau bench: Probably the most relevant benchmark to evaluate agentic model capabilities, the benchmark has been led by Anthropic’s models for quite some time, giving the throne back to Google.

MathArena Apex: But if there’s one benchmark result that will go viral, it’s MathArena Apex, the hardest match benchmark there is. Here, they score 23%… with 1.6% being the previous SOTA by Claude 4.5 Sonnet. This means Gemini 3.0 is likely the most powerful model in terms of knowledge, intelligence, and agentic capabilities, making it the best model overall (again, based on benchmarks only).

ARC-AGI 2 Leaderboard

On the flip side, they didn’t seem to manage to beat Claude 4.5 Sonnet on coding benchmarks, which opens the question to who’s really ahead in that area.

MathArena Apex example question. No model on the planet has got this one correct, including Gemini 3 Pro. Source

As for its price, it’s in line with what other Labs offer, but on the higher end (it’s more expensive than GPT-5.1). They offer $2 and $12 per million input/output tokens, but those numbers grow to $4 and $18 for contexts longer than 200k tokens (~150k words).

The fact that they are publishing a price higher than GPT-5.1, marking…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.