China's Mad Robot, Google Crushes, & More

TLDR;

On this week’s Premium insights, we discuss Google’s firm dominance, robotics and hardware news from China, including a viral ‘robot doom’ video, Anthropic’s concerning behavior and business trends, and OpenAI’s new structure.

Enjoy!

FRONTIER EVALS

Google’s Leadership, Overwhelming

A popular AI insider has painstakingly consolidated the 28 most important benchmarks and aggregated each model's results by the average.

The results show Gemini 2.5 Pro, not OpenAI’s o3 model, as the best overall model.

But perhaps even more impressive are the results of Gemini 2.5 Flash Thinking, Google’s lightweight version. This model nearly matches the performance of Anthropic’s flagship model while also beating other flagship models like the Grok family.

Regarding open-source models (or, dare I say, open-weights), the first model on the list is DeepSeek’s R1, with Alibaba’s flagship model, Qwen 235B, being far behind. Fascinatingly, Meta’s new models, the Llama 4 family, are nowhere to be seen, showcasing how behind this lab has become in the LLM wars.

TheWhiteBox’s takeaway:

I am not surprised at all, Google’s Gemini 2.5 Pro model is the best model I’ve worked with. o3 has it factor in very challenging yet rare cases, but I find it extremely hard to talk to. In fact, I prefer interacting with GPT-4o, and that’s saying something.

And if we factor in prices, well, Google’s position becomes even more dominant, to the point of insulting.

FRONTIER MODELS

Google’s Leadership, Even More Overwhelming

And just as I finished writing the previous story, Google $GOOG ( ▲ 0.47% ) released a newer version of its flagship model, Gemini 2.5 Pro 05 06, which offers striking improvements in code and UI generation (generating beautiful application interfaces).

According to first impressions, it’s a considerable leap in performance, setting an impressive +140 elo jump in the LMArena coding benchmark (they are also leading the main leaderboard too).

The model is #1 in every single benchmark in LMArena. In every single one.

TheWhiteBox’s takeaway:

The best and the cheapest models simultaneously. We are really living in a Google world at this point.

And the funny thing is that we could have easily predicted this the moment leading AI was about Reinforcement Learning, as no lab on the planet knows more about this training method than these guys (AlphaGo, AlphaZero, AlphaProof, etc.).

Importantly, this is just weeks after OpenAI released o3, which already feels outdated in many aspects. When o3 dropped, I mentioned that Google was in no rush to release better models because the previous version of Gemini 2.5 Pro was already as good as o3.

And I was right. With this newer version, the superiority is just insulting. But why am I so confident? Well, it's because this is an iteration of a model we already know is state-of-the-art.

That said, to prevent looking like a complete Google hyper, this improvement we’ve seen with coding on this model doesn’t come without trade-offs. Although it’s still very early to jump to conclusions, it seems that this newer model performs worse in non-coding tasks…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.