The Bubble Grows, SEALs, & China

TLDR;

Welcome back! Much of today’s news revolves around a scary idea: we are terribly deep into an AI bubble, and you’ll see why below. Additionally, we include mentions of China’s remarkably rapid progress, and a notable AI expert makes a case that most AI startups will go to zero.

Beyond that, there’s also room for hope in progress, thanks to insightful research by MIT on Self-Adapting Language Models.

Enjoy!

FRONTIER AI

Gemini 2.5 Pro’s Strong ARC-AGI Results

circles are results in ARCAGI1, triangles ARCAGI2

The ARC team behind super popular benchmarks like ARC-AGI, specifically designed to be memorization-resistant, has published Google’s Gemini 2.5 Pro results.

And they are quite good. But why is this particular benchmark so relevant?

As we have commented several times, most of the benchmarks we test frontier AI models on are susceptible to being memorized, to be cheated, in plain words.

Because how do we discern whether the model is reasoning a result or simply vomiting it?

For that, benchmarks like ARC-AGI have emerged to serve as a way to make that separation; if you solve this benchmark, it’s more or less guaranteed that memorization has not played a crucial part, opening the door to discussing real chances the model is ‘intelligent.’

In thi case, the model reaches 41% in the ARC-AGI 1 and 4.9% in ARC-AGI 2. But those numbers look really low, right?

Why am I saying ‘great results’?

Simple, because I’m resistant to this industry’s gaslighting, and a 41% in this benchmark is leaps and bounds more impressive than a 70% on popular benchmarks like MMMU-Pro, a contaminated benchmark (most models have seen the answers) that can be memorized.

But this is also a reminder of how unintelligent these models are. They are incredible at imitating intelligence, but when they have to elicit real intelligence, well, you get a 4.9% score.

Hence, currently, AI is more useful than it is intelligent. And even the former is exaggerated.

If you’re here, I assume you believe AI will change the world. And it will, but that doesn’t mean we can’t be honest about the current state of the industry.

CODING

OpenAI’s Codex is a PR machine

As shown in this tweet, OpenAI’s autonomous background software agent, Codex, is merging an overwhelming number of pull requests (PRs), demonstrating the product's impact across the software industry.

Pull requests are code edits to a repository, meaning that the AI is actively making a huge amount of changes to coding repositories. The fact that they are being merged means that someone has validated the code, a sign that the model not only identifies issues but also solves them.

TheWhiteBox’s takeaway:

In layman’s terms, the era of autonomous software engineers is well underway. While AI’s impact in other areas is still open for debate, it’s hard to disagree wth the fact that AI is, indeed, transforming the software industry forever.

However, we must be cautious with such claims, as, at least in the case of this metric, one can…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.