AI's Court Win, 11A, Great Small Models, & More

TLDR;

Welcome back! Today, we explore amazing research and models emerging from Japan and China, Google’s new robotics model, and a planetary-scale inference stack.

We then move to discuss Thinking Machines’ strategy, Anthropic’s court win, and new products from ElevenLabs or Google, among other news.

Enjoy!

SMALL MODELS

Polaris, Better than Claude 4 Opus?

A group of Chinese researchers has obtained a pretty remarkable result that shows how far we have yet to go with small models: a four-billion-parameter model, which can be run in many consumer-end hardware like a MacBook, that obtains better results than Claude 4 Opus in the AIME 2024 and 2025 benchmarks, a benchmark to test maths skills.

Departing from a mid-to-low tier Qwen3-4B model, their post-training recipe boosts its performance to 79.4%, surpassing not only Opus but also Grok 3.

TheWhiteBox’s takeaway:

I’m confident that we will see significant progress in small-model land during the second half of the year.

However, seeing results like this one wasn’t in my book; we're talking about a model I can run on my computer yielding better results than one of the top-3 models on the entire planet.

With all said, we can also add some healthy skepticism. Not because we don’t trust the results, but because of the sacrifice that those results meant.

What I mean is that in AI, there’s always a trade-off. In particular, we might be dealing with two issues:

Balancing the bias/variance trade-off

Balancing depth and generality

Both relate to the concept of overfitting, a well-known issue in AI, where the model adjusts to the noise in the data rather than just the signal.

But what is this?

Imagine you are training a model to recognize whether an image contains a cat or not. You provide the model with five training images of cats, all of which show the cats sitting on a red blanket.

The model might “learn” that red blankets also mean ‘cat’ because it over-memorized the specific patterns of the training data instead of learning what actually makes a cat (like fur, ears, whiskers).

So, when you show it a picture of a cat sitting on a blue couch, it might fail because it overfitted to the detail that wasn’t essential: the red blanket.

In that case, our model has adjusted not only to identifying the cat itself, but also to the fact that the blanket was red, which is, naturally, irrelevant.

What I’m trying to get with this is that it’s very likely that this model has inferior generalization capabilities compared to Claude 4 Opus in that it might have overfitted to the AIME data, probably memorizing a lot of the patterns required to solve the problems, and that the superiority over Opus might disappear in other benchmarks.

But even if the model isn’t overfitted to AIME, another point to mention is depth, meaning that this model might have achieved excellent performance on maths, but at the expense of other areas of interest.

This is what I always describe as…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.