Grok-3, Finally Here, OpenAI Back to the Open, & More

TLDR;

Today, you will learn the main takeaways from Elon Musk’s xAI's release of Grok-3, the launch of a Chinese rival to Cursor, and OpenAI’s possible return to open-source.

Moving on, we will look at two striking new valuations, above $30 billion, for companies will close to no revenue (let alone a product in one case), evidencing the power that being an ‘AI company’ still holds on investors.

Finally, we will see evidence that AIs aren’t as smart as they seem, learn how humans use AIs to understand animal facial expressions, and examine the first attempt to consolidate non-reasoning and reasoning models into one single model.

FRONTIER AI

Grok-3 Finally Launches

On Monday night, xAI released Grok-3, hailed by Elon Musk as "the smartest AI on Earth." Here are the takeaways in terms of model, product, and the Colossus cluster: Four models were announced: two non-reasoning models (Grok-3, Grok-3 mini), two reasoning models (mainly aimed for STEM tasks), Grok-3 Reasoning, and Grok-3 mini Reasoning. Reasoning models are on beta, so the results are based on early checkpoints (meaning they should improve). The non-reasoning models (models that don’t generate chains of thought to answer and, thus, think for much less time) display state-of-the-art performance (images below) on hard benchmarks.

On vibe evaluations like LMSYS (score based on relative, user-vibe model-to-model comparison), an earlier version of this model, codenamed Chocolate and silently published two weeks ago, also takes first place, despite being an earlier checkpoint.

As for the reasoning models (again, recall that they haven’t fully finished training yet), Andrej Karpathy puts them at o1-pro level. It's important to note it's $40, 5 times less than OpenAI's price.

Sadly, the full chain of thought (the reasoning process) is partially obscured (although not as much as OpenAI's), and reasoning is activated via the 'Thinking' and 'Big Brain' buttons. To the question whether these buttons activate the use of another model of it’s Grok-3 with dynamically-added compute is unclear, but it’s most probably just two different models (non-reasoning and reasoning get called depending on whether the thinking button is activated, probably the signal xAI uses to know which one to engage).

Interestingly, sometimes, Grok-3 mini outsmarts Grok-3 by thinking on the problem for longer. However, the latter generalizes better (more resistant to prompt variations, aka less overfitted to the benchmarks, aka smarter).

With this, we have confirmation that intelligence scales in two ways: larger model sizes or longer thinking time. However, smaller models are easier to serve, clarifying the product strategy:

Pretrain large model

Add reasoning capabilities

Distill knowledge into the smaller model (DeepSeek proved that this outperformed training the smaller models from scratch as they fail to learn patterns larger models do learn).

Serve smaller model to run for longer to 'recover' the knowledge gap with larger model.

In plain English, the model you get is not the smartest these labs have internally in raw performance terms, but it can be run for longer to close the intelligence gap.

Furthermore, it’s also unclear…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.