Grok 4 & Big Beautiful Research, & More

In partnership with

THEWHITEBOX

TLDR;

This week, we discuss Grok 4’s massive announcement, a super cool robotics launch by HuggingFace, and other news including what I consider one of the most beautiful research papers published in a while, one that shows with striking clarity how promising this new type of training we are doing with reasoning models actually is as a way to build real machine intelligence.

Enjoy!

FRONTIER MODELS

xAI Shocks the World with Grok 4

On Monday, I said that OpenAI’s deep research models were the best agents on the planet. Well, while I was correct, my correctness lasted two days.

In this fast-paced industry, we now have a new king of AI: Grok 4. xAI has released two versions of this model:

Grok 4: The standard reasoning version

Grok 4 Heavy: A multi-agent system (multiple Grok 4s together)

With this release, xAI no longer offers non-reasoning models (traditional LLMs) which seems the most possible outcome for most AI Labs due to the growing importance of post-RL training (more on this in our trend of the week).

Both models are available via the SuperGrok subscription, but you’ll have to pay $300/month to access the Heavy version. In the case of the API, Grok 4 is also available there.

In a nutshell, the model, released in the late hours of July 9th in an hour-long live stream, is staggeringly powerful, the most significant jump in quality over competitors since OpenAI announced o3 back in December.

I could go benchmark by benchmark, but let me simplify it: the model is first everywhere.

It’s first in knowledge-based benchmarks like GPQA, first in math (100% on the competition-level AIME benchmark and leading in the USAMO), and everywhere else, basically.

But if we have to highlight particularly impressive results, we have:

A 44.4% in Humanity’s Last Exam, the hardest knowledge-based benchmark out there where humans score well below most AIs these days, more than doubling OpenAI’s o3 result (although using a multi-agent system, w/o the multi-agent setting, the score is slightly lower, 38%, but still humiliates the rest).

A 73% in Artificial Analysis’ Intelligence Index, slightly ahead of OpenAI’s o3-pro, and that’s without testing Grok 4 Heavy.

A record-setting 15.9% in ARC-AGI 2, probably the hardest non-knowledge-based benchmark out there, doubling the previous best result (Claude 4 Opus). As described by the President of ArcPrize (the company behind the ARC-AGI benchmarks), Grok 4 is the first model that “breaks through the noise barrier”, implying that it’s the first model with results that can’t be attributed to ‘sheer luck’, and that it’s showing “non-zero levels of intelligence” quite the remarks from a company notoriously skeptical of the usual intelligence claims in AI.

75% on SWE Bench, one of the primary coding benchmarks, again defeating the rest of the competitors by quite a margin.

If there’s a relevant benchmark where we don’t yet know if Grok 4 leads is the Aider Polyglot, the primary coding benchmark.

Other points worth considering are the following:

The model…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.