What Has Really Changed in AI?

Other Premium Content

In response to a Premium subscriber question, can DeepSeek’s results be replicated?

The piece below is complemented by the following deep-dive Notion articles: {🥸 Mixture-of-Experts}, {🥸 Soft vs Hard distillations}, {🖋️ Distillation vs Fine-tuning}, {🍏 Why R1 is not as bright as o1}, {💸 Prompt Caching}, {📏 The UAT}.

FUTURE

Have Things Really Changed in AI?

Based on recent events, one might feel confused about the direction AI is really heading toward. Is that one of less spending? Is that one of stagnation and increased geopolitical tensions? Are Chinese models really that much cheaper?

Unsurprisingly, this chaotic soup of new narratives and opinion changes has led to a cascade of horrible takes by “experts” and “AI investors” alike that, quite frankly, are throwing lies at you from all directions. Let’s solve this.

Today, we’ll do the following:

Look at the four fallacies that you have been led to believe but are dumbfounding wrong. No, AI spending will not fall. We will look at the largest companies' training budgets, how they will develop, and what that tells us about the future of Hyperscaler AI spending.

No, model sizes won’t fall either. We will understand DeepSeek’s key contributions to distillation, and what that tells us about the roadmap of frontier AI labs like OpenAI,

No, Chinese models aren’t cheaper. We will uncover ‘the great lie’ that has been diffused by media these past few weeks, and who the real champion of cost efficiency is (you will be surprised),

And no, we haven’t figured out the ‘I’ in ‘AI. ’ We will explain why even top experts are misinterpreting how to evaluate model intelligence and how the hype surrounding DeepSeek’s GRPO algorithm and OpenAI/Google’s Deep Research products is completely overblown, analyzing whether this is the end of McKinsey-type consultant firms and what all this tells us about the timelines for AGI.

Face the three hard truths underpinning AI’s progress that explain the ONE thing that has actually changed recently.

Finally, based on the narrative change, I will give you my hot takes on what will change in 2025.

No, These Four Things Have Not Changed.

Despite all the contradictory claims, if you look carefully, some patterns repeat every single time, whether they come from the US, China, or Mars.

No, AI Spending Will Not Fall

While the paradigm shift in narrative appears to be clear, in that inference-time compute is the new intelligence scaler, pre-training, training a huge Large Language Model (LLM) on a dataset encompassing the entire Internet and some private data, will remain essential despite the stagnation claims.

DeepSeek’s results have been profoundly misinterpreted in this regard, and this is obvious simply by looking at the numbers:

Pre-DeepSeek’s launch of V3, Llama 3.1 405B, the best open-source model at the time, required 15 trillion training tokens (~12 trillion words) and a training run of 3.8×1025 FLOPs (38 trillion trillion total operations).

GPT-4, which continues to be state-of-the-art two years later (GPT-5 is still in training or to be…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.