Sonnet 4.6, A $200 Billion Bet, & DeepSeek v4?

TLDR;

Welcome back! This past week, we’ve had more than six high-quality model releases. Yes, six, the latest Sonnet 4.6 just minutes ago, and the potential release of DeepSeek v4 anytime this week.

We’ll also cover Micron’s $200 billion bet that should clarify doubts on future DRAM demand, a new perspective on job losses caused by AI, Anthropic vs US Gov drama, and other worth-reading news.

Enjoy!

THEWHITEBOX

DeepSeek v4, At Any Point Now

After almost an entire year without a meaningful upgrade, DeepSeek, once considered (and to most, still is) China’s biggest hope in the AI race, is about to drop DeepSeek v4 any day of this week.

The hype is so big that markets are already bracing, with legendary investor Michael Burry echoing this blog that basically leaks what this model looks like.

That said, my bullshitometer is off the charts on this blog: it has a huge mistake in the first paragraph and many others later. For instance, it claims the model can run on two RTX 4090s, despite being WAY beyond what those GPUs can handle.

I believe the writer (i.e., an AI, since the blog is written using AI, like everything these days) confused activated parameters with DRAM parameters, when in fact the entire model, which will require 500 GB or more, to say the least, has to reside in RAM.

At this point, my hopes that this was written by someone who understands AI are basically zero. That said, the model seems to be really coming anytime this week, so the leak could still be real.

Also, my takeaway has nothing to do with the model itself. What matters is not what it is, but what markets will think it is.

TheWhiteBox’s takeaway:

The releases from DeepSeek aren’t just about their technological value; they transcend tech blogs and adoption memos and impact markets directly, a market that is nervous, overreactive, and scared to death of a potential realization that we are overspending.

Let’s not forget why markets crashed in March. It came to investors' attention that DeepSeek had trained a model with just $5.6 million, 20 times less than OpenAI had invested in training GPT-4, and many times more than the US models at the time.

This was a complete misunderstanding of AI costs, but it was “enough” to make NVIDIA have one of its worst days ever.

At the end of the day, not only was the number wrongly interpreted (DeepSeek never lied; they just simply gave us a half-baked number that didn’t include ablations, experiments, or post-training), but it was enough for laymen investors to jump to the conclusion that AI can be done much cheaper (which it can be, but the point is that the demand is so overwhelming that compute will still remain supply constrained).

All things considered, I have plenty of reasons to believe we could see another similar crash. People still don’t understand how AI costs work and still wholeheartedly believe China is more frugal. Yes, Chinese Labs…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.