[Premium] ChatGPT's Court Win, LLM Plateaus, An End to NVIDIA's Moat? & More

TLDR;

Technology:

🚨 Are LLMs Plateauing?

😍 This New Chinese Model is Amazing

Markets:

🇺🇸 Trump Loves Energy, Just Not All Sources

🫢 TSMC to Stop Producing Chips for Chinese Customers

🔐 Crypto Company Wants to Train Largest Open-Source Model Ever

Product:

✋🏼 An End to NVIDIA’s Moat?

👩🏽‍⚖️ ChatGPT Scores a Big Court Win

LARGE LANGUAGE MODELS

Are LLMs Plateauing?

A piece by The Information has sent shockwaves through the industry. This article argues that, according to its sources, the new generation of frontier AI models, after finishing training, has shown a less-than-expected improvement over the current generation led by GPT-4o, Gemini 1.5 Pro, and Claude 3.5 Sonnet.

In other words, frontier AI models are plateauing. Apparently, it’s better at language tasks but ‘not reliably better’ than the previous generation in other crucial tasks like coding. However, some other sources claim we are not plateauing but transitioning into a new scaling paradigm: test-time scaling.

As we’ve discussed several times, the main goal is to increase the training scale. That is, increase both the model size and the training budget (thereby increasing the data set size, too), as scaling has undeniably led to better models up until now.

Importantly, the magnitude of those improvements between generations has been mind-blowing, just like the fantastic performance increase we saw from GPT-2 to GPT-3, and especially from GPT-3 to GPT-4.

And the issue is that this next transition (call it GPT-4 to GPT-5) has been, if the report is true, underwhelming, and that OpenAI is desperately trying to find new ways to achieve these step-function improvements now that scaling doesn’t cut it. And it isn’t just The Information reporters saying this; we have two billionaire venture capitalists (a16z founders) who have acknowledged investments in companies like OpenAI and Mistral, who echoed the same fear.

TheWhiteBox’s takeaway:

If true, this is terrible news for the industry. Not because LLMs will be proven pointless or anything like that, as they already provide some value and have tremendous room for improvement from a deployment perspective. Instead, it would be a huge shock to the industry’s narrative.

The valuations of public and private AI companies echo the vision that we have finally cracked the AI code and that it’s a matter of time till AGI comes. Nonetheless, Sam Altman went on Y Combinator just a few days ago to claim they finally knew how to reach AGI (if you ask me, there’s no way he actually believes that unless he’s high in his own delusion).

This extremely hyped view is the only reason these companies see capital investments in the multiple billions; it’s not what AI is today but the promise of what it can be… and the fact we allegedly know the path to that future.

In fact, as we’ve discussed plenty of times, if we look at AI from a purely economic value perspective, it’s not even close to meeting the expectations markets reflect, with revenues 12-20 times below capital expenditures (and the gap is…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.