[Premium] Fake News, Cerebras' World Record, & More

TLDR;

Technology:

🤔 GPT-5 & Gemini 2 in December?

😱 Cerebras’ Insane World Record

Markets:

🍎 The Weird State of Apple

🚗 Waymo New Monstrous Round

Product:

🐉 The War Between Salesforce and Microsoft Continues

🤯 Ideogram Canvas Looks Amazing

FRONTIER AI

GPT-5 & Gemini 2 in December?

This week, The Verge media published two insider reports that claim that OpenAI and Google will release their next frontier models in December. In OpenAI’s case, the model, which we talked about last month, is codenamed Orion and, according to an OpenAI executive (allegedly), is a 100x improvement compared to GPT-4.

As for Google, The Verge outlines similar plans, but they claim that the preliminary results offered by Gemini 2 aren’t as good as expected, and that this trend is also occurring in other labs.

TheWhiteBox takeaway:

Hold your horses one second because Sam Altman himself was quick to come out and discredit this news, responding to the article’s author directly via X.

fake news out of control — Sam Altman (@sama) 3:55 AM • Oct 25, 2024

So don’t get your hopes up that soon. But, to me, the most relevant insight came from the second article, regarding the allegedly underwhelming improvements of Gemini 2. If you’re a reader of this newsletter, you probably aren’t surprised by this, considering that we know that LLM’s capabilities are largely overstated, especially in terms of intelligence, which is presumably the main metric they are measuring.

But even if the zealous faith on scale turns out badly, LLMs still have a lot of value to offer… if they get more robust. I don’t think enterprises really need human-level AI today; what they are really asking for is a reliable productivity enhancement, one that can be deployed without keeping you awake at night.

AI labs should focus on getting their shit together at the current intelligence level before blindlessly aiming for higher intelligence models.

HARDWARE

Cerebras’ Insane Token Throughput

Cerebras, a US company that plans to become public soon, has broken a new world record for inference token throughput, reaching the astronomical value of 2,100 tokens per second on Llama 3.1 70B, or around 1,600 words per second for text.

This value puts other inference providers completely to shame, especially those that are GPU-based (the long tail to the right) and with inference-purpose hardware like Groq’s LPUs or Sambanova RDUs trailing behind, although more than an order of magnitude behind Cerebras’ performance.

Cerebras’ superior performance is so insulting that it generates tokens faster from Llama 3.1 70B than what other providers are getting from running Llama 3.2 3B. This is no joke, considering that the compute required to perform predictions on the former is exceptionally larger than on the latter, indicating that Cerebras’ inference performance is simply out of this planet.

Using Kipply’s blog post on inference Floating Point Operations (FLOPs) formula {FLOPs = 2 * nº parameters}, we know that the former requires 23 times more FLOPs per prediction than the latter. Yet, Cerebras’ throughput…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.