It was inevitable.

In partnership with

THEWHITEBOX

TLDR;

Technology:

🤩 2024, What a Year

🥳 Cerebras’ Trillion-Parameter Model

Markets:

🧐 AI As A Tool for Phishing

🤓 OpenAI’s AGI Definition

Product:

😍 Okta’s Auth for Agents Platform

🫢 The First AI Movie

TREND OF THE WEEK: The Inevitability of Latent Reasoning Models

RECAP

2024, What a Year

The Information has released a series of graphs (I’m showing two, but the article has more), providing a very visual view of AI in 2024. First, we saw the massive AI spending sprees of the four largest Hyperscalers (Microsoft, Amazon, Google, and Meta), accounting for over $200 billion, a CAPEX larger than many European countries’ annual GDP.

This trend will only accelerate in 2025 with large reasoner models (LRMs) like o3, as these models are much more compute-intensive when run.

On the other hand, we witnessed a decrease in token price (how we measure the cost of running Generative AI models), with examples like the GPT family, which has decreased its price by 85% in less than two years.

If we regard token prices as the ‘cost of intelligence’ (‘intelligence is doing a lot of work here, but bear with me), the several orders of magnitude decrease in costs is essential to making these tools widely accessible.

That said, I’m skeptical about how much these costs can be decreased if computing costs increase exponentially unless energy costs drop to basically zero.

For that reason, nuclear energy, as well as wind and solar, will play a crucial role in AI’s short-term future. Expect many more PPA (power purchase agreements) between Big Tech companies and the utility sector to access power plants, as Small Modular Reactors, which will fuel most future data centers, aren’t coming live before 2030 at the very least.

HEALTHCARE

Cerebras’ Trillion-Parameter Model

Cerebras Systems, in collaboration with Sandia National Laboratories, has achieved a significant milestone by training a 1-trillion-parameter AI model on a single CS-3 system, their AI training clusters. This accomplishment, announced at NeurIPS 2024, showcases the capabilities of Cerebras’ Wafer Scale Cluster technology.

Traditionally, training models of this magnitude necessitate thousands of GPUs, extensive infrastructure, and specialized teams. In contrast, the CS-3 system, utilizing a 55-terabyte MemoryX device, managed this feat without requiring any modifications to the model or infrastructure code. This streamlined approach significantly reduces the complexity and resources typically associated with large-scale AI training.

Following the initial success, the model was seamlessly scaled across 16 CS-3 systems, achieving near-linear performance improvements with a 15.3x speedup.

TheWhiteBox’s takeaway:

What makes Cerebras’ chips different from those of NVIDIA’s GPUs that have allowed this amazing training accomplishment?

The key difference is how they approach chip development.

Chips are electronic circuits sliced and diced from a circular Silicon Wafer after being tested (the defective dies, which happen a lot, are usually discarded). This GPU chip is then mounted on a circuit board with the other components (CPU and memory chips, among others for intra-board communication and networking).

Instead, Cerebras takes the entire wafer and…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.