Deceiving AIs, Ray3, & Hardware Bombshells

In partnership with

THEWHITEBOX

TLDR;

Welcome back! This week, we take a look at fascinating news coming out of China, with Huawei’s new breathtaking SuperCluster, as well as NVIDIA’s billion-dollar bet on Intel.

We also take a look at fascinating research, seeing AIs setting new records in reasoning benchmarks, OpenAI’s alarming findings regarding models that deceive users, and Luma Labs’ incredible Ray3 model, among other news.

Enjoy!

HARDWARE

China’s Incredible MegaPod

The photo above won’t mean much to you, but it has surely sent chills down the spines of many US AI incumbents, especially NVIDIA and AMD.

Huawei has announced the Atlas 950 and 960 SuperPoDs, the former set to launch globally in the second half of next year, each with 8,200 and 15,000 NPUs in a single pod.

They also announced two SuperClusters, one going to 500k Ascend NPUs and the other going up to a million NPUs.

An NPU, a neural processing unit, is a type of AI accelerator (like a GPU), but one that is specialized for AI workloads while also focused on maximizing performance per energy cost.

And the results speak for themselves. Just the 950 PoD, the smallest of the two, blows NVIDIA’s upcoming Rubin servers (expected for the second half of next year and the second half of 2027) out of the water in terms of raw performance.

27 times more inference tokens per second (take this number with a pinch of salt)

17 times more training tokens per second (take this number with a larger pinch of salt)

7 times more compute power

15 times more memory with 62 times more total memory bandwidth

Chip-wise, the improvements are notable, especially when reaching the more advanced versions coming in 2027/2028, with memory sizes matching upcoming American chips, and with intra and cross-NPU memory bandwidths (a measure of how much data per second they can share) also in the same ballpark as American counterparts promise:

An absolute monster. Ok, but where’s the catch?

First and foremost, one thing is to put numbers on a slide; another is to deliver. Additionally, these pods have 57 times more accelerators than NVIDIA’s servers, meaning each single NPU is considerably inferior to every NVIDIA GPU. It’s not even close.

Other stuff worth noting:

I seriously doubt the NPUs are connected in an all-to-all topology, meaning they are separated into ‘subpods’. This means that, at the core, this is a lesser parallelizable cluster than NVIDIA’s, because you can find some NPUs having to communicate with other NPUs several hops in distance from them, considerably increasing communication overhead.

For that reason, it would have been fairer to compare this cluster to Google’s Ironwood. Google’s largest pod has 9,216 TPUs (another accelerator type) and, get this, 42.5 ExaFLOPs, five times the compute power of the 950 SuperPoD. The 960 SuperPoD likely doubles its value to 16 ExaFLOPs with 15,000 NPUs, still dwarfed by Google’s cluster. It’s clear to me why they avoided such a comparison; they aren’t remotely close.

Their previous largest…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.