DeepSeek, Mistral, & Clueless Analysts

TLDR;

Welcome back! This week, we have Amazon’s new flashy AI servers, an essay against space data centers, products from Runway and anti-slop prompts by Cursor, releases by Mistral and DeepSeek with very different outcomes, as well as news in markets, including a potential Big Lab IPO.

Enjoy!

HARDWARE

Amazon Presents Trn3 Servers with a Bang

The AI chip race intensifies. Yesterday, in their re:Invent yearly event, Amazon presented Trn3, their brand-new AI server packed with 144 Trainium chips and offering very impressive specs.

The stock popped on the news (not surprising considering it’s suffering from an AI discount), but is investor optimism warranted?

Let’s cover the specs first:

Compute: 144 Trainium chips (double the scale-up of NVIDIA’s Blackwell Ultra), offering 360 PetaFLOPs at FP8 of dense compute (no sparsity or marketing tricks), meaning this server can output 360 quadrillion math operations per second at peak theoretical throughput, or 360,000 trillion operations per second (or 360×1015 ). Chips are connected in an all-to-all topology, using Amazon’s own NeuronSwitch and NeuronLink (i.e., all Trainium chips are directly connected to each other).

As we learned on Monday, the actual FLOPs you get in practice are way lower than what chip designers claim.

Memory. Up to 20.7 TeraBytes of HBM3e memory, with 144 GB per chip (144×144). This means each chip has 48 3-GB HBM dies, stacked into 4, 12-die stacks (the actual notation would be 144 chips each with 4×36 GB 12-Hi stacks). In layman’s terms, every chip has 4 HBM stacks, each with 12 memory chips, each with 3GB of capacity. Interestingly, you can actually see the HBM stacks (the four squares on the edges)as the stack’s height, or relief, is visible to the human eye:

Memory bandwidth. These HBM stacks give 708 TB/s of aggregate bandwidth (meaning the server’s communication system can move 700 TeraBytes of data in and out of memory every second—not to be confused with chip-to-chip interconnect speeds, which will surely be way slower, probably less than ~300 TB/s).

The server offers five times more token throughput per MW than its previous version, meaning a considerable improvement in efficiency, measured by how much more efficient it is for the same token output.

Render generated using Nano Banana Pro based on a slide in the presentation, which is why you’ll see some mistakes like getting the OpenAI model name wrong (it’s GPT-oos-120B)

But the critical question here is, how does this compare to NVIDIA’s servers (or Google’s)?

For starters, this server is almost guaranteed to require a lot of power. We don’t know the value, but I’ve read some experts estimate 600-800 W of TDP per chip, so we could be moving around 115 kW per rack, which is lower than what NVIDIA’s Blackwell Ultra server requires (140kW) but more than what a TPUv7 rack requires (~70kW).

To put that into perspective how much power that is, data centers from just a few years ago would have around 5 kW per rack. Today, power draw is so high…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.