Elon's world-largest AI Cluster, Cohere's Mounstrous Round, New deep fake method, & more

TLDR;

🤜🏼🤛🏾 Mistral and Nvidia join forces

🫣 Elon announces world’s largest AI cluster

🔍 An overview of the future of search

🫡 Cohere raises monstrous $500 million round

👁️ New method to identify deep fakes

SOFTWARE PARTNERSHIPS

Mistral and NVIDIA join forces 🤜🏼🤛🏾

A state-of-the-art at its size

Mistral, the French star AI company valued at $6 billion, and NVIDIA, the world’s main GPU provider and one of the top three most valuable companies in the world, have presented Mistral-NeMo.

This state-of-the-art model has a 12 billion parameter size and a 128k context window, meaning that you can send up to 100k words in one single prompt, similar to what ChatGPT offers.

TheWhiteBox's take:

Besides being quite small, the model was trained on FP8 precision. In other words, every parameter takes up just one byte, so the model easily fits in mid-tier GPUs and can be served at scale very efficiently, as it only weighs 12 GB.

Also, it has been deployed on an NVIDIA microservice, also known as NIM, ensuring that the model is optimized to run on NVIDIA GPUs. Moreover, it excels at function calling, enabling you to run the model on pipelines with other tools, enhancing its overall usefulness and, importantly, reducing the chances of hallucinations (i.e., make the model call a calculator tool to perform computations instead of using its standard, high-error prone next-word prediction).

FRONTIER AI

Elon Musk’s xAI launches world’s biggest cluster 🫣

Finally delivering on its promise, Elon Musk can now brag about having the world’s largest AI cluster, with a staggering 100,000 NVIDIA H100 GPUs, the state-of-the-art ahead of the release of the NVIDIA Blackwell platform sometime this year.

The average H100 cost is $35k per GPU, although this number is probably far from exact as such large purchase orders that xAI has made to NVIDIA will have included discounts. That said, if we factor in other data center costs, the value of this cluster is surely around $4 billion.

With such a cluster, it’s probably a matter of time before Grok-3, the upcoming version of xAI’s Multimodal model, becomes the largest model ever trained.

TheWhiteBox’s take:

The numbers around this cluster are ridiculous. Nonetheless, the power needed to run this cluster is a staggering 138 MW, enough to power almost 150k homes at average US consumption rates.

For reference, GPT-4 was trained on a 20k A100 cluster for 90/100 days. That same model in this cluster?

- Knowing each H100 has a theoretical FLOPs peak of 198 ExaFLOPs, or 198 × 1018 FLOPs/second,

- and knowing GPT-4 was a workload of 2.15×1025 FLOP, or 21.5 million ExaFLOPs,

- this means that the cluster needs to run for 21,500,000 / 198 = 109k seconds, or 1.25 days. However, as the standard real peak of power in H100 clusters, or MFU, is 40%, that means the cluster would require between 3 and 4 days to train GPT-4.

For reference, that value would have been 100 days in 2022. Or, to be specific,…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.