[Premium] SOTA at Home, Insane CAPEX, & NVIDIA's Cosmos

TLDR;

Technology:

🌏 NVIDIA’s Cosmos World Models

🤩 Visualizing Attention.

😏 Anthropic’s Agents Guide

Markets:

😳 Microsoft’s Insane AI Spending for 2025

😱 Encode Attacks OpenAI

Product:

🧐 Grok-3 Delayed

🤯 NVIDIA’s Personal Supercomputer

FRONTIER RESEARCH

NVIDIA’s Cosmos World Models

A week has passed since we kicked off 2025, and one of my predictions is already shaping up to be a reality (I hope you don’t mind a little bit of self-praise). Nvidia has introduced Cosmos, a suite of foundational AI models designed to enhance the capabilities of humanoid robots, industrial automation, and autonomous vehicles.

These models enable machines to interpret and interact with their surroundings more effectively, facilitating tasks such as navigation and object manipulation. Cosmos aims to improve their adaptability and performance in complex environments by providing robots with a more comprehensive understanding of the world.

TheWhiteBox’s takeaway:

It’s hard to negate the fact that portraying large language models (LLMs) or ‘think for longer’ LLMs, called large reasoner models (LRMs), as the only models needed for AGI seems shortsighted, considering that these models lack any meaningful interaction with the open world. Without that, it’s hard to fathom an AI that can truly understand, predict, and inhabit our world, whether for robotics or simply to create accurate representations of our world, crucial for video generation models.

To embody that reality, we need world models, models that perform a very similar prediction to what the human brain does: for a given state of its surrounding environment, predict what will happen next.

The human brain knows that jumping off a cliff or hitting a cement wall with your bare hands at full force is not the best option.

While LLMs/LRMs can predict those types of situations as dangerous, a true world model has to perform several predictions per second that ensure our body’s survival in a partially predictable environment where things continually change (like the real world). For that, LLMs/LRMs simply lack the experience to make such predictions, as such experiences can’t be portrayed through text.

As we discussed in our predictions issue, world models—and Cosmos is no different—are video generative models, aka models that predict frames in a video instead of text (although Cosmos models can also ingest text). A simple way to frame these models is as world simulators. These models can openly simulate the successive frames in a sequence of video frames, indirectly predicting the next thing to happen in that environment.

NVIDIA isn’t the only company actively building these models. Google is very open about its research on world simulators, mainly through the Genie 2 model. Other companies, like OpenAI, also refer to their video generation efforts as ‘trying to build world simulators.’ This model is on the minds of all of them.

Thus, in 2025, we should expect many more releases of this type. As we predicted, AI labs will eventually combine their efforts on LLMs/LRMs with video models to build an advanced AI system that can reason and accurately inhabit the open world, a crucial…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.