🏝️ Physical AI, OpenAI's Illegal Side, & New Super Models

In partnership with

THEWHITEBOX

TLDR;

🤯 OpenAI Reimagines Image Generation

😍 Anthropic’s New Models

📹 HeyGen Zoom Avatars & RunWay’s Act-One

🧐 Solving the 9.8<9.11 Dilemma

🎭 Adobe’s New Image Rotation Tool

😳😳 Ex-OpenAI Claims Illegal Activities

[TREND OF THE WEEK] Physical AI Foundation Models

NEWSREEL

OpenAI Redefines AI Image Generation

Source: OpenAI

As rare as seeing pigs fly, OpenAI has released open research that, for once, is actually insightful and contributing to the space. They explore the idea of consistency models, diffusion-based architectures that, instead of standard diffusion models that require hundreds of steps to output the image, can generate it almost instantly.

But what are diffusion models? The golden standard for image and video generation models, they aim to denoise a noisy image like the one in the gif above based on the text the user provides to the model (e.g., “draw a butterfly”).

Mathematically speaking, the model receives the text input and a noisy canvas (a random distribution of data). In an iterative step-by-step approach, the model progressively predicts noise in the image and removes it, eventually arriving at a new distribution (butterflies in this case) that semantically relates to the user's request.

Intuitively, think of diffusion as a similar process to creating a marble statue; you initially have a marble block, and by chiseling away the excess, you uncover the hidden figure. Therefore, diffusion can be understood as chiseling out the noise to uncover the image.

TheWhiteBox’s takeaway:

To me, this is an unmistakable sign that OpenAI is no longer an AI lab but a fully-fledged AI product company. The idea behind these models is not to push the industry forward or take us closer to AGI but to make image generators faster and cheaper to run.

It’s also a sign that model distillation is here to stay. These faster models are distillations of diffusion-based models. In other words, you first pre-train a more powerful yet slower diffusion model. Then, you distill its knowledge and capabilities into a smaller and also more sample-efficient one to serve to customers.

Distillation involves taking a more powerful model (teacher) and use it to train a less powerful but more efficient model (student) by teaching it to imitate the responses of the larger model.

Distillation is steadily becoming the main deployment method for generative AI solutions. First with LLMs like GPT-4o and Claude 3.5 Sonnet (both distillation of GPT-4 and Claude 3 Opus, respectively) and now with image generators, deployment has become a game of who deploys the best Pareto-optimized model, a model that gives you 80% performance over the bleeding frontier while requiring 20% of the costs (the numbers are illustrative to illustrate the point).

PRODUCT

Anthropic’s Claude 3.5 Sonnet v2 & Computer Use

Source: Anthropic

Anthropic has released two new models, Claude 3.5 Sonnet (New) and Claude 3.5 Haiku, as well as a new computer use feature that allows Claude to control your screen through the API.

The announcement has generated a lot of hype because the new…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.