A Self-Correcting Humanoid, Grokking & a Tool that Works

In partnership with

‘

THEWHITEBOX

TLDR;

📰 News from FigureAI, OpenAI, and NapkinAI

🧐 Insights on semiconductors, AI economics, and a rundown on what matters in AI today

😍 Trend of the Week: Grokking

The fastest way to build AI apps

Writer Framework: build Python apps with drag-and-drop UI

API and SDKs to integrate into your codebase

Intuitive no-code tools for business users

Start building with AI Studio

NEWSREEL

Figure 02, The Self-Correcting Robot

Without a doubt, in the biggest news of the week, Figure.ai has announced its newest version of its humanoid, with a cleaner look and improved intelligence, to the point that this robot is allegedly self-correcting, which means it can correct its own actions.

The video includes a rundown of the new features and shows the new humanoid performing actions in a BMW factory, which is being tested for eventual deployment as an autonomous factory worker.

TheWhiteBox’s take:

There are many things to explain here.

For starters, these robots run on OpenAI intelligence; they have a multimodal vision language model (probably a tailored variation of GPT-4o) that takes images from its 6 cameras and the robot’s proprioception (the position and angles of its 16 different joints at any given time) to decide the next move. For more details, read my article on HumanPlus by Stanford.

Importantly, it performs speech-to-speech reasoning, meaning it can take instructions by voice and execute the actions while explaining what its doing.

Undeniably, the self-correction mechanism is the most impressive thing. Although they don’t provide details on how this works, this behavior is almost guaranteed to be possible just in very specific situations (like the one shown in the video).

The model has possibly memorized certain poses (what the cameras see and the position and angle of the joints must be a specific pose, just like HumanPlus did) and essentially moves itself toward that pose, correcting when necessary.

On a final note, I’m very impressed with the energy the humanoid consumes, around just 2.25 kWh. For reference, a standard hairdryer consumes around 1.2-1.8 kWh, so from an energy perspective, this humanoid is ‘household ready.’

OpenAI’s Structured Outputs

OpenAI has announced structured outputs, a new feature that allows you to impose the structure of the JSON output on the model’s API via a JSON Schema, a critical step forward in building accurate agents.

In other words, in situations where you need the model’s response to be of a specific format (a critical requirement to connect LLMs to third-party APIs so that the model can perform actions on other systems), you can specify the ad hoc structure and the model will comply with uncanny accuracy.

TheWhiteBox’s take:

It’s been almost a year and a half from OpenAI’s last model leap release with GPT-4. Since then, the start-up that was allegedly conceived to ‘build AGI’, seems much more focused on product than on its supposed vision (this is precisely what Elon is arguing in its latest lawsuit against them).

In fact, according to some, they have…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.