The Anatomy of the AI Mind

In partnership with

THEWHITEBOX

The Anatomy of the AI Mind

Welcome back! This Leaders segment is going to go back to the roots of this newsletter and tackle the recently less discussed part of AI… the AI itself. We are exploring a new paradigm for AI models that may render the Transformer (ChatGPT’s architecture, which has dominated the entire industry for almost a decade) obsolete.

This time, the goal is to conquer continual learning, the ability of AI to learn forever, and the thing that all AI Labs are working on exhaustively, once and for all.

And this proposal isn’t coming from an unknown research team. Today’s proposal is coming from Google itself (creators of the Transformer), and they’ve announced this piece of research with all the bells and whistles, which is not typical behavior at all.

And with Google’s Gemini 3.0 just around the corner, and with some testers arguing that Gemini 3.0 is the biggest leap in performance since GPT-4 (the biggest real update in more than two years), I believe this is too much of a coincidence, and we might be at the cusp of the first true paradigm shift in years.

If AI can truly be a winner-takes-all market, continual learning is as close as it gets to being the ‘it’ factor.

Take today’s post as less about investing, bubbles, and debt, and more so about what this newsletter was always about: understanding AI like no one around you without unnecessary jargon or esoteric wording.

Let’s dive in.

This is a long read that requires concentration. So, please leave it for a time when you’re not too mentally drained. It’s not dull, but it will make you think.

Anterograde Amnesia, ChatGPT’s Natural State

Current models “suffer” from a condition that in humans we know as ‘anterograde amnesia’.

This is a memory impairment in which the ability to form new long-term memories after the onset of the condition is disrupted. In contrast, older memories from before the event are largely preserved.

This is actually a spot-on way of understanding how models behave. They can learn new things, but the moment we train them on new material, they easily forget what they learned before, an “illness” known as catastrophic forgetting, which prevents this update and thus condemns the AI to this amnesia-like state.

But to understand why this might soon be a thing of the past, we must first answer: What is a model?

What it means to ‘train’ AI

An AI model is nothing but a ‘map’ between an input and an output. It takes an input, and it gives you back an output. More formally, it’s a learning algorithm that “studies” the data and finds the patterns that allow it to predict well an output derived from that data.

Suppose you are Zillow, the largest US real estate marketplace, and you want to offer customers a model that predicts the price of a home. They could theoretically train a model that takes in a lot of…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.