← All editions

TheWhiteBox · Independent AI analysis

There's really little to celebrate this week

Ignacio de Gregorio Noblejas ·

Sign in or subscribe to keep reading ↓

TLDR;

It’s safe to say AI has had better weeks than this one we’re discussing today. From stories about Chinese spies, valuation cuts, and revenue stagnation, this week has it all (bad).

It has some good things to, like the hottest new thing in tech circles: formal methods (which aren’t precisely new), as well as products to do more humane fishing.

Enjoy!

RESEARCH

Formal Proofs are the future of Vibe Coding

In the film Margin Call, the bank's CEO says his job is knowing when the music is about to stop. He then says he “can’t hear a thing” on the brink of collapse, amid a potential huge devaluation of its subprime mortgages.

My job here isn't that of a CEO, but it’s definitely knowing what music is coming from the AI trade so they can hear it coming. And what I’m hearing the most these weeks is formal methods, which might be the solution to the problems of AI-generated coding.

That’s quite a statement, so let’s dive a little.

San Francisco tech circles now think they know the secret to solving code (unsurprisingly, it seems coding was not actually solved, unlike what some of these pundits claimed). But now they think they have it.

People from places like OpenAI, or startups like Axiom, are defining formal methods, proofs specifically (more on this in a second), as the “solution” to solving code once and for good.

But to understand formal methods, we first need to understand the problem with vibe coding. As I’ve explained time and time again, AIs can only learn what can be measured, and the quality of that measurement defines the quality of learning.

Put simply, AIs learn much better and faster when they can verify the correctness of their outputs (if they add 1+2 and they get 4, they immediately know something is wrong).

In fact, this is so true that one can argue that, without extremely objective and clear verification, what AIs learn is the average measurement, resulting in pretty average results.

This is why AIs are now great at maths but suck at poetry. It’s because maths is intensely verifiable and provable, while poetry, defining what “great poetry” is, is hard and, most of the time, lies in the eyes of the beholder.

Therefore, for domains like maths, learning means sending AIs gradually more complex problems and, because most maths is verifiable, AIs getting better is mostly an exercise in compute and time.

But with less verifiable domains like poetry, all we can do is send the model millions of poems and hope it learns poetry. It does learn it, but without measurable progress from average to great, AIs collapse into the mean; they become the representation of average poetry.

That is, without problem-solving-oriented training, all they can do is imitate, and imitation yields average results spread across the millions of poems they've imitated.

One verifiable domain is coding. However, coding is not as verifiable as maths. Defining “correctness” isn't as simple as finding…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.

TheWhiteBox · Subscriber access

Keep reading.

Already subscribed? Sign in with your newsletter email.

New here?Subscribe free ↗

Then return here to sign in and continue reading.