o1's True Nature, Chiva vs US, & More

TLDR;

Technology:

😏 Decoding the Llama

😎 Building state-of-the-art SRMs

Markets:

😔 New Escalation of Chip Restrictions

Product:

🧐 o1 Isn’t a Chatbot, and That’s Ok.

RESEARCH

Decoding the Llama

Goodfire has open-sourced a series of Sparse Autoencoders (SAEs) that provide insight into the Llama 3.3 models by decoding their activations into human-interpretable features.

Importantly, SAEs allow you to intervene these models, and crank up or dial down different behaviors you want to enhance/prevent from the model. Using the example above, a user used SAEs to turn Llama 70B into ‘Worm Llama,’ a model that thinks it’s a worm (by jacking up the ‘worm’ feature).

Interestingly, you can try using the following link for free.

TheWhiteBox’s takeaway:

SAEs are an essential part of Mechanistic Interpretability, which aims to decode frontier AI models. In layman’s terms, the field that wants to help us understand AI behavior and, crucially, show us how we should steer these models.

While our current models are helpful and powerful, they are also very unpredictable, especially in areas that are unfamiliar to them.

They have even been accused by their own creators of deceiving the user by committing to a plan the user did not define while pretending to be helping him/her.

Alignment, the idea that AIs remain loyal and obedient to humans, is the main reason behind many frontier researchers’ sleepless nights (whether that’s exaggerated or not lies in the eyes of the beholder, but many of the most competent people alive are indeed scared).

Either way, we must find ways to steer and control these models to make them predictable. However, for now, SAEs are closer to toys than real products.

If you test the Goodfire SAEs in the provided link, you’ll see how the model can be steered. However, the behavior is still unpredictable in areas it doesn’t know very well.

I tried turning Llama into ‘Paella Llama,’ and while the model does acknowledge its sudden passion for cooking, particularly Paellas, I was nowhere close to the results of the above image; the model's steerability is still pretty much dependent on its knowledge of the subject.

It’s not ‘Paella Llama’, but it’s ‘Chef Llama.’ Close enough

LRMs

Building state-of-the-art SRMs

A team at UC Berkeley has achieved something amazing. Dubbed Sky-T1-32B-Preview, it’s a reasoning model that matches the performance of o1-preview on popular reasoning and coding benchmarks.

Sky-T1-32B-Preview was fine-tuned from Qwen2.5-32B-Instruct, a powerful open-source model. The training process utilized 17,000 curated data points, including 5,000 coding problems from APPs and TACO, 10,000 math problems from AIME, MATH, and Olympiads subsets of the NuminaMATH dataset, and 1,000 science and puzzle problems from STILL-2.

Impressively, the model’s training cost was only $450. It was completed in 19 hours on eight H100 GPUs, and it still performed comparable to that of o1-preview in complex benchmarks like Math500 or AIME 2024.

TheWhiteBox’s takeaway:

One of the most exciting things about large and Small Reasoner Models (SRMs) is that adding a new scaling law (which improves performance by…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.