The David Mayer Case, AI Valuations Down, & Desktop SOTAs

TLDR;

Technology:

😍 A SOTA Model in Your Desktop Computer?

🤯 Intellect-1 Finally Finishes Training

😩 Jailbreaking Robots is Easy

Markets:

🌏 AI Valuations Are Coming Down to Earth

🔐 US Imposes Additional Export Controls on China

Product:

🧐 The Weird David Mayer Case

🎭 The Race to AI Shopping Assistants is On

FRONTIER AI

A SOTA Model in Your Computer

Probably the most remarkable result of the week, an X user has managed to turn Alibaba’s new flagship model, Qwen QwQ 32B, into a state-of-the-art model despite being extremely quantized.

The results are so remarkable that you could run a fully state-of-the-art large reasoner model (an LLM with search, the seventh on the list above) on your desktop hardware if you have at least 20 GB of RAM (which is not a lot in today’s terms).

Most models have a 2 bytes (16 bits) per weight precision. Thus, if a model has 32 billion parameters, it weighs 64 GBs or RAM. Here, models are being quantized (reducing precision) up to 4 or 3 bits per weight, effectively cutting down memory requirements by more than four times.

I’ll soon publish a Notion article covering the technical details.

TheWhiteBox’s takeaway:

There are several things to discuss here. First, China is dead serious about claiming the first spot in frontier AI. Alibaba's and Deepseek’s models are crazy good (and open-source).

The second point is that you may have noticed that the framed models in the image above have two models in one. What this means is even more fascinating: They achieved these impressive results using a speculative decoding setting.

How speculative decoding works

In these settings, to speed up token generation and reduce costs, a smaller model drafts possible continuations, with the larger model choosing the best one. This allows the overall system to generate multiple tokens for every forward pass of the large model and draft them much faster, as the drafting model is much smaller.

The thought of having a SOTA model in your computer during 2025 is not as far as it seems.

DECENTRALIZED AI

The Biggest Decentralized Model Has Finished Training

Prime Intellect has announced the release of INTELLECT-1, a 10-billion-parameter language model we discussed a while back that was collaboratively trained across five countries and three continents, utilizing up to 112 H100 GPUs simultaneously.

This initiative marks a significant advancement in decentralized AI development. It demonstrates that large-scale model training can be achieved through distributed, community-driven approaches rather than being confined to large corporations.

TheWhiteBox’s takeaway:

This is hugely positive news. We, as a society, need this. We need to know that we can build powerful AI models through peer-to-peer collaboration, giving us comfort that the only people on this planet who can build frontier AI models, behind a handful of corporations in the US and China, aren’t the only ones that can build powerful AI models, and that we can also reap the benefits and get paid for this collaboration.

This makes AI a consumption tool…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.