Claude 3.7 Sonnet, Nightmare AIs, & More.

TLDR;

Today, we examine several product releases, mainly Anthropic’s Claude 3.7 Sonnet and its coding product Claude Code, as well as new feature releases by OpenAI, Google, or Microsoft.

We will also review the latest AI nightmare from two AIs talking in unintelligible language, NVIDIA scoring massive performance results with Blackwell, LLMs playing Pokemon, and the harsh reality of model censorship.

Enjoy!

RESEARCH

LLMs Playing Pokemon

A group of researchers at Anthropic is working towards achieving a significant milestone in AI: making it play complex games like Pokemon.

While previous LLMs were terrible, it seems that Claude 3.7 Sonnet’s thinking mode (more on this model below), when LLMs think for longer on problems, coupled with tools and vision to move and see the game, turns out to be the real unlocker, as the model went from barely being able to beat a contender to actually defeating Brock and Misty (Pokemon works by you training small “animals” to get passed several ‘battle gyms.’ Brock and Misty are the leaders of the two first gyms), making Claude 3.7 Sonnet the first model to play the game successfully for a long time.

TheWhiteBox’s takeaway:

Defeating these two gyms takes hours, proving that these models can handle complex tasks requiring hundreds of steps. This is a huge milestone, considering that, as we discussed in the past, agents' accuracy falls dramatically the larger the number of steps in the task they have to solve.

But does this mean agents have finally conquered robustness?

No, not quite. This video game allows the user to make mistakes, meaning that the AI can try and try until it meets the goal, with no clear penalties because it’s a video game. Real life, especially enterprise use cases, doesn’t work that way, so agents' robustness limitations remain a problem today.

AI MODELS

Which Models Let You Criticize the Governments

A researcher has conducted a censorship study to analyze how different models respond to user requests to criticize world governments. Unsurprisingly, Chinese models reject or are evasive about criticizing the Chinese government. On the other hand, ChatGPT tends to deny criticizing Chinese and Russian governments but is happy to criticize Western ones, probably because progressive ideas from the West will define doing this as ‘morally good.’

Of all the models, the most censored (by far) was Claude 3.5 Sonnet from Anthropic, which mostly outright rejected criticism of any government.

Anthropic’s new model, Claude 3.7 Sonnet, mentioned below, improves refusal rates by 45%. That said, as you’ll see below, this doesn’t really matter because you won’t be using Claude to talk about world governments.

TheWhiteBox’s takeaway:

But it’s funny to see that while most people claim Chinese models are the most censored, no one comes close to the levels of censorship that Anthropic’s models have.

Anthropic has notoriously favored more AI safety, as many of its leaders are considered AI doomers or people somewhat scared of what AI can become. However, AI safety has seen a massive loss of interest lately, as…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.