Plato, DeepSeek, SaaS' Future, & Much More

TLDR;

Welcome to this week’s premium news rundown. Today, we discuss a lot of products, from o3’s increasingly dangerous behavior to DeepSeek’s newest model.

We also take a more philosophical look at AI, with research that proves all models converge on the same ‘geometry of meaning.’

Additionally, you will see me rant about what I believe is the obvious future of software, based on new releases by Mistral and the latest hot startup in AI, Superblocks.

Enjoy!

FRONTIER AI MODELS

System Prompts of Popular Products, Leaked

This repository provides free access to the system prompts of several models and products, ranging from OpenAI Codex to Devin. Reading system prompts explain a lot of the behavior of these models and, crucially, the best practices engineers apply to their own creations.

For example, the system prompt was where ‘someone who shall not be named’ modified Grok’s behavior to over-index on the topic of South African racism toward whites. This should give you an idea of the importance the system prompt plays in the behavior of these models when you interact with them.

FRONTIER CODING

Claude 4 Scores on Aider, Worse than Expected

Days after its release, we now have Claude 4's results in the Aider Polyglot benchmark, which is considered the primary coding benchmark. The results are not great when compared to OpenAI or Google, with Claude 4 Opus falling to third place in performance and even costing around twice as much as Google’s model despite yielding worse results.

Surprisingly, Claude 4 Sonnet underperformed Claude 3.7 Sonnet. Not only did the results fail to surpass older models like o3 or Gemini 2.5 Pro, but they also underperformed their previous generation.

Not the best look.

TheWhiteBox’s takeaway:

It must be noted that benchmarks are continually being lowered in my ‘value list’ by the day; they are merely one more reference to frame the comparison. However, they don’t command the level of importance they once did.

Nonetheless, my experience with Claude 4 for coding, especially Opus, has been great, surpassing that of Gemini 2.5 Pro. So, once again, I must remind you that there isn’t the best model for everyone; you need to try them and see what’s best for you.

People were also surprised that Claude 4 Opus with thinking was cheaper than the non-thinking version. In other words, Claude 4 Opus in reasoning form (which think for longer by generating more tokens) was more affordable than the non-thinking version, which doesn’t make much sense at first.

However, if you recall what we discussed a few weeks ago, reasoning models aren’t necessarily more intelligent but rather more sample-efficient, which rapidly explains why they might be cheaper in reasoning tasks; they arrive at the response faster.

Thus, for future reference, don’t think that non-reasoning models will be cheaper in all cases; if you need the model for a challenging task, thinking models are generally better and can even be cheaper.

But how can benchmarks stay relevant?

To me, the new frontier for model evaluations…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.