OpenAI Goes Mad, Music LLMs, & Beating Pokemon

In partnership with

THEWHITEBOX

TLDR;

This week, we comment on OpenAI’s insane price for o1-pro, a striking difference in the size and quality of HuggingFace’s new Olympic-level coder, new models from Perplexity, and an interesting Deepmind/Disney/NVIDIA partnership.

We then move into markets to discuss SoftBank’s latest acquisition and Apple’s fragile status quo. Finally, we look at Anthropic’s new feature for Claude and creating music with LLMs using MCP.

Enjoy!

FRONTIER

$600 for OpenAI o1-Pro

OpenAI has set a new record in both price and deliria with their latest prices for the o1-pro API access, their best—released—reasoning model.

The price?

$150/$600 per million input/output tokens respectively, 270x the price of DeepSeek R1.

TheWhiteBox’s takeaway:

The audacity of the price is incredible.

$150 per million input tokens and $600 per output million tokens are outrageous prices. A million tokens are around 750k-ish words, which sounds like a lot, but it’s not for reasoning models.

These models need to speak to think, so they tend to generate an absurd amount of tokens to answer, around twenty times what non-reasoning models require, skyrocketing prices.

If you’re wondering why the output tokens are four times higher than the input tokens (usually, it’s three times), this is due to the nature of both computations. Generating words is much more inefficient to run on GPUs, as a greater percentage of the GPU’s energy consumption is used to move the different model parts and the KV Cache from memory to the cores and vice versa, instead of actually making computations.

On the other hand, during input processing, we are building the cache by processing all words in the input at once, which is very heavy computationally speaking, ideal for GPUs (more energy consumed on compute than on memory transfer).

The ratio of compute/data transfers in proportion to total energy consumed is called arithmetic intensity, a term AI incumbents love to use to sound smarter.

These companies make money with compute, not data transfers, so word generation has to be priced higher to make money.

BUT NOT THAT HIGH.

While nobody doubts that the o1-Pro model is the best around, it is heavily rumored to be a best-of-N-chains model. It generates several plausible chains of thought (multi-step problem-solving) to answer your request and choose the best one, a technique known as majority sampling or ‘self-consistency sampling.’ If that is true, this price, exactly 1000x that of GPT-4o mini, is audacious.

For the non-technical reading this, if your CIO/CTO suggests using this model for this price, fire them. And if you really find the use case, just pay the $200 subscription.

OPEN-SOURCE

A Small Olympian-Level Coder

A HuggingFace team has released OlympicCoder 7B and 32B, which showcase borderline state-of-the-art (only behind ChatGPT) results when tested on last year’s International Olympiad in Informatics exam.

To achieve this, the team built a custom dataset by distilling reasoning traces from DeepSeek R1 (sampling R1 to solve complex problems and then using its responses to fine-tune a base model). Interestingly, despite being trained on…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.