Near-telepathic AI, billion$ settlements, & More

TLDR;

Welcome back! Brace yourself because this has been a very eventful week so far. We have it all: billion-dollar settlements, new great models, more stellar fundraisings, and even a near-telephatic AI product (yes, you read that right).

Enjoy!

RESEARCH

Grok’s “Background Thinking”

As shown in a screenshot shared by an X user, xAI is allegedly preparing the release of a new feature in the Grok app that allows users to interact with the model while it’s thinking.

The feature is not official, and might just be xAI beta testing the product (testing on a small subset of users), seeing if it makes sense for them.

But why not simply open a new conversation?

Of course, opening a new conversation lets you start off and immediately ask Grok new questions.

However, the point here is for cases where you are deep into the weeds of a particular topic with the chatbot, and opening a new conversation causes the model to lose the context. While you might want to send the model in a long, deep research endeavor that may take 20 minutes, you may also want to discuss other parts while the Deep Research instance goes at it. Thus, this feature enables you to do just that.

→ READ TWEET

TheWhiteBox’s takeaway:

As you have probably realized, interacting with AI models has become a “slower” experience because reasoning models take longer to think about a task and, consequently, to answer.

It’s a trade-off most of us are willing to make, but that doesn’t necessarily make us feel better about the wait. For instance, GPT-5 is an impressive system, but at times I wish it were faster; it can be painfully slow.

Therefore, the fact that these Labs start offering a solution to this problem that allows you to continue to interact with the model as it thinks about your previous question is excellent news.

Theoretically, this isn’t that hard to do: store the cache until that particular point, and simply open two parallel model instances with the same context but responding to different questions. This doesn’t sound that dissimilar to OpenAI’s recent new feature, ‘branching,’ which allows you to open new conversation paths in a single conversation. Of course, this is a reductionist point of view of how this is being applied, as we know the real issues come from the engineering side, aka how to handle this branching mechanism at the GPU level, because opening new model instances is very memory-demanding, even potentially making these companies raise an entire new server based on this new branch.

Instead, the most likely process is to introduce both conversation paths in the same sequence batch being processed (GPUs process multiple sequences in parallel, meaning several users are being served by the same GPU at once). This can be done because, by branching, you ‘lose’ the causality of the interaction, as both sequences are now independent of each other and not one based on the other. Therefore, they can be processed in parallel.

Enjoyed the introduction? Subscribe or sign in to continue with this edition.