Premium News Recap

TLDR;

Technology:

😣 AI’s Threat to Itself

🦟 For a Vector-Borne Free World

🤔 The First EU-Compliant Models are Here

Markets:

🤯 What to Expect from NVIDIA’s Earnings

📰 This week’s news on AI venture

Product:

🥳 The Pursuit of God is no More

👮🏼 Using AI to Write Crime Reports

TRAINING

AI’s Threat to Itself

The New York Times has released an interesting piece covering the problem of ‘model collapse.’ The article includes visuals and examples that really help understand the phenomenon where an AI trained on its own responses collapses and becomes useless.

This raises important questions about how AI labs will train their models once an important part of the Internet becomes AI-generated.

This is Important Because…

When training an AI in an uncontrolled way, the distribution of words the AI knows does indeed collapse into a smaller amount, increasing the likelihood the model outputs the exact words continuously.

However, training on synthetic data (AI-generated content, too) can also lead to great outcomes when the data is heavily curated.

Nevertheless, GPT-4o is a distilled version of GPT-4 Turbo. In other words, OpenAI researchers took the large model, used it to generate a huge dataset of synthetic data, and trained the smaller model on the outputs of the large one to create an even better model, a process known as distillation.

And how do we do that?

In this case, the objective function isn’t cross-entropy like in a standard LLM (maximizing the probability assigned to the most likely word) but a distribution divergence (usually using the KL Divergence formula).

In layman’s terms, the model (the student) is trained to not only match the probability assigned to the correct token but essentially match the entire distribution, aka learning to behave just like the parent model (the teacher).

But the article does make a point: People willing to train a model from scratch will have to deal with an increasingly complex open dataset; I don’t need to tell you how unreliable the open Internet has become as a knowledge source, and how vital LLMs’ role will be in the future as interfaces of global knowledge.

This means that, more than ever, we need open-source software to prevent a few people in Silicon Valley from controlling this interface.

HEALTHCARE

For a Vector-Borne Free World

VectorCam is an AI project supported by several universities and notably by the Melinda & Gates Foundation. It aims to support the fight against vector-borne diseases, pathologies carried out by vectors, and living organisms that transmit the disease, especially mosquitoes.

VectorCam identifies mosquito classes almost instantly, quickly identifying whether the mosquito could be carrying—or is carrying—diseases like malaria. These diseases require being nimble and acting quickly, which is precisely what rapid identification enabled by AI offers.

TheWhiteBox’s take:

I’ll say it again: the world of AI is much more than Generative AI, and part of my job with this newsletter is to go beyond the GenAI hype. Needless to say, computer vision algorithms like CNNs are already…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.