What Birds Tell Us About Whales, & More

In partnership with

THEWHITEBOX

TLDR;

Welcome back! Today, we have a newsletter packed with insights, from the ongoing memory supercycle, Google’s incredible Perch model that uses birds to understand whales, China’s new SOTA video model, and deep dives into the importance of debt in AI and new products like OpenAI’s updated Deep Research.

Enjoy!

AI & ANIMALS

Google Does it Again. What Birds Teach About Whales

There are some AI applications that make it really hard to believe they actually exist and aren’t a complete fabrication, and this is one of them.

Google has trained a bioacoustics model called Perch 2.0, primarily on birds and other terrestrial vocalizing animals, and shown that the same model can be used to identify marine animals with few-shot learning (simply giving it examples of new data it has never seen, and the model still works).

Specifically, a model trained mostly on birds is given a handful of whale examples and can suddenly identify whale species, no problem. That’s how crazy this research is.

But how is this possible?

TheWhiteBox’s takeaway:

The answer is, funnily enough, already known. In simple terms, even if it sounds crazy, there’s an overlap between the sounds of birds and the sounds that whales emit. That is, some acoustic features are common across the animal kingdom, and some other acoustic features are common across all sounds, both living and not.

By training an AI in large amounts of terrestrial animal data, the model might not have seen any whale sounds, but it’s already capable of picking up the two sets of acoustic features I just mentioned. In layman’s terms, this model, even if whales are new to it, already knows, for lack of a better expression, “what sound is,” and even picks up the particularities of whale sounds.

Consequently, when faced with what seem to be totally new sounds, in the model's eyes, much of what these sounds are is already familiar. Therefore, with just a few extra training sessions, or even just examples, the model is now an expert on whale sounds and can identify them with no problem.

For a better intuition, always think of this in human terms. Imagine your brain had to relearn what sound “sounds like” every time it heard a new sound.

Luckily for us, the human brain already knows what sounds “sound like”, so it just needs to “fine-tune” itself to the new sound, so that it becomes familiar for future reference. Which is to say, our brains perform transfer learning all the time, applying learnings from one dataset to another.

The only difference, and what makes AIs unique here, is that the level at which AIs can process these patterns goes above and beyond our capacity, meaning we can use these AIs to identify sounds we can’t quite tell what they are with surprisingly low effort.

As I always like to say, hype makes it really hard not to hyperfocus on Large Language Models (LLMs), but we have to remember that AI…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.