Microsoft's Orca2 and Q*, the Secret that Almost Killed OpenAI

🏝 TheTechOasis 🏝

Breaking down the most advanced AI systems in the world to prepare you for your future.

5-minute weekly reads.

TLDR:

AI Research of the Week: Orca2, an Amazing New Type of LLM

Leaders: Q*, the Secret Model that Almost Killed OpenAI

🤯 AI Research of the week 🤯

As we discussed the previous week, Small Language Models (SLMs) are all the rage right now.

Also, Microsoft launched the new version of their SLM crown jewel, Orca2, creating a new type of language model, Cautious Reasoners.

They set a new threshold in the AI industry by beating models up to 10 times larger in highly complex reasoning tasks.

Also, they have provided a clear view of their AI strategy, as well as invaluable insights into the complex world of Transformer learning.

And today we are going to deep dive into how they created this new paradigm.

The Imitation Game

When Microsoft presented Orca’s first version, the first open-source model that was truly at the level of ChatGPT-3.5, the AI industry finally started paying attention to smaller models.

Today, Orca is not only viewed as a critical innovation, but it’s also a centerpiece in Microsoft’s strategy, as it is rumored that the LLM running behind Microsoft’s Copilots, the talk of the town in the industry, is not ChatGPT but Orca, due to the insane costs of running +100 billion parameter models.

Microsoft’s premise is simple, if we can get a model that gives us 90% of the larger model’s capabilities at a 10x fraction of the cost, we are taking that route.

But you may wonder, how are we building models ten times smaller than the big guys while having most of their capabilities?

And the answer to that is distillation.

Teach me the ways, ChatGPT

Far and wide, the most common way of training small language models is through distillation, the imitation game.

First and foremost, there’s an inescapable truth that as language models grow bigger, they become better as they learn new capabilities.

Research from Google and Anthropic prove that LLMs show no signs of saturation. In other words, no one has reached the point LLMs stop learning as they grow in size.

Consequently, as neural networks can learn anything if provided with sufficient data, researchers realized: What if instead of training a model to learn to model language by itself, we taught it to copy another model?

This is what we call distillation, the training process where a student model learns to imitate a teacher model by learning the data distribution of its responses. In other words, to learn to imitate it.

But how does this work?

Taking as an example LLMs like ChatGPT, they output the probability distribution over the next token.

In layman’s terms, given a text sequence, they output the word/subword with the highest probability of being ‘adequate’ out of the complete LLM vocabulary.

For instance, for the text sequence “The UK’s capital is London”, we mask—hide—the word “London” and the model’s objective…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.