Behold the Power of Smaug & AI's Holy Grail, Neurosymbolic Systems

🏝 TheTechOasis 🏝

Breaking down the most advanced AI systems in the world to prepare you for your future.

5-minute weekly reads.

TLDR:

AI Research of the Week: Behold the Power of Smaug

Leaders: The Holy Grail of AI, Neurosymbolic models, are Here

🤩 AI Research of the week 🤩

"My armor is like tenfold shields, my teeth are swords, my claws spears, the shock of my tail is a thunderbolt, my wings a hurricane, and my breath death!"

That was the words of the main antagonist in one of The Hobbit films, the huge dragon that almost killed our dear protagonist Bilbo while scaring him to death.

Now, the same feeling might have been felt by the open-source community as Abacus.ai has released Smaug, a fine-tuned version of Qwen_72B by Alibaba that is unequivocally the new open-source king, and the first open-source model in history to reach an average of 80 points across benchmarks.

Also, it’s undeniable proof that we finally have a canonical method that will close the gap between open-source and proprietary models, DPO, which we talked about a few weeks ago.

Let’s unpack the secrets of the king under the mountain.

A Thing of Beauty

You don’t have to take my word that this is the best open-source model, as it already appears as such in the most famous Open LLM leaderboard from HuggingFace.

And although Alibaba is the one who grants the license (it’s not a truly open license like Apache 2.0) it’s fairly permissive, meaning you only have to request permission to use it commercially if your product has more than 100 million users.

I’m sure you can accept that.

So, what is Smaug?

Smaug is a fine-tuned, DPO-aligned version of Qwen-72B, a great Large Language Model (LLM) by the Alibaba group that is heavily influenced by Meta’s LLaMa 2 model.

As you probably know, LLaMa is a family of pre-trained generative transformers (meaning they are the equivalent of GPT to OpenAI or Gemini to Google) that are widely considered the best open-source base models in the industry.

In fact, they are the seminal model sitting behind most open-source models today.

Moreover, as Mark Zuckerberg himself acknowledged, they are already training the 3.0 version, which means we could be weeks away from the next big leap in open-source capabilities.

In the meantime, nothing beats Smaug in the world of open-source and, incredibly, it manages to beat some of the best proprietary models in the world too, including Gemini Pro from Google (in some benchmarks) and Mistral-Medium from Mistral, according to Abacus AI founder, Bingu Reddy.

Were these claims to hold true at scale, that would make Smaug a top-5 foundation model overall, putting it almost at the level of GPT-4 and Gemini Ultra despite being more than ten times smaller!

But what are these benchmarks?

Winograd: Tests AI's understanding of language ambiguity requiring common sense.

TruthfulQA: Assesses AI's ability to provide accurate and non-misleading answers.

GSM-8k: Challenges AI with grade-school level math problems.

ARC:…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.