Unraveling Google's super AI week. Gemini and the super coding model AlphaCode2

🏝 TheTechOasis 🏝

Breaking down the most advanced AI systems in the world to prepare you for your future.

5-minute weekly reads.

TLDR:

AI Research of the Week: The Day ChatGPT was Finally Defeated

Leaders: AlphaCode2, Google’s proof that Coding will Never be the Same

🤯 AI Research of the week 🤯

The day has finally come.

GPT-4, at least in terms of benchmarks, is no longer the best-in-class model.

Gemini Ultra, the biggest model in Google’s Gemini family of Large Language Models (LLMs), or dare I say, MLLMs (for Multimodal), has claimed the throne.

Gemini Ultra completely obliterates ChatGPT’s best model, GPT-4, by beating it in 30 out of 32 popular benchmarks.

Sadly, not all that glitters is gold, as several issues have overshadowed some of Google’s claims, to the point of being accused of faking some of the announcements.

A perfectly crafted marketing campaign, or is Google truly at the lead now?

Let’s find out.

Multimodal at the Core

With Gemini, Google has launched a family of Generative AI models.

From your pocket to the world

Developed by Google, it includes three main sizes: Ultra, Pro, and Nano.

Gemini Ultra: This is the most capable model in the family, designed for highly complex tasks. It delivers state-of-the-art performance across a range of benchmarks, including reasoning and multimodal tasks.

Gemini Ultra, to be released on Q1 2024 is the model that competes, and improves, over GPT-4.

Gemini Pro: Optimized for performance in terms of cost and latency, this model demonstrates strong reasoning performance and broad multimodal capabilities across various tasks.

In an apples-to-apples benchmark comparison, Gemini Pro showcases similar results to ChatGPT-3.5 Turbo, although beating it most of the time.

Gemini Nano: This model targets on-device deployment and comes in two versions, Nano-1 and Nano-2, designed for low and high-memory devices respectively. It is notable for its efficiency and is trained by distilling from larger Gemini models.

Nano-1 has 1.8 billion parameters and Nano-2 3.25 billion, and both have a 4-bit precision. This means that the former only requires 0.9 GB and the latter 1.7 GB, making them usable in many smartphones.

But what makes Gemini special is its multimodal approach.

From the ground up

Gemini represents the first frontier model—a best-in-class model—that is truly multimodal from the ground up.

As explained by Google, Gemini has been trained not only with loads of text, but text interleaved with images, audio, and even video, and is also capable of generating both images and text.

This means that, unlike most common multimodal solutions, where an LLM is frozen and the representations (vector embeddings) of other modalities are projected into the LLM’s input space, a process called grafting, in Gemini’s case all components are trained simultaneously so the model learns a global representation of features no matter their modality.

In layperson’s terms, a description of coffee, the sound of coffee being brewed, and an image of a cup of coffee will share very similar embeddings, signaling the model that they all refer…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.