Google Opens its Magic Department

🏝 TheTechOasis 🏝

Breaking down the most advanced AI systems in the world to prepare you for your future.

5-minute weekly reads.

TLDR:

AI Research of the Week: Google releases RealLife, a new Diffusion Model that Feels like Magic

Leaders: The ‘Everything will be an LLM’ Paradigm

🤯 AI Research of the week 🤯

What I’m about to show you is unprecedented and almost magic.

Google Research, in collaboration with Cornell University, has announced RealFill, an image inpainting and outpainting model with shocking results.

The model takes as a reference a set of images and allows you to fill in missing parts of a target image based on the former.

But what does that mean?

The model is capable of using the references on a handful of pictures to fill (inpaint) or expand (outpaint, the case below) while respecting the reference.

It’s capable of doing this despite the references being in other camera angles or lighting, as the model extracts the key features of those images and successfully applies them to the new generations.

This way, you don’t have to imagine how the perfect-but-cropped image should have been like.

Now, you can simply ask for it.

But let’s dive deep into how they actually created this magical model.

The 21st Century Michelangelo

Like most state-of-the-art Generative AI models today, RealLife is a diffusion model.

Diffusion models are AI systems that learn to transform an image in a Gaussian (random) distribution of data into a target one.

In other words, they take a ‘noised’ image, they predict the noise it has, and they take it out, uncovering the new image.

The image above portrays the training process. From a ground truth image (left), noise (random pixel values) is consistently added to the image, returning the image on the far right.

At the same time, the model takes in a condition describing the original image, such as “a cat lying down”.

Then, the model has to predict the noise, take it out, and reconstruct the initial image using the text as a ‘clue’.

Consequently, when generating new images, the model can successfully depart from an arbitrary noised ‘canvas’ and a text condition like ‘draw me a cat’ and get a new image that matches the original.

If you’re having trouble comprehending what this actually means, think about this quote from one of the most talented humans to ever work the Earth, the Rennaissance man Michelangelo:

The sculpture is already complete within the marble block. I just have to chisel away the superfluous material.

In a way, the model is doing just the same.

From a bunch of random pixels placed in an arbitrary way, the model is capable of ‘chiseling away’ the noise and uncovering the desired image:

However, not everything was going to be the bells and whistles.

In order to avoid generating the same image every time, the diffusion process always starts from random noise, thus the request “a portrait of a man” will give you slightly different results every…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.