TheWhiteBox · Independent AI analysis

Meta's ultimate AI object detector, and the convergence between AI and Crypto with zk-proofs

Ignacio de Gregorio Noblejas · April 9, 2023

🏝 TheTechOasis 🏝

🤖 This Week’s AI Insight 🤖

Meta announces the ultimate AI object detector, which you can try now for free

Foundation models like GPT-4 powering tools like ChatGPT are the reason you’re reading this newsletter now.

The paradigm shift they’ve caused in AI has transformed the sector from one with great potential to one that’s shaping not only how we work, but ultimately how we live our lives.

Until now, this shift was only visible in Generative AI, the AI field that generates text like ChatGPT, images like Stable Diffusion, and soon video, while other applications remained wishful thinking.

But all of a sudden, Meta has released what we can consider the first foundation model for image segmentation, and the results are breathtaking.

And more importantly, you can try it right now from your own browser.

Computer Vision on Steroids

Meta research has released its Segment Anything Model, or SAM.

This model, trained with a huge dataset of 11 million images with up to a billion masks (a mask being the detected object in an image), shows incredible promise to be used as a generalistic model for object detection or generation.

In layman’s terms, given an image and a prompt (an instruction to what you want to detect, given as a cursor point, a box, or even free text) the model identifies the object you’re aiming for.

And as you can see in the above image sampled from the model, SAM is capable of identifying up to 100 different objects per image with uncanny accuracy.

But how does this model work and how can you try it?

Transformers as a way of living

SAM relies heavily on Transformers, the architecture used by ChatGPT that, by introducing the concepts of positional encoding, multi-head attention, and parallelizable computation, allows machines to extract context from data (text or images) incredibly well.

SAM works as follows:

A Vision Transformer (ViT) generates an embedding of the image (a representation of the image in the form of a vector)

A prompt encoder (they used OpenAI’s CLIP encoder) obtains the embedding of the prompt

A mask decoder then combines the information from these two embeddings and maps it into its output, a mask that represents the detected object in the image.

Et voilà.

Bottom line, SAM can represent the next big leap for AI, taking computer vision to the same levels of hype that Generative AI solutions are enjoying today.

Luckily for you, you don’t have to trust Meta or me to understand how amazing this tool is, you can try it for yourself and get mindblown by giving it your own images.

Enjoy!

👾AI news for the week👾

🤔 Thought-provoking article to why LLMs like GPT-4 “make stuff up”, and ways to mitigate it.

♾ Google announces new supercomputer, says it beats the industry-leading NVIDIA chips