Mixture-of-Depths, Unveiling the Future of Siri, 🦙LLaMa 3, & more

Sponsored by

🏝 TheTechOasis 🏝

part of the:

Breaking down the most advanced AI systems in the world to prepare you for the future.

5-minute weekly reads.

TLDR:

AI News of the Week

AI Research of the Week: Google presents Mixture-of-Depths, and it looks amazing

Apple ReALM and Ferret-UI, the Future of Siri?

Leaders: Here’s why AI Will be Decentralized

🚨 Week’s Update 🚨

Welcome back! This week is packed with very important news.

Starting off, Meta has announced that they are soon releasing LLaMa 3, the model family of LLMs.

I suspect they will be multimodal at least for images, and considering how Meta is consistently pushing forward the state-of-the-art for audio and speech, with examples like Seamless M4T, I would not discard the possibility of seeing the first audio/speech native Large Language Model.

ChatGPT’s GPT-4V LLM is not audio native, it’s just calling APIs to Whisper for speech/audio transcription and a TTS model for the opposite.

Talking about our friends at OpenAI, GPT-4 Turbo Vision is now generally available to developers through the API, so builders can now handle image and text processing from one single API (as it’s the same model).

Moving on to hardware, some really spicy news came from Phoenix. Intel presented its new accelerator chip, the Gaudi 3, that beats NVIDIA’s H100 chips by a landslide, especially regarding training speed and power efficiency.

This isn’t that surprising as Stability AI already pointed out recently how they were obtaining better results with Intel’s previous chips, the Gaudi 2, over the H100, so this margin should increase more with the new version. That being said, NVIDIA has yet to release its new accelerator, Blackwell.

Fun fact, Intel is 13 times less valuable on the stock market compared to NVIDIA. Thus, is Intel massively undervalued, or is NVIDIA outrageously overvalued?

On the inference side, we still need to pay attention to Groq, as their hardware, the Language Processing Units (LPUs) appear to be outright superior to any GPU in that regard. You can try Groq’s hardware here for free.

However, some are arguing for some time that the return of analog systems is inevitable. I highly recommend this video if interested.

Continuing on hardware, in a more futuristic fashion, it seems Sam Altman is pairing with Jony Ive, legendary product designer of the iPhone, iPad, and others, to build a personal AI device that ‘won’t be similar to an iPhone’, and are seeking up to $1 billion in funding.

Hard to imagine what they are referring to, but the closest thing we have to an AI personal device is Rabbit’s R1, which relies entirely on voice interaction to execute the tasks you require.

Fun fact, Rabbit’s AI system is neurosymbolic. Unlike standard LLMs, it combines neural networks with high-level human-coded programming to increase performance and decrease training times and costs.

I want to end this week’s update with an interesting piece by The New York Times on how most AI labs are desperately—and illegally—cutting corners to amass…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.