Google's Infinite Attention, What Grok Tells about Tesla , & More

In partnership with

🏝 TheTechOasis 🏝

part of the:

Breaking down the most advanced AI systems in the world to prepare you for the future.

10-minute weekly reads.

TLDR:

AI News of the Week: Apple, Stanford, OpenAI…

AI Research of the Week: Toward Infinite Attention, the Breakthrough Google was Waiting for?

Why Grok 1.5 Vision Might Safe Tesla

🚨 Week’s Update 🚨

Welcome back!

As usual in AI, the week has plenty of new announcements. For starters, we got ourselves the first important firing of researchers for leaking information. And the executor was none other than OpenAI.

Apple announced that all new AI enhancements coming to the iPhone with iOS 18 will be privacy-preserving by being “on-device”.

In layman’s terms, the iPhone won’t require an Internet connection to run your AI systems. This is important because, as we saw last week, Apple will introduce Multimodal Large Language Models into their stack to revamp Siri, among other products.

Contrary to what many believe, MLLMs do not require Internet connection. As long as you can store the weights file and the executable on your device, you are good to go.

Apple’s research on Flash LLMs starts to make much more sense now…

For the seventh year straight, Stanford has released its AI Index Report, with notable insights on how AI is accelerating scientific progress (more on that in Sunday’s Leaders newsletter) and how regulators worldwide are increasing their pace.

Moving on, Google Deepmind has unveiled the ALOHA project to build low-cost highly-dexterous robots. The videos uploaded by researchers show robots tying shoelaces, or hanging a shirt.

For the next news, the video speaks for itself. This robot coming from your darkest nightmares is Boston Dynamics’ latest humanoid.

Fun fact, Boston Dynamics has historically avoided using AI. I wonder if that continues to be the case as Tesla or Figure.ai rely entirely on LLMs to work.

On the hardware side of things, Logitech has unveiled an ‘AI mouse’ that in reality, it’s simply your standard mouse seamlessly integrated with ChatGPT to access the LLM much faster and efficiently.

It’s nice to see AI being embedded into products. Can’t wait to see Logitech brand itself as “an AI company” though, get ready for that nonsense too!

On a final note, let’s get our tin hats on, as reports suggest that OpenAI and Meta’s new models (potentially GPT-5 and LLaMa 3) will show a stark improvement in reasoning and planning.

Surely not AGI, but something better than what we have seen until now.

📈 Toward Infinitely-Long Sequences 📈

Long sequences are an absolute unlocker for multiple use cases like DNA processing, long summarization, or video processing, which require orders of magnitude more data than text.

This week, Google presented Infini-attention, a new attention mechanism variant that promises the capacity to scale input sequences to infinite lengths.

What’s more, as we’ll see in a minute, it’s highly likely this is the breakthrough that allowed Google to release Gemini 1.5 weeks after announcing Gemini 1.0 but with…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.