Astonishing breakthrough cuts LLM training costs by 100x!

🏝 TheTechOasis 🏝

Breaking down the most advanced AI systems in the world to prepare you for your future.

5-minute weekly reads.

Before we delve into this breakthrough, let me celebrate something with you guys, 😊.

This week, I was mentioned by Adrienne Gibbs, Director of Creator Growth @ Medium, in her piece about the importance of AI in the Medium platform and in our lives, part of the “What We’re Reading” issue Medium’s staff recurrently publishes to their 75 million followers.

Not mean to boast here, just wanted to simply express how happy I am when people point to me as a, if I may, “reference” in the field of AI, the industry I love and to which I dedicate countless hours of study and research every week of my life.

And the fact she’s doing so is thanks to you guys.

🤯 AI Research of the week 🤯

Turning off the “I appreciate you guys” mode to “value” mode, today we’re talking about something big.

No super-mega-amazing-world-changing released LLM, no mind-boggling AI-generated images or 3D renders.

There’s plenty of room for that in our newsletter, but today we’re talking about money.

“Ugh, irrelevant and boring”, you may think.

Well, actually quite the contrary, as there’s no industry in the world today more heavily influenced by money than AI.

It drives every decision, both technical and strategic.

And this week a group of researchers have announced an unprecedented feat.

The elephant in the room

“Costs, man.”

That’s my bet on what’s probably coming across Sam Altman’s mind, the CEO of OpenAI, every waking hour.

In fact, according to Analytics India Magazine, the company could actually go bankrupt by 2024.

I don’t believe for a second that will happen, as OpenAI is already “en route” to $1 billion in revenues, but the $700,000 costs per day (previous link) of running the models are a statement of how problematic costs are in the world of AI.

Be that training or inferencing a model, be that your bottleneck is computation (training) or memory (inferencing)… costs are huge.

Taking Microsoft Azure prices as a reference, running an 8-GPU NVIDIA A100 cluster on their cloud costs around 27 $/per hour. Considering that big models run for months at times during training, and with much, much bigger clusters, the costs rise at an unprecedented scale.

For reference, the first family of LLaMa models, burned $5 million in just 21 days when training their 65-billion-parameter model.

GPT-4 is reportedly a trillion-parameter-sized model, so Sam Altman wasn’t really exaggerating when he estimated the total cost at around $100 million.

Yes, you read that right. And he actually said it was “more than that”.

And as size has only proven to be hugely correlated with increased quality (and showing no signs of saturation) the temptation to get big is natural, so the world is off to the races to burn. more. money.

But do we need to? Well, these researchers say no.

The problems of today’s AI

There…

Enjoyed the introduction? Subscribe or sign in to continue with this edition.