Meta unveils a new, more efficient Llama model

Meta has unveiled the latest member of its Llama generative AI model family: Llama 3.3 70B. Ahmad Al-Dahle, VP of Generative AI at Meta, shared on X that Llama 3.3 70B offers the same performance as Meta's largest model, Llama 3.1 405B, but at a significantly lower cost.

"By utilizing the latest advancements in post-training techniques, this model enhances core performance at a much lower cost," Al-Dahle wrote. He also shared a chart highlighting Llama 3.3 70B's superior performance over Google's Gemini 1.5 Pro, OpenAI's GPT-4, and Amazon's newly released Nova Pro across several industry benchmarks, including MMLU, which tests a model's language comprehension. In an email, a Meta spokesperson noted that the model is expected to show improvements in areas such as mathematics, general knowledge, instruction following, and app usage.

Introducing Llama 3.3 — a new 70B model that delivers the performance of our 405B model but is easier & more cost-efficient to run. By leveraging the latest advancements in post-training techniques including online preference optimization, this model improves core performance at… pic.twitter.com/6oQ7b3Yuzc

— Ahmad Al-Dahle (@Ahmad_Al_Dahle) December 6, 2024

Llama 3.3 70B, available for download on platforms like Hugging Face and the official Llama website, is Meta’s latest effort to lead the AI space with "open" models that can be used and commercialized for various applications. However, Meta's terms limit how certain developers can use the Llama models, requiring platforms with over 700 million monthly users to obtain a special license. Despite these restrictions, the Llama models have been downloaded more than 650 million times, reflecting their widespread appeal even if they aren’t "open" in the strictest sense.

Meta has also integrated Llama into its own operations. Meta AI, the company’s AI assistant powered entirely by Llama models, now boasts nearly 600 million monthly active users, according to CEO Mark Zuckerberg. He claims Meta AI is on track to become the world’s most-used AI assistant.

For Meta, the open nature of Llama has brought both opportunities and challenges. In November, a report claimed that Chinese military researchers had used a Llama model to create a defense chatbot, prompting Meta to make its Llama models available to U.S. defense contractors. Additionally, Meta has raised concerns about its ability to comply with the EU’s AI Act, which it sees as too unpredictable for its open-release strategy. The company also faces challenges related to the General Data Protection Regulation (GDPR), particularly regarding AI training on the public data of Instagram and Facebook users who have not opted out, as this data is subject to GDPR protections in Europe.

Earlier this year, EU regulators asked Meta to suspend training on European user data while they evaluated the company’s compliance with GDPR. Meta agreed but simultaneously supported an open letter calling for a "modern interpretation" of GDPR that doesn’t hinder progress.

Like other AI labs, Meta is grappling with technical challenges and is increasing its computing capacity to train and support future generations of Llama. On Wednesday, the company announced plans to build a $10 billion AI data center in Louisiana, its largest to date.

During Meta’s Q4 earnings call in August, CEO Mark Zuckerberg revealed that training the upcoming Llama 4 models will require 10 times more computing power than was needed for Llama 3. To support this, Meta has secured a cluster of over 100,000 Nvidia GPUs for model development, putting it in competition with other AI leaders like xAI.

Training generative AI models is an expensive endeavor. Meta’s capital expenditures surged nearly 33% to $8.5 billion in Q2 2024, up from $6.4 billion the previous year, driven by investments in servers, data centers, and network infrastructure.