Move over GPT-4o, Meta releases its biggest open-source AI model yet
Meta has announced a significant milestone in the AI industry: the release of Llama 3.1, the largest open-source AI model to date. This groundbreaking model, which Meta claims outperforms leading private models like OpenAI's GPT-4o and Anthropic’s Claude 3.5 Sonnet, is set to revolutionize the field.
Alongside this, Meta is expanding its Meta AI assistant's availability to more countries and languages, introducing a feature that can generate images based on a person's specific likeness. CEO Mark Zuckerberg predicts that by the end of this year, Meta AI will surpass ChatGPT to become the most widely used assistant.
Llama 3.1 is an advancement from its smaller predecessors released a few months ago. With 405 billion parameters, it was trained using over 16,000 of Nvidia’s high-end H100 GPUs. Although Meta hasn't disclosed the exact cost, it's estimated to be in the hundreds of millions of dollars.
Why Open Source?
Despite the hefty price tag, Meta is releasing Llama 3.1 as open-source. Zuckerberg, in a blog post, explained that open-source AI models are advancing faster than proprietary ones, similar to the rise of Linux in the operating system domain. He likened Meta's investment in open-source AI to its earlier Open Compute Project, which saved billions by collaborating with companies like HP to improve and standardize data center designs.
Verge reported that Zuckerberg believes the Llama 3.1 release marks a pivotal moment, where most developers will start to prefer open-source models. To support this transition, Meta is collaborating with over two dozen companies, including Microsoft, Amazon, Google, Nvidia, and Databricks, to help developers deploy their versions of Llama 3.1.
Meta claims that Llama 3.1 is approximately half the cost of running OpenAI’s GPT-4o in production. By releasing the model weights, companies can train it on custom data and fine-tune it to their needs.
While Meta remains tight-lipped about the specific data used to train Llama 3.1, citing trade secrets, it's known that synthetic data—generated by models rather than humans—played a significant role. Ahmad Al-Dahle, Meta’s VP of generative AI, suggests that Llama 3.1 will serve as a "teacher" for smaller models, making them more cost-effective for deployment.
Addressing concerns about the quality of training data, Al-Dahle acknowledges a potential ceiling but believes there's still room for more training runs. For the first time, Meta’s red teaming (adversarial testing) of Llama 3.1 included exploring cybersecurity and biochemical applications. Emerging “agentic” behaviours, where the model performs complex tasks like integrating with a search engine API to retrieve and process information, were also tested.
Meta AI assistant and new features
Meta's AI assistant, built on Llama 3.1, is a general-purpose chatbot now integrated into Instagram, Facebook, and WhatsApp. Starting this week, Llama 3.1 will be accessible through WhatsApp and the Meta AI website in the US, with Instagram and Facebook following soon. New languages, including French, German, Hindi, Italian, and Spanish, are also being added.
Despite offering the advanced 405-billion parameter model for free, Meta's assistant will switch to a more scaled-back 70-billion model after a certain number of prompts per week, indicating the high cost of running the larger model at scale. Meta spokesperson Jon Carvill mentioned that more information on the prompt threshold will be provided after assessing early usage.
One of the notable new features is "Imagine Me," which scans a user’s face via their phone’s camera, allowing them to insert their likeness into generated images. This method aims to avoid creating a deepfake machine by not using profile photos. Meta anticipates a demand for AI-generated media, even if it blurs the line between reality and AI.
Meta AI will also be incorporated into the Quest headset, replacing its voice command interface. Users will be able to utilize Meta AI on the Quest to identify and learn about their surroundings through the headset’s passthrough mode.
The road ahead
Zuckerberg’s prediction that Meta AI will be the most-used chatbot by the end of the year sets a high bar, especially given that ChatGPT already boasts over 100 million users. However, Meta has yet to share specific usage numbers for its assistant. Al-Dahle acknowledges that the industry is still in its early stages, striving for product-market fit.
Despite the hype surrounding AI, it's evident that Meta and other key players view this as just the beginning of the race. The release of Llama 3.1 and the expansion of Meta AI demonstrate Meta's commitment to leading the AI revolution, emphasizing the potential of open-source models to drive innovation and inclusivity in the AI landscape.