Key Takeaways
- Llama 3.1 outperforms the competition in benchmarks.
- Meta provides Llama 3.1 as an open source model.
- Commercial use of Llama 3.1 is restricted by the terms of the Community License.
Meta has released a new AI model called Llama 3.1. The company says that this new open source model can compete with, and even surpass in some benchmarks, the best proprietary models from other major AI companies, such as OpenAI’s GPT-4o and Anthropic’s Claude 3.5 Sonnet.
The official name of this model is Llama 3.1 405B. From this name, the name GPT-4o mini seems to be quite inspirational. But there is a reason for this name. This model is a 405 billion parameter model, a significant evolution from the Llama 3 8B and 70B models. This new model is available for free on Meta AI and WhatsApp in the US and can be downloaded by anyone.
Meta believes this model could be a turning point for AI development, comparing it to how open source Linux became the industry standard even as companies developed their own closed-source Unix systems. However, Llama 3.1 is not as open source as it may seem.
Related
Meta announced that its AI will be coming to Quest VR headsets in the US in August.
Meta AI will be coming to the company’s Quest headsets in the US and Canada in “experimental mode.”
How does Llama 3.1 compare to other AI models?
Mehta’s new model outperforms rivals in several benchmarks
Meta
As is common with any new AI model these days, Meta released some benchmark scores comparing its performance to other models. Looking at the table, the Llama 3.1 405B came out on top in 8 out of 15 benchmarks. While these tests should always be taken with a pinch of salt, the numbers show that Llama 3.1’s performance is on par with the best models currently available.
Meta
Interestingly, Google’s Gemini is absent from these tables. Meta claims that using Google’s API makes it difficult to reproduce previously published results, but this seems like an excuse. However, in a Threads post, Meta claims that Llama 3.1 405B “competes with leading closed source models such as GPT-4, Claude 2, and Gemini Ultra on a range of tasks.” However, none of these results are published in the post.
Is Llama 3.1 really open source?
To some extent, but not Linux
Meta is clearly bucking the trend by offering Llama 3.1 as an open source model. Other big players like OpenAI and Google make money through app subscriptions or usage fees via APIs. Meta is essentially giving away its models for free, so what’s the problem?
A big reason Meta has been able to do this is that it has an entirely different business model than a company like OpenAI, which is entirely an AI company and the only way it can make money is by selling access to its models. Meta, on the other hand, is almost entirely driven by advertising revenue. Developing AI models is a costly business, so Meta must be confident that it can be profitable in the long term.
To use Llama 3.1 in its products, Apple must apply for a license from Meta, which may be rejected at Meta’s discretion.
Now, let’s discuss just how open source Llama 3.1 actually is. While Llama 3.1 is open source to some extent, the Community License Agreement contains some interesting terms. Most interesting is that Llama 3.1 is free for commercial use as long as the product or service offered by the licensee does not exceed 700 million monthly active users. This seems like an oddly specific number.
This means, for example, that Apple cannot include Llama 3.1 for free in iOS 18 and make it available within Siri. To use Llama 3.1 in its products, Apple would have to apply for a license from Meta, which could be rejected at Meta’s discretion. Meta is not as altruistic as it may seem at first glance. Its intention is more likely to be to become an industry standard in AI and charge large companies large fees for licenses.
