Meta launched 3 new models, the largest one being Llama 3.1 405B. They are also calling it “Open-Source”. But Is It? Despite the official announcement from Meta, people are debating online whether the models can be called open-source.
Highlights:
- Meta CEO Mark Zuckerberg shared an open letter explaining why the Llama model is ‘Open-Source’.
- Developers can run inference on Llama 3.1 405B at a 50% cheaper cost than GPT-4o.
- Some critics are not happy with the ‘open-source’ approach by Meta.
Is Meta’s Llama 3.1 405 Open-Source?
The latest model Llama 3.1 405B has been released by Meta after teasing it for months. It has been continuously trained with a 128K context length following pre-training with an 8K context length, supports multiple languages and accepts multi-media inputs like images, videos and files.
In an official letter, Meta’s Founder and CEO Mark Zuckerberg said this about Llama’s latest model launch:
“Today we’re taking the next steps towards open source AI becoming the industry standard. We’re releasing Llama 3.1 405B, the first frontier-level open source AI model, as well as new and improved Llama 3.1 70B and 8B models.“
Mark Zuckerberg
The term “open-source” usually means that the software is free to use, and anyone can look at the code, change it, and deploy it. In this case, Open-Source AI Models mean that anyone can freely download them and run them on their machines. You can use it to train your own models for specific tasks, like a “teaching model”.
Mark believes that by making these models open to everyone, more people can work on them, improve them, and create new things with them.
He gave the example of Linux to explain his point. At the time before its origin, major tech companies were working on their closed-sourced version of Unix. But then came Linux. People feared it at first, but now it has become a standard. The ability to modify the code because of its “open-source” nature made it more advanced. He also wants Llama the new standard in AI.
His thought is simple. If the model is open, it will become more efficient, and if it does, it will become the next industry standard.
Here are some of the main reasons for making it open-source:
- Most Organizations want models that they can run themselves.
- They don’t want to send their sensitive data to closed-sourced models.
- They need a model that they can train and fine-tune.
And most importantly, it’s cheaper. Developers can run inference on Llama 3.1 405B at a 50% cheaper cost than GPT-4o.
Finally, he said how this decision will have a big impact on the world:
“I believe that open source is necessary for a positive AI future. AI has more potential than any other modern technology to increase human productivity, creativity, and quality of life – and to accelerate economic growth while unlocking progress in medical and scientific research.”
Mark Zuckerberg
That’s easy to understand. If more people will have access to AI, it will have more benefits for humankind. It will be secure because historically, that has been the case. If it is open for all, anyone can scrutinize it.
The Counter Argument
While Mark says it is ‘open-source’, not everyone believes it. While they say that the model is trained to not do anything harmful, the open nature makes it possible to remove such safeguards. The same argument of it being risky came with the same Linux story that Mark shared (we discussed it above).
There are also some restrictions put on by Meta, and that’s why experts are not sure to call Llama an ‘open-source’ AI model as of yet.
Via ZDNet, here’s what Stephen O’Grady from RedMonk industry analyst has to say:
“It’s welcome news that Meta has dropped some of the use restrictions around Llama, but as long as it still restricts which companies can use the software, as the new license does, it’s clearly and inarguably not open source. If Linux, for example, were released under this license, Meta would be entitled to use it, but companies like Amazon, Google, and Microsoft could not. That’s not open source, nor would we accept it.”
-Stephen O’Grady
Some people are also speculating that Meta is doing all of this for distribution and profit from its massive user base as it is a social media company.
Still, we can say that with Llama 3.1 405B, we now have an open-source model that beats some of the best closed-source ones available today on selected benchmarks.
Conclusion
So, no one can say who is completely right or wrong, but the argument is in front of you and now you can decide what you think about it. Also, we should mention that in March this year, Elon Musk and xAI openly released Grok-1, a 314B parameter language model.
