For developers focused on building professional artificial intelligence (AI) models, a long-standing challenge is obtaining high-quality training data. Smaller expert models (with 1 billion to 10 billion parameters) often use the "distillation technique" to enhance their training datasets using the output of larger models. However, the use of such data from closed-source giants such as OpenAI is strictly restricted, thus greatly limiting commercial applications.
On the evening of July 23 (Tuesday) Beijing time, the long-awaited open source large model "ChatGPT moment" by developers in the global AI field finally arrived - Meta released the latest AI model Llama 3.1, among which the version with the largest parameter scale is Llama 3.1-405B.
Zuckerberg called Llama 3.1 "the starting point of art" and will benchmark against OpenAI and Google's large models. Test data shows that Meta Llama 3.1-405B surpassed the most advanced closed-source model OpenAI GPT-4o in multiple AI benchmarks such as GSM8K. This means that the open source model has defeated the most advanced closed-source large model for the first time.
Moreover, the launch of Llama 3.1-405B means that developers can freely use its "distilled" output to train niche models, greatly accelerating the innovation and deployment cycle in professional fields.
01 Milestones of the open source community
In April 2024, Meta launched the open source large language model Llama 3. Among them, Llama 3-8B and Llama 3-70B set new benchmarks for large models of the same size. However, in just three months, with the iteration of AI functions, other large models quickly surpassed them.
In a competitive environment, Meta has recently released the AI model Llama 3.1, which includes three models: Llama 3.1-8B, Llama 3.1-70B, and Llama 3.1-405B. The first two are updated versions of the Llama 3-8B and Llama 3-70B models released in April. The Llama 3.1-405B version has 405 billion parameters, making it one of Meta's largest open source models to date.
In the early hours of the day of the release (Beijing time), the LocalLLaMA sub-forum of the "American Post Bar" reddit leaked early benchmark results of the three upcoming models.
Leaked data shows that Meta Llama 3.1-405B surpasses OpenAI's GPT-4o in several key AI benchmarks. This is an important milestone for the open source AI community: for the first time, an open source model has defeated the most advanced closed source large model.
The content published by Meta team research scientist Aston Zhang on X also confirmed the leaked test data.
Image source: X
Specifically, Meta Llama 3.1-405B outperforms GPT-4o in multiple tests such as IFEval, GSM8K, ARC Challenge, and Nexus. However, it lags behind GPT-4o in multiple MMLU tests and GPQA tests. In addition, the context window of Llama 3.1 covers 128,000 tokens, which is larger than the previous Llama model and is approximately equivalent to the length of a 50-page book.
Image source: X
It is important to note, however, that these benchmarks reflect the performance of the base Llama 3.1 models. The true potential of these models can be realized through instruction tuning, a process that can significantly improve the capabilities of these models. The upcoming instruction-tuned versions of the Llama 3.1 models are expected to produce even better results.
02 Llama 4 started training in June
Although OpenAI’s upcoming GPT-5 is expected to have advanced reasoning capabilities that may challenge Llama 3.1’s potential leadership in the large model field, Llama 3.1’s strong performance against GPT-4o still highlights the power and potential of open source AI development.
You know, for developers who focus on building professional AI models, the long-term challenge they face is obtaining high-quality training data. Smaller expert models (with a parameter scale of 1 billion to 10 billion) usually use "distillation technology" and need to use the output of larger models to enhance their training data sets. However, the use of such data from closed-source giants such as OpenAI is strictly restricted, which greatly limits commercial applications.
The launch of Llama 3.1-405B means that developers can freely use its "distilled" output to train niche models, greatly accelerating the innovation and deployment cycle in professional fields. It is expected that there will be a surge in the development of high-performance, fine-tuned models that are both powerful and in line with open source ethics.
“If this data is true, then it could be argued that the top AI models will become freely available to everyone starting this week,” wrote Ethan Mollick, an associate professor at the University of Pennsylvania’s Wharton School. “Everyone in the world will have access to the same AI capabilities. This will be interesting.”
The open source of Llama 3.1-405B also proves that the gap between the open source model and the closed source model has narrowed again.
Image source: X
The reporter of Daily Economic News also noticed that in addition to the highly anticipated Llama 3.1-405B, foreign media reported that Llama 4 had started training in June, and the training data included public posts of users on social platforms Facebook and Instagram. Before the start, Mate had sent more than 2 billion notifications to users in the European Union, where data privacy regulation is the strictest, providing the option of not agreeing to use their data for large model training.
It is reported that Llama 4 will include text, image, video and audio modalities. Meta plans to apply the new model to mobile phones and smart glasses.
This article comes from the WeChat public account "Daily Economic News" , the author is a reporter from Daily Economic News, and is authorized to be published by 36Kr.
