Meta Open-Sources Llama 4: How Free Frontier AI Changes the Cost of Building

What Actually Shipped

Meta released the Llama 4 model family as open-weights software, meaning anyone can download the weights, run inference on their own hardware, and modify the model without paying per-token fees. That is a real and significant event. The largest variant in the family is a mixture-of-experts architecture reported to activate roughly 17 billion parameters per forward pass from a total pool of around 400 billion — a design that keeps inference costs lower than the headline parameter count implies. Meta published the weights, a system card, and an acceptable-use policy simultaneously. Those three things together constitute the release. Everything else is context you should evaluate carefully.

What the Architecture Actually Does

Mixture-of-experts, or MoE, works like a building with many specialist departments: when a question arrives, a routing layer directs it to only the relevant specialists rather than consulting every employee in the building. This keeps the computational cost of answering any single query manageable, even though the full "building" is very large. The tradeoff is that MoE models are harder to fine-tune than dense models — you have to train the router and the experts together, which requires more memory and more careful engineering than adjusting a single block of weights. Meta's multimodal claims — that Llama 4 processes text, images, and video natively — are technically present in the architecture, but the degree to which video understanding is production-ready versus early-stage differs substantially across reported evaluations.

What the Benchmarks Do and Do Not Prove

Meta's published scores place Llama 4 Scout and Llama 4 Maverick at or near GPT-4o and Gemini 1.5 Pro on several standard tests. Standard benchmarks — MMLU, MATH, HumanEval — measure specific, bounded tasks under controlled conditions. They are useful but routinely optimized against during training, which means a high score tells you the model was trained on data resembling those tasks, not necessarily that it will perform equivalently on your actual workload. Independent reproducers have noted that scores on some reasoning benchmarks drop when prompting style deviates from the format used in Meta's evaluation setup. That is not unusual; it is, however, a fact that should sit between you and any cost projection built on headline numbers. The open-weights nature of the release is a genuine advantage here: unlike a closed API, you can run your own evaluation rather than trusting vendor-reported figures.

Questions You Should Be Asking

• Can you reproduce Meta's reported benchmark scores on your own hardware using your own prompting strategy? If the numbers drop when you change the prompt format, the model has learned to answer that specific test, not the underlying skill.
• What does your actual inference infrastructure cost look like for the MoE variant at production load? A model that activates 17B parameters per token still requires all 400B parameters to be loaded into memory, which dictates your hardware floor before you write a line of application code.
• Which modalities are covered by the acceptable-use policy, and what commercial use cases are explicitly carved out? Open-weights does not mean unrestricted; the license terms for Llama 4 contain category restrictions that may affect certain verticals.
• What is your fine-tuning plan, and have you budgeted for the added complexity of tuning an MoE model? Teams that found dense Llama fine-tuning straightforward should not assume the same process transfers without significant re-engineering.
• How will you evaluate the model's video understanding claims against your specific use case before committing engineering resources? Native multimodal architecture and reliable multimodal performance are not synonymous, and the gap between them is where production timelines disappear.

What To Watch Next

The signal that will clarify Llama 4's real-world position is the accumulation of third-party evaluations on domain-specific tasks — legal, medical, coding, and multilingual workloads — published by teams with no relationship to Meta. If those scores hold within a few points of Meta's figures across varied prompting strategies, the release is as consequential as claimed. If they don't, the story becomes about how frontier-model marketing is outrunning frontier-model measurement, and the open-weights advantage shifts from performance to auditability.