What MLPerf v5.0 Measured
MLPerf v5.0 shipped in two rounds. Inference v5.0 (April 2, 2025) reported 17,457 results from 23 organizations and added four tests: Llama 3.1 405B, Llama 2 70B Interactive, RGAT, and Automotive PointPainting. The official inference results list the submitters and system configurations.
Training v5.0 (June 4, 2025) reported 201 results from 20 organizations and introduced a Llama 3.1 405B pretraining benchmark that replaces GPT-3. The official training results describe the suite and the submitted systems.
The suite is useful because it defines tasks, scenarios, and rules. It is not a forecast of your workload, and a result only compares cleanly with another result from the same task, scenario, division, and comparable system scale.
Headline Results, and What They Hide
The linked v5.0 announcements do not substantiate this article’s previous B300-versus-H200 multiplier or its tokens-per-second table. Headline claims should be traced to the exact benchmark, task, scenario, division, system or GPU count, precision, and software stack. A system-level result often reflects many accelerators working together, so it cannot be read as a per-GPU speedup.
Compare only results from the same task and scenario, and note the submitter, system size, and precision. Correction: this article previously included B300 and H200 throughput figures and multipliers that were not supported by the linked MLPerf v5.0 sources. Those figures have been removed rather than relabeled as illustrative.
| Round | Date | Task or change | Official result |
|---|---|---|---|
| Inference v5.0 | Apr 2, 2025 | Llama 3.1 405B; Llama 2 70B Interactive; RGAT; Automotive PointPainting | 17,457 results, 23 submitters |
| Training v5.0 | Jun 4, 2025 | Llama 3.1 405B pretraining (replaces GPT-3) | 201 results, 20 submitters |
The Submissions That Matter for Buyers
Closed division constrains the implementation so results are comparable across submitters. It does not require every system to use identical hardware or the same scale. Open division permits broader implementation changes, so it can show more of what a tuned software stack can do, but it is harder to compare across vendors.
For a buying decision, match the task, scenario, precision, system size, and division before comparing. A gap between open and closed is not a universal optimization budget for your workload; it is evidence about one submitted implementation.
Inference vs Training Takeaways
Inference results depend on model, context length, batch or concurrency, latency target, precision, and serving software. Training results depend on model, sequence length, global batch size, optimizer, parallelism strategy, interconnect, and time to target quality. Neither can be reduced to a single winner across all workloads.
Use v5.0 to shortlist systems that have submitted a relevant task, then test your model and traffic on the candidate configuration. Ask providers for the same task, scenario, and precision when they quote a comparison.
The Cost-Per-Result Lens
MLPerf reports performance, not price. Turning a result into cost requires the provider’s quote for the exact system (GPU count, interconnect, storage, support, term, egress) and a measured run of your workload. Compare dollars per completed unit of work at the latency or quality target you need.
Do not apply rates from a different system size or task and call it a ranking. Cost per result changes with utilization, batching, and contract terms. Ask for itemized quotes and normalize them by measured throughput.
How We Use These Numbers in Sourcing
MLPerf is one input. We use it to identify which tasks a platform has published results for, to confirm that the task, scenario, and system scale are comparable, and to sanity-check a provider’s claimed throughput.
We do not treat MLPerf as a winner-take-all ranking, and we do not claim a customer-specific cost analysis without that customer’s workload data. The buying decision comes from a matched result plus a trial on the customer’s workload. Where a quote lacks a matching configuration, ask for the benchmark details or run your own.