The Blackwell Benchmark: Scaling the Frontier of AI Training

The speed of AI development is dictated by the efficiency of the training run. As models grow in complexity, the infrastructure supporting them determines whether a team can iterate or if they will stall under the weight of their own compute requirements.

In the latest MLPerf Training 6.0 benchmarks, the NVIDIA Blackwell platform demonstrated dominance across every category. The results show the platform achieved the fastest time to train on every benchmark and was the only platform with submissions across all seven benchmarks in the suite. This performance is not merely about raw speed; it is about the ability to manage massive-scale training, including operations across 8,192 GPUs using NVIDIA Blackwell NVL72 systems.

The shift toward mixture-of-experts (MoE) architectures is now a central reality of AI development. MLPerf Training 6.0 introduced two new MoE pretraining workloads: DeepSeek-V3 671B and GPT-OSS-20B. Managing these architectures requires solving the all-to-all communication challenge, where tokens must be routed across GPUs to reach the correct expert subnetwork. NVIDIA uses fifth-generation NVLink Switches to connect all 72 GPUs within rack-scale systems into a unified pool of compute and memory, allowing them to function as one giant GPU.

The hardware evolution is moving toward higher density and specialized precision. NVIDIA submitted results for both NVIDIA GB200 NVL72 and GB300 NVL72 rack-scale systems. The GB300 NVL72 delivered up to 1.6x faster training than the GB200 NVL72 at the same scale. Furthermore, the use of NVFP4 training methods is increasing performance while maintaining strict accuracy requirements. This low-precision innovation was recently used to pretrain the 550-billion-parameter NVIDIA Nemotron 3 Ultra model.

For model builders, the implication is clear: the gap between those who can afford to train at scale and those who cannot is widening. The ability to minimize training costs and launch frontier models faster is becoming a function of hardware architecture and interconnect bandwidth. As the industry moves toward larger MoE models, the bottleneck will shift from raw compute to the efficiency of the network connecting that compute.

The question for the next cycle of AI development is whether software optimization can keep pace with the rapid scaling of Blackwell-class infrastructure.