In the latest MLPerf Inference V5.0 benchmarks, NVIDIA´s Blackwell platform has set new records, marking a significant achievement in artificial intelligence inference capabilities. For the first time, NVIDIA used its GB200 NVL72 system, a rack-scale solution designed for AI reasoning, for the submission. This system effectively connects 72 NVIDIA Blackwell GPUs into a single massive GPU, achieving up to 30x higher throughput on the Llama 3.1 405B benchmark compared to previous submissions.
Designed for AI factories, the NVIDIA Blackwell platform demonstrates the future of data processing by transforming raw data into real-time insights. The emphasis is on delivering accurate and swift responses to queries at minimal costs to numerous users. Innovations across technology stacks in silicon, network systems, and software are pushing the boundaries of AI capabilities, enabling smarter models with billions of parameters to efficiently deliver insights in real-time, while managing costs and computing resources.
Apart from Blackwell, the NVIDIA Hopper platform also displayed exceptional performance, significantly improving throughput on the Llama 2 70B benchmark due to full-stack optimizations. This ongoing enhancement in NVIDIA´s platforms underscores the sustained value and adaptability of its AI solutions amidst growing model complexities and demand for responsive user experiences. The inclusive participation of several partners and rigorous peer-reviewed benchmarking further highlights the comprehensive reach and influence of NVIDIA´s evolving AI technologies.
