Setting a World Record for MoE Pre-Training on NVIDIA GB300 NVL72

Frontier model pre-training has converged on mixture of experts (MoE), which is fundamentally changing what limits large-scale AI training. As compute per token...

Jul 22, 2026 - 00:04

Setting a World Record for MoE Pre-Training on NVIDIA GB300 NVL72

Frontier model pre-training has converged on mixture of experts (MoE), which is fundamentally changing what limits large-scale AI training. As compute per token... Decorative image.

Frontier model pre-training has converged on mixture of experts (MoE), which is fundamentally changing what limits large-scale AI training. As compute per token falls, communication increasingly determines how efficiently models scale across thousands of GPUs. NVIDIA GB300 NVL72 set a world record for pre-training DeepSeek-V3 671B at 1,648 TFLOPs per GPU, showing how advances across the entire AI…

Source

Tags:

Первый легендарный скакун «Красный Заяц»! Не более 4 на сервер

What's Your Reaction?

Dislike

Love

Funny

Angry

Sad

Wow

XINKER - Business and Income Tips Explore XINKER, the ultimate platform for mastering business strategies, discovering passive income opportunities, and learning success principles. Join a community of thinkers dedicated to achieving financial freedom and entrepreneurial excellence.