Qwen 3.5 397B-A17B — GB200 NVL72 vs H100
Head-to-head AI inference benchmark comparison of GB200 NVL72 (NVIDIA Blackwell) and H100 (NVIDIA Hopper) on Qwen 3.5 397B-A17B. Latency, throughput, and cost across LLM workloads. Use the chart controls below to switch sequences, precisions, and metrics — same interactions as the main inference chart.
Throughput at 64 tok/s/user on Qwen 3.5 397B-A17B: GB200 NVL72 hits 1239 tok/s/GPU, H100 hits 433. Per-million costs land at $0.51 and $0.85 respectively. GB200 NVL72 is 68% cheaper per token; GB200 NVL72 delivers 186% more tok/s/GPU.
GB200 NVL72 / H100 on Qwen 3.5 397B-A17B at 100 tok/s/user: 472 / 311 tok/s/GPU, $1.27 / $1.15 per million tokens. H100 is 11% cheaper per token; GB200 NVL72 delivers 52% more tok/s/GPU.
Toward the upper edge of the 29–171 tok/s/user interactivity band, at 135 tok/s/user on Qwen 3.5 397B-A17B: GB200 NVL72 runs 217 tok/s/GPU at $2.84/M tokens, H100 runs 236 at $1.54/M. H100 is 85% cheaper per token; H100 delivers 9% more tok/s/GPU. (Numbers reflect the default 1k/1k · fp8 selection for this URL — table and chart below update if you change sequence, precision, or model in the controls.)
| Metric | Interactivity (tok/s/user) | Interactivity (tok/s/user) | Interactivity (tok/s/user) |
|---|---|---|---|
| Throughput (tok/s/gpu) | GB200 NVL72:1239.3H100:433.3 | GB200 NVL72:472.0H100:310.5 | GB200 NVL72:216.7H100:236.1 |
| Cost ($/M tok) | GB200 NVL72:$0.506H100:$0.848 | GB200 NVL72:$1.272H100:$1.151 | GB200 NVL72:$2.838H100:$1.536 |
| tok/s/MW | GB200 NVL72:662711H100:316265 | GB200 NVL72:252382H100:226676 | GB200 NVL72:115883H100:172330 |
| Concurrency | GB200 NVL72:~113H100:~28 | GB200 NVL72:~20H100:~13 | GB200 NVL72:~7H100:~7 |
Inference Performance
Inference performance metrics across different models, hardware configurations, and serving parameters.