B300 FP4: MTP vs Off Speculative Decoding

MTP posts 5480 tok/s/chip for $0.11 per million tokens at 36 tok/s/user on DeepSeekv4 Pro 0813 1.6T (B300 FP4); Off posts 1127 tok/s/chip for $0.56. MTP is 386% cheaper per token; MTP delivers 386% more tok/s/chip. Draft-token acceptance rates determine whether speculative decoding helps or hurts at a given concurrency level.

Throughput at 59 tok/s/user on DeepSeekv4 Pro 0813 1.6T (B300 FP4): MTP hits 838 tok/s/chip, Off hits 520. Per-million costs land at $0.75 and $1.21 respectively. MTP is 61% cheaper per token; MTP delivers 61% more tok/s/chip. Speculative decoding trades extra compute on draft tokens for fewer decoding steps — the payoff depends on sequence length and batch size.

Toward the upper edge of the 13–105 tok/s/user interactivity band, at 82 tok/s/user on DeepSeekv4 Pro 0813 1.6T (B300 FP4): MTP runs 558 tok/s/chip at $1.13/M tokens, Off runs 154 at $4.07/M. MTP is 262% cheaper per token; MTP delivers 262% more tok/s/chip. Gains from speculative decoding vary by workload; short-output prompts tend to benefit less. (Numbers reflect this URL's pinned 1k/1k · fp4 workload — changing sequence or model updates both the table and chart; the table stays pinned to this page's precision, so precision toggles in the controls affect the chart only.)