Agentic Inference Costs

Hyperscaler cost ↓ Lower is better Source: InferenceX & SemiAnalysis Market July 2026 Pricing Surveys & AI Cloud TCO Model

Current cost and change versus the latest validated platform result 30–60 days earlier.
Model · Scenario	B200	MI355X	B300	GB200 NVL72	GB300 NVL72
DeepSeek V4 Pro 0813 1.6T Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	$0.02453% cheaper than this platform’s Aug 8 result vLLM · FP4 Compare curves	$0.03172% cheaper than this platform’s Aug 8 result SGLang · FP4 Compare curves	$0.01945% cheaper than this platform’s Aug 7 result vLLM · FP4 Compare curves	$0.046 llm-d vLLM · FP4	$0.011 Dynamo vLLM · FP4
Kimi K3 2.8T Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	$0.10592% cheaper than this platform’s Jul 29 result Dynamo vLLM · FP4 Compare curves	$0.084 vLLM · FP4	$0.103 vLLM · FP4	$0.11379% cheaper than this platform’s Aug 7 result Dynamo vLLM · FP4 Compare curves	$0.080 Dynamo vLLM · FP4
MiniMax M3 428B Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	— no exact @50 result	$0.014 vLLM · FP4	— no exact @50 result	— no exact @50 result	— no exact @50 result
GLM5.2/GLM5.3 744B Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	— no exact @50 result	— no exact @50 result	— no exact @50 result	— no exact @50 result	— no exact @50 result
Qwen3.5 397B Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	— no exact @50 result	$0.008 SGLang · FP4	$0.014 SGLang · FP8	— no exact @50 result	— no exact @50 result
Qwen3.8 Flash Next 176B Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	— no data for this scenario	— no data for this scenario	— no exact @50 result	— no data for this scenario	— no data for this scenario
DeepSeek V4 Pro 0813 1.6T 8K/1K	$0.300About the same cost as this platform’s Jul 12 result vLLM · FP4 Compare curves	$0.346About the same cost as this platform’s Jul 14 result MoRI SGLang · FP4 Compare curves	$0.245 SGLang · FP4	$0.514 Dynamo vLLM · FP4	$0.213 Dynamo SGLang · FP4 · STP
Qwen3.5 397B 8K/1K	$0.038About the same cost as this platform’s Aug 10 result SGLang · FP4 Compare curves	$0.05617% cheaper than this platform’s Aug 6 result SGLang · FP4 Compare curves	$0.040 SGLang · FP4	$0.142 Dynamo SGLang · FP8 · STP	$0.133 Dynamo SGLang · FP8 · STP
DeepSeek R1 0528 671BMaintenanceModel is no longer actively benchmarked. 8K/1K	$0.104 SGLang · FP4	$0.259 SGLang · FP8	$0.353 SGLang · FP8	$0.192 Dynamo SGLang · FP8 · STP	$0.263 Dynamo SGLang · FP8 · STP
Kimi K2.5/2.6/2.7-Code 1TDeprecatedModel is no longer actively benchmarked. 8K/1K	$0.139 Dynamo vLLM · FP4 · STP	$0.147 vLLM · FP4 · STP	$0.179 vLLM · FP4 · STP	$0.112 Dynamo vLLM · FP4 · STP	$0.115 Dynamo vLLM · FP4 · STP
GLM5/5.1 744BDeprecatedModel is no longer actively benchmarked. 8K/1K	$0.172 SGLang · FP4	$0.431 SGLang · FP8	$0.210 SGLang · FP4	$0.333 Dynamo SGLang · FP4 · STP	$0.433 Dynamo SGLang · FP4 · STP
gpt-oss 120BDeprecatedModel is no longer actively benchmarked. 8K/1K	$0.011 vLLM · FP4 · STP	$0.011 vLLM · FP4 · STP	— no data for this scenario	— no data for this scenario	— no data for this scenario
MiniMax M2.5/2.7 230BDeprecatedModel is no longer actively benchmarked. 8K/1K	$0.041 vLLM · FP4 · STP	$0.054 vLLM · FP4 · STP	$0.051 vLLM · FP4 · STP	$0.041 Dynamo vLLM · FP4 · STP	$0.047 Dynamo vLLM · FP4 · STP
Llama 3.3 70B InstructDeprecatedModel is no longer actively benchmarked. 8K/1K	$0.078 vLLM · FP4 · STP	$0.145 vLLM · FP4 · STP	— no data for this scenario	— no data for this scenario	— no data for this scenario

DeepSeek V4 Pro 0813 1.6T
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
$0.02453% cheaper than this platform’s Aug 8 result
vLLM · FP4
Compare curves
MI355X
$0.03172% cheaper than this platform’s Aug 8 result
SGLang · FP4
Compare curves
B300
$0.01945% cheaper than this platform’s Aug 7 result
vLLM · FP4
Compare curves
GB200 NVL72
$0.046
llm-d vLLM · FP4
GB300 NVL72
$0.011
Dynamo vLLM · FP4
Kimi K3 2.8T
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
$0.10592% cheaper than this platform’s Jul 29 result
Dynamo vLLM · FP4
Compare curves
MI355X
$0.084
vLLM · FP4
B300
$0.103
vLLM · FP4
GB200 NVL72
$0.11379% cheaper than this platform’s Aug 7 result
Dynamo vLLM · FP4
Compare curves
GB300 NVL72
$0.080
Dynamo vLLM · FP4
MiniMax M3 428B
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
—
no exact @50 result
MI355X
$0.014
vLLM · FP4
B300
—
no exact @50 result
GB200 NVL72
—
no exact @50 result
GB300 NVL72
—
no exact @50 result
GLM5.2/GLM5.3 744B
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
—
no exact @50 result
MI355X
—
no exact @50 result
B300
—
no exact @50 result
GB200 NVL72
—
no exact @50 result
GB300 NVL72
—
no exact @50 result
Qwen3.5 397B
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
—
no exact @50 result
MI355X
$0.008
SGLang · FP4
B300
$0.014
SGLang · FP8
GB200 NVL72
—
no exact @50 result
GB300 NVL72
—
no exact @50 result
Qwen3.8 Flash Next 176B
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
—
no data for this scenario
MI355X
—
no data for this scenario
B300
—
no exact @50 result
GB200 NVL72
—
no data for this scenario
GB300 NVL72
—
no data for this scenario
DeepSeek V4 Pro 0813 1.6T
8K/1K
B200
$0.300About the same cost as this platform’s Jul 12 result
vLLM · FP4
Compare curves
MI355X
$0.346About the same cost as this platform’s Jul 14 result
MoRI SGLang · FP4
Compare curves
B300
$0.245
SGLang · FP4
GB200 NVL72
$0.514
Dynamo vLLM · FP4
GB300 NVL72
$0.213
Dynamo SGLang · FP4STP
Qwen3.5 397B
8K/1K
B200
$0.038About the same cost as this platform’s Aug 10 result
SGLang · FP4
Compare curves
MI355X
$0.05617% cheaper than this platform’s Aug 6 result
SGLang · FP4
Compare curves
B300
$0.040
SGLang · FP4
GB200 NVL72
$0.142
Dynamo SGLang · FP8STP
GB300 NVL72
$0.133
Dynamo SGLang · FP8STP
DeepSeek R1 0528 671BMaintenanceModel is no longer actively benchmarked.
8K/1K
B200
$0.104
SGLang · FP4
MI355X
$0.259
SGLang · FP8
B300
$0.353
SGLang · FP8
GB200 NVL72
$0.192
Dynamo SGLang · FP8STP
GB300 NVL72
$0.263
Dynamo SGLang · FP8STP
Kimi K2.5/2.6/2.7-Code 1TDeprecatedModel is no longer actively benchmarked.
8K/1K
B200
$0.139
Dynamo vLLM · FP4STP
MI355X
$0.147
vLLM · FP4STP
B300
$0.179
vLLM · FP4STP
GB200 NVL72
$0.112
Dynamo vLLM · FP4STP
GB300 NVL72
$0.115
Dynamo vLLM · FP4STP
GLM5/5.1 744BDeprecatedModel is no longer actively benchmarked.
8K/1K
B200
$0.172
SGLang · FP4
MI355X
$0.431
SGLang · FP8
B300
$0.210
SGLang · FP4
GB200 NVL72
$0.333
Dynamo SGLang · FP4STP
GB300 NVL72
$0.433
Dynamo SGLang · FP4STP
gpt-oss 120BDeprecatedModel is no longer actively benchmarked.
8K/1K
B200
$0.011
vLLM · FP4STP
MI355X
$0.011
vLLM · FP4STP
B300
—
no data for this scenario
GB200 NVL72
—
no data for this scenario
GB300 NVL72
—
no data for this scenario
MiniMax M2.5/2.7 230BDeprecatedModel is no longer actively benchmarked.
8K/1K
B200
$0.041
vLLM · FP4STP
MI355X
$0.054
vLLM · FP4STP
B300
$0.051
vLLM · FP4STP
GB200 NVL72
$0.041
Dynamo vLLM · FP4STP
GB300 NVL72
$0.047
Dynamo vLLM · FP4STP
Llama 3.3 70B InstructDeprecatedModel is no longer actively benchmarked.
8K/1K
B200
$0.078
vLLM · FP4STP
MI355X
$0.145
vLLM · FP4STP
B300
—
no data for this scenario
GB200 NVL72
—
no data for this scenario
GB300 NVL72
—
no data for this scenario

Current cost and change versus the latest validated platform result 30–60 days earlier.

Platforms without a valid 30-day comparison show current cost only.

If a chip does not have FP4 spec decoding available, the next best available configuration is used.

DeepSeek V4 Pro 0813 1.6T

Kimi K3 2.8T

MiniMax M3 428B

GLM5.2/GLM5.3 744B

Qwen3.5 397B

Qwen3.8 Flash Next 176B

DeepSeek V4 Pro 0813 1.6T

Qwen3.5 397B

DeepSeek R1 0528 671BMaintenanceModel is no longer actively benchmarked.

Kimi K2.5/2.6/2.7-Code 1TDeprecatedModel is no longer actively benchmarked.

GLM5/5.1 744BDeprecatedModel is no longer actively benchmarked.

gpt-oss 120BDeprecatedModel is no longer actively benchmarked.

MiniMax M2.5/2.7 230BDeprecatedModel is no longer actively benchmarked.

Llama 3.3 70B InstructDeprecatedModel is no longer actively benchmarked.

DeepSeek V4 Pro 0813 1.6T

Kimi K3 2.8T

MiniMax M3 428B

GLM5.2/GLM5.3 744B

Qwen3.5 397B

Qwen3.8 Flash Next 176B

DeepSeek V4 Pro 0813 1.6T

Qwen3.5 397B

DeepSeek R1 0528 671BMaintenanceModel is no longer actively benchmarked.

Kimi K2.5/2.6/2.7-Code 1TDeprecatedModel is no longer actively benchmarked.

GLM5/5.1 744BDeprecatedModel is no longer actively benchmarked.

gpt-oss 120BDeprecatedModel is no longer actively benchmarked.

MiniMax M2.5/2.7 230BDeprecatedModel is no longer actively benchmarked.

Llama 3.3 70B InstructDeprecatedModel is no longer actively benchmarked.