Agentic Inference Costs

Hyperscaler cost ↓ Lower is better Source: InferenceX & SemiAnalysis Market July 2026 Pricing Surveys & AI Cloud TCO Model

Current cost and change versus the latest validated platform result 30–60 days earlier.
Model · Scenario	B200	MI355X	B300	GB200 NVL72	GB300 NVL72
DeepSeek V4 Pro 0813 1.6T Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	$0.02453% cheaper than this platform’s Aug 8 result vLLM · FP4 Compare curves	$0.03172% cheaper than this platform’s Aug 8 result SGLang · FP4 Compare curves	$0.01945% cheaper than this platform’s Aug 7 result vLLM · FP4 Compare curves	$0.046 llm-d vLLM · FP4	$0.011 Dynamo vLLM · FP4
Kimi K3 2.8T Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	$0.10592% cheaper than this platform’s Jul 29 result Dynamo vLLM · FP4 Compare curves	$0.084 vLLM · FP4	$0.103 vLLM · FP4	$0.11379% cheaper than this platform’s Aug 7 result Dynamo vLLM · FP4 Compare curves	$0.080 Dynamo vLLM · FP4
MiniMax M3 428B Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	— no exact @50 result	$0.014 vLLM · FP4	— no exact @50 result	— no exact @50 result	— no exact @50 result
GLM5.2/GLM5.3 744B Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	— no exact @50 result	— no exact @50 result	— no exact @50 result	— no exact @50 result	— no exact @50 result
Qwen3.5 397B Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	— no exact @50 result	$0.008 SGLang · FP4	$0.014 SGLang · FP8	— no exact @50 result	— no exact @50 result
Qwen3.8 Flash Next 176B Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	— no data for this scenario	— no data for this scenario	— no exact @50 result	— no data for this scenario	— no data for this scenario
DeepSeek V4 Pro 0813 1.6T 8K/1K	$0.112 TRTLLM · FP4	$0.159 ATOM¹ · FP4	$0.134 TRTLLM · FP4	$0.514 Dynamo vLLM · FP4	$0.213 Dynamo SGLang · FP4 · STP
Qwen3.5 397B 8K/1K	$0.036 TRTLLM · FP4	$0.05617% cheaper than this platform’s Aug 6 result SGLang · FP4 Compare curves	$0.040 SGLang · FP4	$0.142 Dynamo SGLang · FP8 · STP	$0.04037% cheaper than this platform’s Aug 10 result Dynamo TRTLLM · FP4 · STP Compare curves
DeepSeek R1 0528 671BMaintenanceModel is no longer actively benchmarked. 8K/1K	$0.077 TRTLLM · FP4	$0.157 ATOM¹ · FP4	$0.139 Dynamo TRTLLM · FP4	$0.192 Dynamo SGLang · FP8 · STP	$0.230 Dynamo TRTLLM · FP8 · STP
Kimi K2.5/2.6/2.7-Code 1TDeprecatedModel is no longer actively benchmarked. 8K/1K	$0.084 Dynamo TRTLLM · FP4 · STP	$0.132 ATOM¹ · FP4 · STP	$0.179 vLLM · FP4 · STP	$0.079 Dynamo TRTLLM · FP4 · STP	$0.098 Dynamo TRTLLM · FP4 · STP
GLM5/5.1 744BDeprecatedModel is no longer actively benchmarked. 8K/1K	$0.172 SGLang · FP4	$0.431 SGLang · FP8	$0.210 SGLang · FP4	$0.260 Dynamo TRTLLM · FP4 · STP	$0.281 Dynamo TRTLLM · FP4 · STP
gpt-oss 120BDeprecatedModel is no longer actively benchmarked. 8K/1K	$0.010 TRTLLM · FP4 · STP	$0.010 ATOM¹ · FP4 · STP	— no data for this scenario	— no exact @50 result	— no data for this scenario
MiniMax M2.5/2.7 230BDeprecatedModel is no longer actively benchmarked. 8K/1K	$0.041 vLLM · FP4 · STP	$0.051 ATOM¹ · FP4 · STP	$0.051 vLLM · FP4 · STP	$0.041 Dynamo vLLM · FP4 · STP	$0.047 Dynamo vLLM · FP4 · STP
Llama 3.3 70B InstructDeprecatedModel is no longer actively benchmarked. 8K/1K	$0.065 TRTLLM · FP4 · STP	$0.145 vLLM · FP4 · STP	— no data for this scenario	— no data for this scenario	— no data for this scenario

DeepSeek V4 Pro 0813 1.6T
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
$0.02453% cheaper than this platform’s Aug 8 result
vLLM · FP4
Compare curves
MI355X
$0.03172% cheaper than this platform’s Aug 8 result
SGLang · FP4
Compare curves
B300
$0.01945% cheaper than this platform’s Aug 7 result
vLLM · FP4
Compare curves
GB200 NVL72
$0.046
llm-d vLLM · FP4
GB300 NVL72
$0.011
Dynamo vLLM · FP4
Kimi K3 2.8T
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
$0.10592% cheaper than this platform’s Jul 29 result
Dynamo vLLM · FP4
Compare curves
MI355X
$0.084
vLLM · FP4
B300
$0.103
vLLM · FP4
GB200 NVL72
$0.11379% cheaper than this platform’s Aug 7 result
Dynamo vLLM · FP4
Compare curves
GB300 NVL72
$0.080
Dynamo vLLM · FP4
MiniMax M3 428B
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
—
no exact @50 result
MI355X
$0.014
vLLM · FP4
B300
—
no exact @50 result
GB200 NVL72
—
no exact @50 result
GB300 NVL72
—
no exact @50 result
GLM5.2/GLM5.3 744B
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
—
no exact @50 result
MI355X
—
no exact @50 result
B300
—
no exact @50 result
GB200 NVL72
—
no exact @50 result
GB300 NVL72
—
no exact @50 result
Qwen3.5 397B
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
—
no exact @50 result
MI355X
$0.008
SGLang · FP4
B300
$0.014
SGLang · FP8
GB200 NVL72
—
no exact @50 result
GB300 NVL72
—
no exact @50 result
Qwen3.8 Flash Next 176B
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
—
no data for this scenario
MI355X
—
no data for this scenario
B300
—
no exact @50 result
GB200 NVL72
—
no data for this scenario
GB300 NVL72
—
no data for this scenario
DeepSeek V4 Pro 0813 1.6T
8K/1K
B200
$0.112
TRTLLM · FP4
MI355X
$0.159
ATOM¹ · FP4
B300
$0.134
TRTLLM · FP4
GB200 NVL72
$0.514
Dynamo vLLM · FP4
GB300 NVL72
$0.213
Dynamo SGLang · FP4STP
Qwen3.5 397B
8K/1K
B200
$0.036
TRTLLM · FP4
MI355X
$0.05617% cheaper than this platform’s Aug 6 result
SGLang · FP4
Compare curves
B300
$0.040
SGLang · FP4
GB200 NVL72
$0.142
Dynamo SGLang · FP8STP
GB300 NVL72
$0.04037% cheaper than this platform’s Aug 10 result
Dynamo TRTLLM · FP4STP
Compare curves
DeepSeek R1 0528 671BMaintenanceModel is no longer actively benchmarked.
8K/1K
B200
$0.077
TRTLLM · FP4
MI355X
$0.157
ATOM¹ · FP4
B300
$0.139
Dynamo TRTLLM · FP4
GB200 NVL72
$0.192
Dynamo SGLang · FP8STP
GB300 NVL72
$0.230
Dynamo TRTLLM · FP8STP
Kimi K2.5/2.6/2.7-Code 1TDeprecatedModel is no longer actively benchmarked.
8K/1K
B200
$0.084
Dynamo TRTLLM · FP4STP
MI355X
$0.132
ATOM¹ · FP4STP
B300
$0.179
vLLM · FP4STP
GB200 NVL72
$0.079
Dynamo TRTLLM · FP4STP
GB300 NVL72
$0.098
Dynamo TRTLLM · FP4STP
GLM5/5.1 744BDeprecatedModel is no longer actively benchmarked.
8K/1K
B200
$0.172
SGLang · FP4
MI355X
$0.431
SGLang · FP8
B300
$0.210
SGLang · FP4
GB200 NVL72
$0.260
Dynamo TRTLLM · FP4STP
GB300 NVL72
$0.281
Dynamo TRTLLM · FP4STP
gpt-oss 120BDeprecatedModel is no longer actively benchmarked.
8K/1K
B200
$0.010
TRTLLM · FP4STP
MI355X
$0.010
ATOM¹ · FP4STP
B300
—
no data for this scenario
GB200 NVL72
—
no exact @50 result
GB300 NVL72
—
no data for this scenario
MiniMax M2.5/2.7 230BDeprecatedModel is no longer actively benchmarked.
8K/1K
B200
$0.041
vLLM · FP4STP
MI355X
$0.051
ATOM¹ · FP4STP
B300
$0.051
vLLM · FP4STP
GB200 NVL72
$0.041
Dynamo vLLM · FP4STP
GB300 NVL72
$0.047
Dynamo vLLM · FP4STP
Llama 3.3 70B InstructDeprecatedModel is no longer actively benchmarked.
8K/1K
B200
$0.065
TRTLLM · FP4STP
MI355X
$0.145
vLLM · FP4STP
B300
—
no data for this scenario
GB200 NVL72
—
no data for this scenario
GB300 NVL72
—
no data for this scenario

Current cost and change versus the latest validated platform result 30–60 days earlier.

Platforms without a valid 30-day comparison show current cost only.

If a chip does not have FP4 spec decoding available, the next best available configuration is used.

DeepSeek V4 Pro 0813 1.6T

Kimi K3 2.8T

MiniMax M3 428B

GLM5.2/GLM5.3 744B

Qwen3.5 397B

Qwen3.8 Flash Next 176B

DeepSeek V4 Pro 0813 1.6T

Qwen3.5 397B

DeepSeek R1 0528 671BMaintenanceModel is no longer actively benchmarked.

Kimi K2.5/2.6/2.7-Code 1TDeprecatedModel is no longer actively benchmarked.

GLM5/5.1 744BDeprecatedModel is no longer actively benchmarked.

gpt-oss 120BDeprecatedModel is no longer actively benchmarked.

MiniMax M2.5/2.7 230BDeprecatedModel is no longer actively benchmarked.

Llama 3.3 70B InstructDeprecatedModel is no longer actively benchmarked.

DeepSeek V4 Pro 0813 1.6T

Kimi K3 2.8T

MiniMax M3 428B

GLM5.2/GLM5.3 744B

Qwen3.5 397B

Qwen3.8 Flash Next 176B

DeepSeek V4 Pro 0813 1.6T

Qwen3.5 397B

DeepSeek R1 0528 671BMaintenanceModel is no longer actively benchmarked.

Kimi K2.5/2.6/2.7-Code 1TDeprecatedModel is no longer actively benchmarked.

GLM5/5.1 744BDeprecatedModel is no longer actively benchmarked.

gpt-oss 120BDeprecatedModel is no longer actively benchmarked.

MiniMax M2.5/2.7 230BDeprecatedModel is no longer actively benchmarked.

Llama 3.3 70B InstructDeprecatedModel is no longer actively benchmarked.