Trace replay
Also known as workload replay, session replay
In plain English
Trace replay recreates the timing and shape of recorded sessions so a benchmark sends requests like the original workload.
Technical definition
Trace replay is a benchmarking method that converts recorded request relationships, lengths, and timing into a repeatable workload for a system under test.
Engineering details
A replay can preserve a directed graph of main-agent turns, parallel subagent branches, and auxiliary requests. Deterministic synthetic tokens replace private content while retaining token counts and prefix relationships. Recorded gaps between turns reproduce the periods when an agent was using tools or waiting on dependencies.
Why it matters
The method captures traffic features that a list of independent prompts cannot express. It also makes repeated hardware and software comparisons possible from the same session shapes. AgentX removes source-conversation content before publishing replay data.
How to read it in InferenceX
AgentX replays trace-derived sessions through AIPerf. A fixed seed selects sessions, starting points, and synthetic content. Reported results cover the profiling window after cache warmup, which keeps run-to-run comparisons focused on steady-state serving behavior.
Source material
See the concept in real benchmarks
A Brief Overview of Agentic Workloads
Multi-turn sessions, long contexts, and near-total prefix reuse make agentic inference a systems problem, and change what a benchmark has to measure
Agentic Benchmark for LLM Inference: Metrics and Methodology
How an agent benchmark replays long-context, multi-turn workloads to measure latency, throughput, cache behavior, and serving cost
Ultra-High Interactivity on NVIDIA GPUs? TileRT on InferenceX
Can TileRT software on NVIDIA GPUs compete with Cerebras, Groq LPU, and SambaNova? Batch size 1, disaggregated engine, high-throughput prefill engine, high-interactivity decode engine
Vera Rubin NVL72 vs GB200 NVL72? Inference TCO & Architecture Analysis
Rubin LUT Based Tensor Core, Feynman, Rack Scale, Perf Per MegaWatt, Perf Per Dollar, Software Improvements, Public Rubin Software, PyTorch, vLLM, OpenAI Triton
AgentX - InferenceXv3: Does the CUDA Moat Hold Up in Agentic Inferencing?
$3 Million USD dataset open sourced, 1 Mil+ Context Length, Multiturn, Sub Agents 95%+ KVCache HitRate, GB300 NVL72, MI355X, B200