Latency Loom
Loading hourly results…
| Loading results… | |||||||||||||
E2E is the median wall-clock time to complete one fixed no-tool task through each vendor's subscription CLI — ready session to final result, process startup excluded, thinking time included. Every valid run must reproduce the same fixed passage byte-for-byte (SHA-verified), and runs that touch tools or web search, hit an API retry, or get served by a different model are dropped from the timer, so cross-vendor seconds time identical visible work; correctness beyond the exact-output match is not graded. Claude Code's measured window also includes the CLI's small concurrent Haiku utility call and result wrap-up (sub-second), which the Codex turn boundary does not have. Detail columns add stream rates and the retired agentic-suite metrics: token units are vendor-specific (this passage is 1,692 Claude tokens but 1,026 GPT tokens), so tokens-per-second ranks models only within one vendor, and OpenAI's subscription stream is delivery-paced (~55 tok/s normal, ~84 fast) — transport policy, not decoder speed — while Claude Code passes model speed through. Agentic E2E and Passes come from the earlier hidden-verifier repair-task protocol, which timed different tool-using work per model.
What a step of reasoning costs
Median no-tool completion seconds at each reasoning effort. The first card compares every model in normal mode; each fast-capable model then gets its own fast-versus-normal card (green is fast, grey is normal). Each card owns its seconds scale.