Verified Codex subscription end-to-end completion time, task passes, direct-API output tokens per second, and subscription-CLI stream tokens per second by model, reasoning effort, and mode
Loading results…

E2E is the median verified completion time of the same three hidden-verifier repair tasks, measured through each vendor's subscription CLI (Codex app-server turn; Claude Code ready-session to final result; process startup, workspace setup, and verification excluded). Passes count only attempts that survive hidden verification; "no verified passes" marks configurations that were measured but where no attempt survived it, unlike unmeasured cells, which stay blank. API TPS is a direct paid-API stream estimate. Subscription TPS is the client-observed stream through the subscription CLI — OpenAI meters delivery at a fixed per-token cadence (~55 tok/s normal, ~84 fast), so it reflects transport policy rather than decoder speed, while Claude Code passes model speed through. Token units are vendor-specific: this fixed passage tokenizes to 1,692 Claude tokens but 1,026 GPT tokens, so cross-vendor throughput is best compared in seconds (E2E) or bytes, not tokens.