Are we missing a benchmark for agent runtimes, not just models?

We have SWE-bench, Terminal-Bench, OSWorld, BrowseComp, etc. But I haven’t seen a good apples-to-apples benchmark for platforms like OpenAI Agents, Anthropic’s agent stack, AWS AgentCore, Google’s agent platform, and local-alternatives like LangGraph, etc.

What I’d want measured:
- task success rate
- cost per successful task
- wall-clock time
- tool calls / retries
- reliability over long-running tasks
- (optionally) human interventions required

The really interesting experiment would control for both sides:
- Same model, different harness → how much does the runtime matter?
- Same harness, different model → how much does the model matter?

Feels like we’re increasingly evaluating “model + harness” systems, but our benchmarks still mostly treat the model as the unit of comparison.
Does something like this already exist?