Agent Usability - API, MCP & CLI Performance Benchmarks
Benchmark your API, MCP, or CLI across coding agents
Get LLM-specific scores for discoverability, tool calls, and errors. Real sandbox execution across Claude Code, Cursor, Codex and 13 agents, not just prompt analysis.
Run your first eval See Devtool Arena
Benchmark your API, MCP, or CLI across all major coding agents
Claude Code
Cursor
Codex
OpenCode
Hermes
Muse Code
+6 more
API Performance
Track how your API performs with AI coding agents
Run Evals
Coding Agent Leaderboard
Which coding agents work best with your API
| Agent / Average | Payments | Billing | Checkout | Connect | Webhooks |
|---|---|---|---|---|---|
| 1 Claude Code94% |
97% | 96% | 92% | 90% | 95% |
| 2 Codex87% |
91% | 89% | 85% | 81% | 88% |
| 3 Cursor76% |
82% | 78% | 72% | 68% | 80% |
| 4 GitHub Copilot71% |
78% | 74% | 70% | 61% | 72% |
| 5 Hermes63% |
71% | 65% | 58% | 56% | 63% |
| 6 OpenCode58% |
66% | 60% | 54% | 51% | 59% |
Endpoint Evaluations
How each endpoint performs across eval types
Payments API(5 endpoints)
82% pass rate
- POST
/v1/payment_intents- 94%
- POST
/v1/charges- 88%
13 coding agents benchmarked
3 score dimensions
Every endpoint evaluated
Visibility ≠ Usability
Consumer AI visibility tools track whether ChatGPT mentions your brand. But for APIs, MCP servers, and CLIs, a mention means nothing if the generated code doesn't compile.
When Claude Code recommends your tool but the code fails, developers switch to a competitor. You need to measure what actually matters: can agents discover, call, and use your endpoints and tools?
Agent Usability benchmarks your API, MCP, or CLI with real sandbox execution. Get scores for discoverability, tool calls, and error rates broken down by agent and endpoint.
Endpoint-level benchmarks
Every endpoint. Every agent. Every metric.
Real Sandbox Execution
Spin up actual environments and run your API, MCP server, or CLI against real coding agents. This is actual code generation and execution with pass/fail results, not prompt analysis.
Custom Eval Scenarios
Write your own eval prompts. Test specific endpoints, MCP tools, CLI commands, auth flows, and edge cases. Define what success looks like.
Error & Failure Detection
See exactly where agents fail: 404s, auth errors, malformed requests, missing docs. Get the specific URLs and step-by-step traces to debug.
LLM-Specific Scores
Get separate scores for discoverability, tool calls, and error rates. Each endpoint, agent, and metric is broken down so you know what to fix.
Competitor Benchmarks
See how your API, MCP, or CLI ranks against competitors on Devtool Arena. Track your position over time and identify gaps.
12 Coding Agents
Claude Code, Cursor, Codex, GitHub Copilot, Antigravity, and more. Each agent behaves differently, so know your compatibility scores across all of them.
Free & Public
Devtool Arena
See how your API ranks against competitors on our free public leaderboard. Based on real benchmarks across discoverability, tool calls, and error rates.
#1Stripe
94%
#2Twilio
91%
#3OpenAI
87%
Is your API agent-ready?
Find out how well AI coding agents can use your API. Get specific recommendations to improve.