> For a complete page index of the Captain API documentation, fetch https://docs.captain.dev/llms.txt?excludeSpec=true

# Evaluation Tools over MCP

* **Run an eval over MCP**: the hosted [MCP server](/guides/mcp) adds `captain_create_eval_upload`, `captain_run_eval` and `captain_get_eval_results`, the three steps of the [Evaluation API](/guides/evaluations): upload up to 10,000 questions, queue them under one to eight named [v3 query](/reference/v3/query) configurations, poll for scorecards. They need an API-key connection, since the Evaluation API accepts API keys only in v1.
* **Retries replay instead of rerunning**: `captain_run_eval` derives an idempotency key from the collection, upload and configurations, so a repeated call returns the original eval instead of billing a second one.
* **Results sized for a conversation**: `captain_get_eval_results` returns status, progress, per-configuration scorecards, billing and a compact row per case (hit, rank, latency, `query_id`, error code), paged by cursor; `include_answers` adds the stored request and top results for up to 20 cases. `captain_eval` stays for quick client-side comparisons. The hosted server is now at 70 tools.