Query latency

Query v3 returns timing on every successful query, including empty results. timing.total_ms is Query processing time. The stages array explains where that time went, with start offsets and durations that form a waterfall.

Use it to see whether a slower query spent its time creating an embedding, retrieving candidates, reranking them, or loading context. Compare the same query with different settings, then check whether the extra work improves the results.

The interactive example uses illustrative data, not a benchmark or live API call. Its controls demonstrate which operations a setting can affect. Hover or focus a trace row for its start, duration, and end.

Read the waterfall

Each stage has a stable public name, start_ms, and duration_ms. All offsets share one monotonic clock origin. Stages are ordered by start offset, with each name appearing once. Repeated work uses segments, and operations can overlap. New operation names may be added; clients should display an unfamiliar name rather than reject the response.

Only operations that ran are returned. Reranking is omitted when disabled or when there are no candidates. Asset signing is omitted when no eligible crop URL is signed. No artificial zero-duration bars stand in for skipped work. A recorded operation can still have duration_ms: 0 when its start and end fall within the same millisecond.

The waterfall reports the instrumented public operations above. It does not claim to provide separate timers for every retrieval leg, every metadata fetch, or each attempt within a retry loop. Uninstrumented orchestration can appear as gaps.

Understand overlap and retries

Each stage name appears once. start_ms is its first start and duration_ms spans through its last finish. When the stage runs more than once, segments contains its actual intervals. Draw those intervals instead of filling the gaps between them. For example, preparation at 0–10 ms and 80–90 ms has a 90 ms extent, with two 10 ms segments.

total_ms is measured from the shared origin to completion. It is never calculated by adding stage durations. Parallel work counts once in the total.

Asset signing currently runs inside hydration. Its interval identifies the signing work within that enclosing interval; it is not extra time after hydration. Repeated signing intervals correspond to actual signing calls. Schema preparation can run at the same time as embedding. A waterfall preserves these relationships instead of forcing operations into a sequence.

Retries, backoff, and fallback remain inside the operation that caused them. A reranker that retries and then falls back still reports that full elapsed interval as reranking. Read the existing rerank.used, rerank.model, rerank.candidate_limit, rerank.fallback, and warnings fields alongside timing to understand the outcome. A candidate limit is a configured limit, not an invented count of candidates actually ranked.

Measurement boundary

Processing starts when the Query v3 handler begins, before authentication, and ends after the response model is constructed. Integer millisecond offsets and endpoints are rounded down from the same monotonic origin. Stage duration is the difference between those endpoints.

The total excludes gateway routing, request-body parsing before the handler, JSON serialization, compression, network transfer, client decoding, and background usage or query-history writes. No diagnostics flag is required.

Measure Round-trip time in the caller, through response-body decoding. If displaying the difference from query processing time, label it Outside measured processing. It includes server work outside the boundary as well as transport and client work; it is not network latency.

const started = performance.now();
const response = await fetch(
  "https://api.captain.dev/v3/collections/example/query",
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${apiKey}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ query: "invoice approval policy", limit: 10 }),
  },
);
if (!response.ok) throw new Error(`Query failed: ${response.status}`);
const data = await response.json();
const roundTripMs = performance.now() - started;
const processingMs = data.timing?.total_ms ?? data.execution_time_ms;
console.log({ processingMs, roundTripMs });

Summarize latency across many queries

The timing object explains one query. When the question is about a period rather than a request, GET /v2/queries/latency answers it in a single call: exact p50, p95 and p99 over every query that matched your filters, a histogram with its bin edges, and the same bins for each collection so you can compare their shapes.

This is not a page of records. GET /v2/queries lists individual queries a hundred at a time, which means summarizing a busy month would take hundreds of requests and leave the statistics up to you. The latency endpoint reads the whole filtered population and returns only the summary, so the response size depends on the number of bins rather than the number of queries.

curl "https://api.captain.dev/v2/queries/latency?from=2026-09-01&to=2026-09-08" \
  -H "Authorization: Bearer $CAPTAIN_API_KEY"

from is inclusive and to is exclusive, so consecutive windows neither overlap nor leave a gap. Windows can cover up to 90 days.

Three parts of the response are worth reading carefully, because each one keeps a summary from implying something it cannot support.

Measured and missing are separate counts. counts.measured are the queries with a recorded processing time and counts.missing are the ones without. A query whose timing was never recorded is left out of the percentiles rather than counted as zero milliseconds, which would pull the median toward zero and make a healthy period look fast.

An unrecorded setting reads as unknown, not as off. You can filter by collection, environment, API version, the rerank setting, the search mode and whether the request carried a metadata filter. Older queries predate some of those fields, so each filter also accepts unknown for the queries that never recorded it, and the coverage object tells you what share of the population recorded each one. When coverage is low, most of your history will answer unknown, and that is a description of the records rather than a description of your requests. A recorded rerank setting also means the request asked for reranking, not that reranking necessarily ran.

Recent windows say when they are still filling. Timing is recorded just after a query finishes, so a window that reaches up to the present moment can still be gaining rows. completeness.complete is false while that is true, and completeness.watermark tells you how far the data currently reaches.

Two companion endpoints cover the cases a single summary cannot. GET /v2/queries/latency/collections searches and pages through the collections that ran queries in the window, which helps when an organization has more collections than the summary returns series for. GET /v2/queries/latency/slowest returns the slowest queries with their durations and identifiers, and each identifier can be passed to GET /v2/queries/{query_id} to read that query in full.

The metric covers completed queries. A query that failed records no usage row, and the queries an evaluation runs are excluded so that evaluation traffic never moves your production numbers.

Inspect queries in Studio

Open Activity, select a query in the recent-query feed, and read the waterfall in its query investigator. Model, candidate limit, fallback, and warning facts remain alongside it when recorded. The response and query history contain the identical timing object.

Playground uses the same waterfall for query results and displays the measured round trip separately. Compare rerank models or candidate limits to investigate ranking work. Compare include controls to investigate context hydration and signing. Evaluate result quality alongside latency rather than optimizing one response in isolation.

• File Search API
• Query v3 request and response schema

Older responses and history

execution_time_ms remains an exact compatibility alias for timing.total_ms. Existing clients can continue reading it.

Older records may have only execution_time_ms, or a coarse timing object with preparation_ms, search_ms, and hydration_ms. Display the available total and coarse durations. Those records contain no real stage offsets, so a client must not reconstruct a waterfall by guessing sequential start times. Missing timing stays unavailable, not zero.

Earlier stage records can contain duplicate names. Group their recorded intervals for display, preserving gaps and overlaps. An embedding interval absent from the original measurement remains unavailable.