Connections and retries

Every request to a tenant endpoint, for example https://acme.captain.dev, goes through Captain’s edge network. The edge keeps connections open between requests, and it can also close one at any time: when the connection has been idle too long, or when the edge moves traffic between its servers. A connection closed that way while a request is on it fails before any part of the answer arrives. The request never reached Captain, and sending it again on a new connection is the fix.

The rules

1. Reuse connections. A new connection costs a TCP and TLS handshake, one or two extra round trips. Keep connections open between requests, and expect the edge to close any of them at any time.
2. Retry once when a reused connection fails before any answer. The signs are an end of stream before the status line, a connection reset, or a broken pipe while the request is still being sent. Send the request once more, on a new connection, with no delay.
3. Reads are always safe to send twice. Search, typeahead, browse, similar, multi search, the Algolia query routes (/1/indexes/*/queries and /1/indexes/{index}/query) and every GET change nothing. Most of them are POST requests, so many HTTP clients won’t retry them on their own. The examples below add the retry.
4. Retry a write only with an Idempotency-Key. A write that fails in transit may already have been applied. Send it again with the same Idempotency-Key header, and Captain returns the first reply instead of applying it twice. See Retries in Records.
5. Close idle connections after about 5 seconds. Under load, the edge has closed client connections that sat idle for as little as 8 seconds. A client that drops its own idle connections sooner rarely sends a request onto one the edge is closing. The retry in rule 2 covers the rest.

Retry only these connection failures. A request that timed out waiting for its answer did reach Captain, and sending it again adds load where there is none to spare. An answer with a status code, such as 503 with Retry-After, is Captain’s reply: follow its status (see Errors).

Examples

Each example sends a search and retries it once on a new connection when the connection closes before any answer. Use the same function for every read route. Writes keep their Idempotency-Key instead.

const BASE_URL = "https://acme.captain.dev";
const SEARCH_KEY = process.env.CAPTAIN_SEARCH_KEY;
// For reads only: search, typeahead, browse, similar, multi search and GETs.
async function captainRead(path, body) {
  const send = () =>
    fetch(BASE_URL + path, {
      method: body === undefined ? "GET" : "POST",
      headers: { Authorization: `Bearer ${SEARCH_KEY}`, "Content-Type": "application/json" },
      body: body === undefined ? undefined : JSON.stringify(body),
      signal: AbortSignal.timeout(10_000),
    });
  try {
    return await send();
  } catch (err) {
    // fetch rejects only when no answer arrived. A status such as 503 resolves instead.
    // A timeout is not retried: the request reached Captain.
    if (err.name === "TimeoutError" || err.name === "AbortError") throw err;
    return await send(); // once more; the closed connection is not reused
  }
}
const response = await captainRead("/v1/indexes/products/search", { query: "waterproof jacket", k: 10 });
console.log(response.status, await response.json());

Browsers manage their own connections. A page that calls fetch gets the same protection from the JavaScript example.