> For a complete page index of the Captain API documentation, fetch https://docs.captain.dev/llms.txt?excludeSpec=true

# Index Supabase Manifest

POST https://api.captain.dev/v2/collections/{collection_name}/index/supabase/manifest
Content-Type: application/json

Index an explicit list of objects from a Supabase Storage bucket, given as a JSON Lines manifest in the same bucket. One bulk job; nothing is listed or scanned.

One object per line, UTF-8, uncompressed. `uri` is required, the rest are optional:

```json
{"uri": "s3://documents/inbox/2026-q4-msa.pdf", "source_identity": "contract-8841", "custom_metadata": {"department": "legal", "counterparty": "Northwind"}}
{"uri": "s3://documents/inbox/2026-q4-sow.pdf", "source_identity": "contract-8842", "size": 482113, "etag": "9b2cf535f27731c974343645a3985328"}
{"uri": "s3://documents/archive/2025-renewal.pdf", "source_identity": "contract-7310"}
{"uri": "s3://documents/inbox/handbook.pdf"}
```

### Physical location vs. logical identity

`uri` is **where the file currently lives**: this bucket, this key. It must be inside `bucket_name`, and it is the only field Captain reads bytes from.

`source_identity` is **what the document is**, in your own terms. Captain keeps one document per identity, so the identity is what decides whether a later index is an update or a new document:

* Give `s3://documents/inbox/2026-q4-msa.pdf` the identity `contract-8841`. Move the file to `s3://documents/archive/2026-q4-msa.pdf` and index it again with the same identity, and Captain updates that one document. Its chunks are replaced and the old ones are retired.
* Omit `source_identity` and the identity is the `uri`. The same file at a new key is a different identity, so you get a second document and both versions answer queries.

A repeat of an identity already in the collection is skipped while `skip_existing` is true (the default), and replaced when you send `overwrite_existing: true` or `skip_existing: false`. While it is skipping, Captain compares content. A repeat whose content is unchanged is skipped. A repeat whose content differs is an identity conflict: it is neither skipped nor indexed, and the job reports it under `identity_conflicts` so that you can send it again with `overwrite_existing: true`. When either version has no checksum, the repeat is skipped. Use your own record id, not a path, whenever content can move or be delivered twice.

### The other line fields

* `custom_metadata`: merged over the job-level `custom_metadata`, key by key, for this file only. A line's value wins on a shared key.
* `checksum` and `version_id`: recorded on the document. A `checksum` that is the object's ETag, written bare or as `etag:...`, is compared with the object before anything is read, and a line that does not match is reported as unreachable instead of being indexed. Write any other algorithm with its prefix, for example `sha256:...`; it is recorded but not verified. `version_id` pins the object version that is read.
* `size` and `etag`: when both are present, as in a bucket listing export, Captain does not read the object's metadata before indexing. That is what lets a large manifest start quickly.

### Limits and error handling

A manifest holds up to **1,000,000 lines** and **500 MB**, with each line at most **10 KB**. It must be plain UTF-8 `.jsonl` or `.ndjson`; a compressed body is refused. Per line, `custom_metadata` takes at most 50 keys, keys up to 64 characters and string values up to 1,024 characters. The job-level `custom_metadata` is capped at 64 KB serialized. It is attached to every file in the job, so metadata size times file count must stay under 2 GB: a few hundred bytes fits the full million lines, and the full 64 KB fits about 32,000 files. Larger jobs should carry less job-level metadata or be split.

A wrong `manifest_uri` is a 400 before the job is created, as is a manifest larger than the size cap. Request bodies are capped at 1 MB, which is why a large object list is a manifest in storage rather than an inline array.

Unusable lines (invalid JSON, not a JSON object, a `uri` outside the bucket, a duplicate `uri`, an unknown key) and unreachable objects (missing, access denied, unsupported file type) are counted on the job. `GET /v2/jobs/{job_id}` returns a `manifest` object with the totals and a sample of up to 20 of each, giving the line number or `uri` and a reason. Neither fails the job: a million-line manifest with a few thousand bad lines indexes the rest.

Headers:

* Authorization: Bearer \{api\_key} - Captain API key for authentication
* X-Organization-ID: Organization UUID

Returns:
\{ job\_id, status: "pending" }

Reference: https://docs.runcaptain.com/reference/indexing/supabase/manifest

## Request

### Path parameters

- `collection_name` (string, required)

### Headers

- `authorization` (string, optional, nullable)

### Body (application/json)

This endpoint expects an IndexSupabaseManifestRequest.

- `access_key_id` (string, required) — Supabase S3 access key ID (Storage → S3 Connection → access key).
- `bucket_name` (string, required)
- `endpoint_url` (string, required) — S3-compatible endpoint URL for your Supabase project's Storage S3 connection, e.g. https\://\{project\_ref}.storage.supabase.co/storage/v1/s3 (Storage → S3 Connection). Required.
- `manifest_uri` (string, required) — URI of the manifest, inside bucket\_name (s3://bucket/path/manifest.jsonl or supabase://bucket/path/manifest.jsonl). UTF-8 JSON Lines, uncompressed, one object per line: \{"uri": ...} plus optional source\_identity, custom\_metadata, checksum, version\_id, size and etag. Up to 1,000,000 lines, 500 MB total and 10 KB per line.
- `processing_type` (enum, required) — Document processing type. 'advanced' uses agentic OCR with AI-enhanced extraction for complex layouts, tables, figures, charts, and documents containing images. 'basic' provides reliable OCR optimized for general document indexing and high-volume processing. 'auto' inspects each PDF and uses advanced only when it has photos, drawings, tables or scanned pages (logos and signatures do not count), otherwise basic; each file bills at the tier it used.
  - Allowed values: `advanced`, `basic`, `auto`
- `secret_access_key` (string, required) — Supabase S3 secret access key (shown once when the S3 access key is created).
- `custom_metadata` (map from string to IndexSupabaseManifestRequestCustomMetadata, optional, nullable) — Custom metadata attached to every indexed chunk. A line's own custom_metadata is merged over it for that file. Keys are strings; values: str, int, float, bool, or List[str].
- `mask_pii` (boolean, optional, default: false) — When true, detected PII (emails, phone numbers, SSNs, credit cards, names, and locations) is masked in the parsed content before it is embedded and stored — replaced with entity tags like \<PERSON> and \<EMAIL\_ADDRESS>. For images (including images embedded in PDFs), PII text visible in the image is also pixel-redacted. Opt-in; defaults to false, which leaves content unchanged.
- `max_files` (integer, optional, nullable) — Stop after this many manifest lines, in file order.
- `overwrite_existing` (boolean, optional, default: false) — When true, files that already exist in the collection are re-indexed and replaced with zero downtime: the new version is built alongside the live one and atomically swapped in when complete, so the previous version keeps serving search results throughout the rebuild. The document keeps the same document_id across overwrites, and its status reads 'updating' in the document listing while the rebuild runs. Requires skip_existing=false. Setting both to true returns a 400 error.
- `parsing_script` (string, optional, nullable) — Relative path to a JS parsing script for JSON files (e.g. 'research/paper-parser'). When provided, .json files are processed through a sandboxed V8 isolate. Without this, .json files are indexed as raw text.
- `pii_engine` (enum, optional, default: kev) — Masking engine for this job. `kev` (default): the Kev classifier model on Captain's infrastructure; supports `pii_fields` and `pii_instructions`; no fallback, so a file kev cannot mask fails to index and the rest of the job continues. `jev`: TypeSafe's hosted Jev model, slightly more accurate, best-effort availability; supports custom fields and instructions and may list fallbacks in `pii_fallback`. `presidio`: the legacy pattern engine, built-in categories only (rejects `pii_fields` or `pii_instructions` with 422), never a fallback. Requires `mask_pii: true`. The earlier names `captain-jev` and `captain-presidio` keep working as before; new requests should use `kev`, `jev` or `presidio`.
  - Allowed values: `kev`, `jev`, `presidio`, `captain-jev`, `captain-presidio`
- `pii_fallback` (IndexSupabaseManifestRequestPiiFallback, optional) — Ordered fallback engines for `jev`, as `{"engines": [...], "retry_budget_seconds": n}`. Omitted or an empty `engines` list means no fallback. Only valid with `pii_engine: "jev"`; with `kev` or `presidio` a non-empty list answers 422. On the multipart file upload endpoint, send it as a JSON string form field. With the earlier engine name `captain-jev` it is `true` (default, fall back to `captain-presidio`) or `false`.
- `pii_fields` (list of PIIField, optional, nullable) — Additional PII categories to mask for this job, on top of the built-in set. Requires mask_pii to be true. Up to 20 fields. Each masked value is replaced with the field's tag in angle brackets and appears in the masking report under the field's name.
- `pii_instructions` (string, optional, nullable) — Plain-language guidance that steers what counts as personal data for this job, for the built-in categories and any pii_fields. For example: 'Names of hospital staff may stay; patient names, record numbers and bed assignments must be masked.' Requires mask_pii to be true. Up to 1,000 characters.
- `region` (string, optional, default: us-east-1) — Region configured for your Supabase project's S3 endpoint (Storage → S3 Connection). Required by the S3 protocol; defaults to 'us-east-1'.
- `skip_existing` (boolean, optional, default: true) — When true, files already indexed in the collection are skipped and will not be re-indexed with incoming changes. When false, all incoming files are indexed regardless of whether they already exist.
- `transcription_language` (string, optional, nullable) — AWS Transcribe language code for the spoken audio (e.g. 'es-US', 'pt-BR'). Omit to auto-detect per file. Video and audio files only. Supported codes: https://docs.aws.amazon.com/transcribe/latest/dg/supported-languages.html

## Response

### 200

Successful Response

- `job_id` (string, required)
- `status` (string, optional, default: pending)
- `custom_metadata` (map from string to any, optional, nullable) — The custom_metadata Captain accepted for this job, echoed back as validated. Null when none was supplied.

## Errors

### 400 Bad Request Error

manifest missing or unreadable, or overwrite_existing and skip_existing both true

- `error` (string, required) — Error kind. `HTTP_ERROR` for errors raised by the route.
- `message` (string, required) — What went wrong.
- `path` (string, required) — The request path.
- `status_code` (integer, optional) — The HTTP status code.

### 422 Unprocessable Entity Error

Validation Error

- `error` (string, required) — `VALIDATION_ERROR`.
- `message` (string, required)
- `details` (list of V2CollectionsCollectionNameIndexSupabaseManifestPostResponsesContentApplicationJsonSchemaDetailsItems, required) — One entry per invalid field, with `type`, `loc` and `msg`.
- `path` (string, required)

## Types

### IndexSupabaseManifestRequestCustomMetadata

- string
- integer
- double
- boolean
- list of string

### IndexSupabaseManifestRequestPiiFallback

Ordered fallback engines for `jev`, as `{"engines": [...], "retry_budget_seconds": n}`. Omitted or an empty `engines` list means no fallback. Only valid with `pii_engine: "jev"`; with `kev` or `presidio` a non-empty list answers 422. On the multipart file upload endpoint, send it as a JSON string form field. With the earlier engine name `captain-jev` it is `true` (default, fall back to `captain-presidio`) or `false`.

- PiiFallbackPolicy: PiiFallbackPolicy
- boolean

### PIIField

A customer-defined PII category to mask in addition to the built-in ones.

- `name` (string, required) — Tag for this category. Uppercase letters, digits and underscores, 2 to 40 characters, unique within the request and not one of the built-in tags. Masked values are replaced with the tag in angle brackets (a field named EMPLOYEE\_ID produces \<EMPLOYEE\_ID>) and the masking report lists them under this name.
- `description` (string, required) — What the category covers, in plain language, as you would explain it to a colleague. This is what decides whether a value is masked, so say what the values look like and where they appear.
- `examples` (list of string, optional, nullable) — Up to 10 sample values, used only to make the description concrete. They are not a pattern: values that do not look like the examples are still masked when the description covers them.

### V2CollectionsCollectionNameIndexSupabaseManifestPostResponsesContentApplicationJsonSchemaDetailsItems

### PiiFallbackPolicy

Fallback policy for `pii_engine: "jev"`. If every engine fails, the file fails to index and the rest of the job continues.

- `engines` (list of enum, optional, default: []) — Engines to try, in order, when `jev` fails with a rate limit, timeout, capacity limit, provider error or invalid answer. `jev-openrouter` and `jev-ai-gateway` are the same Jev model served through OpenRouter and Vercel AI Gateway. Custom fields and instructions apply on every engine. `presidio` is not allowed, and an engine may appear only once.
  - Allowed values: `jev-openrouter`, `jev-ai-gateway`, `kev`
- `retry_budget_seconds` (integer, optional, default: 60) — Extra retry and wait time per engine, for each piece of a file, before moving to the next engine. Time spent on successful answers does not count. 0 means one attempt per engine. Normal request timeouts still apply.

## Examples

**Request**

```json
{
  "access_key_id": "your_supabase_access_key",
  "bucket_name": "documents",
  "endpoint_url": "https://abcdefgh.storage.supabase.co/storage/v1/s3",
  "manifest_uri": "s3://documents/manifests/batch-2026-09.jsonl",
  "processing_type": "advanced",
  "secret_access_key": "your_supabase_secret_key",
  "custom_metadata": {
    "batch": "2026-09"
  },
  "region": "us-east-1"
}
```

**Response**

```json
{
  "job_id": "3f9c2b1e7d5a4c8b9e0f1a2b3c4d5e6f-1789700000",
  "status": "pending",
  "custom_metadata": {
    "batch": "2026-09"
  }
}
```

**SDK Code**

```python Python REST
import json
import boto3
import requests

BASE_URL = "https://api.captain.dev"
API_KEY = "your_api_key"

headers = {
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "application/json"
}

lines = [
    {
        "uri": "s3://documents/inbox/2026-q4-msa.pdf",
        "source_identity": "contract-8841",
        "custom_metadata": {"department": "legal", "counterparty": "Northwind"},
    },
    {
        "uri": "s3://documents/inbox/2026-q4-sow.pdf",
        "source_identity": "contract-8842",
        "size": 482113,
        "etag": "9b2cf535f27731c974343645a3985328",
    },
    {
        "uri": "s3://documents/archive/2025-renewal.pdf",
        "source_identity": "contract-7310",
    },
    {
        "uri": "s3://documents/inbox/handbook.pdf",
    },
]
boto3.client(
    "s3",
    endpoint_url="https://abcdefgh.storage.supabase.co/storage/v1/s3",
    aws_access_key_id="your_supabase_access_key",
    aws_secret_access_key="your_supabase_secret_key",
    region_name="us-east-1",
).put_object(
    Bucket="documents",
    Key="manifests/batch-2026-09.jsonl",
    Body="\n".join(json.dumps(line) for line in lines),
)

response = requests.post(
    f"{BASE_URL}/v2/collections/my_documents/index/supabase/manifest",
    headers=headers,
    json={
        "bucket_name": "documents",
        "manifest_uri": "s3://documents/manifests/batch-2026-09.jsonl",
        "endpoint_url": "https://abcdefgh.storage.supabase.co/storage/v1/s3",
        "access_key_id": "your_supabase_access_key",
        "secret_access_key": "your_supabase_secret_key",
        "region": "us-east-1",
        "processing_type": "advanced",
        "custom_metadata": {
            "batch": "2026-09"
        }
    },
    timeout=60.0
)

if response.status_code in [200, 201]:
    data = response.json()
    print(f"Job {data['job_id']} is {data['status']}")
else:
    print(f"Error {response.status_code}: {response.text}")

```

```typescript TypeScript REST
import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";

const BASE_URL = "https://api.captain.dev";
const API_KEY = "your_api_key";

const lines = [
    {
        uri: "s3://documents/inbox/2026-q4-msa.pdf",
        source_identity: "contract-8841",
        custom_metadata: { department: "legal", counterparty: "Northwind" },
    },
    {
        uri: "s3://documents/inbox/2026-q4-sow.pdf",
        source_identity: "contract-8842",
        size: 482113,
        etag: "9b2cf535f27731c974343645a3985328",
    },
    {
        uri: "s3://documents/archive/2025-renewal.pdf",
        source_identity: "contract-7310",
    },
    {
        uri: "s3://documents/inbox/handbook.pdf",
    },
];
await new S3Client({
    region: "us-east-1",
    endpoint: "https://abcdefgh.storage.supabase.co/storage/v1/s3",
    forcePathStyle: true,
    credentials: { accessKeyId: "your_supabase_access_key", secretAccessKey: "your_supabase_secret_key" },
}).send(new PutObjectCommand({
    Bucket: "documents",
    Key: "manifests/batch-2026-09.jsonl",
    Body: lines.map((line) => JSON.stringify(line)).join("\n"),
}));

const response = await fetch(
    `${BASE_URL}/v2/collections/my_documents/index/supabase/manifest`,
    {
        method: "POST",
        headers: {
            "Authorization": `Bearer ${API_KEY}`,
            "Content-Type": "application/json"
        },
        body: JSON.stringify({
            bucket_name: "documents",
            manifest_uri: "s3://documents/manifests/batch-2026-09.jsonl",
            endpoint_url: "https://abcdefgh.storage.supabase.co/storage/v1/s3",
            access_key_id: "your_supabase_access_key",
            secret_access_key: "your_supabase_secret_key",
            region: "us-east-1",
            processing_type: "advanced",
            custom_metadata: {
                batch: "2026-09"
            }
        })
    }
);

if (response.ok) {
    const data = await response.json();
    console.log(`Job ${data.job_id} is ${data.status}`);
} else {
    console.error(`Error ${response.status}: ${await response.text()}`);
}

```