Index S3 Bucket | Captain Docs

import requests
BASE_URL = "https://api.runcaptain.com"
API_KEY = "your_api_key"
headers = {
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "application/json"
}
response = requests.post(
    f"{BASE_URL}/v2/collections/my_documents/index/s3",
    headers=headers,
    json={
        "bucket_name": "my-documents-bucket",
        "aws_access_key_id": "AKIAIOSFODNN7EXAMPLE",
        "aws_secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
        "bucket_region": "us-east-1",
        "processing_type": "advanced",
        "skip_existing": True
    },
    timeout=60.0
)
if response.status_code in [200, 201]:
    data = response.json()
    print(f"Job started! ID: {data['job_id']}")
else:
    print(f"Error: {response.status_code}")

Index all files from S3 bucket into a collection.

Headers:

Authorization: Bearer {api_key} - Captain API key for authentication
X-Organization-ID: Organization UUID
Idempotency-Key: UUID for request deduplication (optional)

Args: collection_name: Name of the collection (path parameter) body: S3 bucket configuration

Returns: { job_id, status: “pending” }

Index all files from S3 bucket into a collection. Headers: - Authorization: Bearer {api_key} - Captain API key for authentication - X-Organization-ID: Organization UUID - Idempotency-Key: UUID for request deduplication (optional) Args: collection_name: Name of the collection (path parameter) body: S3 bucket configuration Returns: { job_id, status: "pending" }

Path parameters

collection_namestringRequired

Request

This endpoint expects an object.

bucket_namestringRequired

processing_typeenumRequired

Document processing type. ‘advanced’ uses agentic OCR with AI-enhanced extraction for complex layouts, tables, figures, charts, and documents containing images. ‘basic’ provides reliable OCR optimized for general document indexing and high-volume processing.

authobject or nullOptional

Cross-account role-assumption auth. When provided, Captain calls sts:AssumeRole on the supplied role_arn (with the supplied external_id) instead of using static IAM-user keys. No long-lived secrets cross the boundary; recommended for production. Mutually exclusive with aws_access_key_id/aws_secret_access_key.

aws_access_key_idstring or nullOptional

AWS access key ID with read access to the bucket. Use this for long-lived IAM-user credentials. Omit when using the role-based ‘auth’ block.

aws_secret_access_keystring or nullOptional

AWS secret access key. Use this for long-lived IAM-user credentials. Omit when using the role-based ‘auth’ block.

bucket_regionstringOptionalDefaults to us-east-1

custom_metadatamap from strings to strings or integers or doubles or booleans or lists of strings or nullOptional

Custom metadata to attach to all indexed chunks. Keys must be strings. Values: str, int, float, bool, or List[str].

max_filesinteger or nullOptional

overwrite_existingbooleanOptionalDefaults to false

When true, files that already exist in the collection are re-indexed and replaced with zero downtime: the new version is built alongside the live one and atomically swapped in when complete, so the previous version keeps serving search results throughout the rebuild. The document keeps the same document_id across overwrites, and its status reads ‘updating’ in the document listing while the rebuild runs. Requires skip_existing=false. Setting both to true returns a 400 error.

parsing_scriptstring or nullOptional

Relative path to a JS parsing script for JSON files (e.g. ‘research/paper-parser’). When provided, .json files are processed through a sandboxed V8 isolate. Without this, .json files are indexed as raw text.

skip_existingbooleanOptionalDefaults to true

When true, files already indexed in the collection are skipped and will not be re-indexed with incoming changes. When false, all incoming files are indexed regardless of whether they already exist.

mask_piibooleanOptionalDefaults to false

When true, detected PII (emails, phone numbers, SSNs, credit cards, names, and locations) is masked in the parsed content before it is embedded and stored — replaced with entity tags like <PERSON> and <EMAIL_ADDRESS>. For images (including images embedded in PDFs), PII text visible in the image is also pixel-redacted. Opt-in; defaults to false, which leaves content unchanged.

transcription_languagestring or nullOptional

AWS Transcribe language code for the spoken audio (e.g. ‘es-US’, ‘pt-BR’). Omit to auto-detect per file. Video and audio files only. Supported codes: https://docs.aws.amazon.com/transcribe/latest/dg/supported-languages.html

Response

Successful Response

job_idstring

statusstringOptionalDefaults to pending

Errors

400

Bad Request Error

1	{
2	"job_id": "job_s3_abc123",
3	"status": "pending"
4	}