> For a complete page index of the Captain API documentation, fetch https://docs.captain.dev/llms.txt?excludeSpec=true

# Text and Code

> How Captain indexes plain text, Markdown, JSON, YAML, logs, and source code. One lane for 156 extensions, read as text, with a parsing script for JSON that needs shaping first.

## Agent Quick Reference - Text and Code

* 156 extensions (`.txt`, `.md`, `.json`, `.yaml`, `.yml`, and every source, markup and config extension on Supported File Types) share one lane and are read as plain text. No syntax parsing: a `.py` and a `.txt` are handled the same way.
* Excluded on purpose: compiled or binary variants (`.pyc`, `.pyo`, `.class`, `.o`, `.so`, `.dll`, `.wasm`, `.jar`, `.svgz`), `.plist`, and secret-bearing files (`.env`, `.pem`, `.key`, `.crt`). A file with a text extension whose first 8 KB contains a null byte is skipped as binary, with zero chunks and no job failure.
* Chunking: a file up to about 40,000 tokens is one section; longer files are split at Markdown headings and fenced code blocks when the file is Markdown, otherwise at line boundaries. A single line longer than the budget is cut at whitespace, then at `,` `;` `}` `]`, then hard. Sections are summarised and tagged, then tiled into the passages that are embedded. No overlap between passages.
* Chunk metadata: `summary`, `file_tags`, `chunk_index`, `parent_chunk_index`, `tokens`, `file_type`; `page_start`/`page_end` are null for text and code (pages exist only for documents).
* Encoding: UTF-8, falling back to Latin-1. A UTF-8 BOM is kept as a character.
* JSON: without a `parsing_script`, a `.json` file is indexed as raw text. With one, a JavaScript function `export default function (doc) { return string }` runs in a sandbox (250 MB document, 60 s, single JSON value, no `require`/`fetch`) and its returned text is what gets indexed.
* Raw text: `POST /v2/collections/{name}/index/text` with `content` (10 MB), optional `filename` (forced to `.txt`), `mask_pii`, `custom_metadata`. Identity is `upload://<filename>`, so re-sending the same name updates the same document.
* Limits: 250 MB per file (rejected before download when the storage reports size); the raw text route is 10 MB. Billing: 0.5 credits per 3,000 characters.

Captain indexes plain text, Markdown, JSON, YAML, logs and source code through one lane that
reads every file as text. This page covers which files go through it, how they are chunked,
the JSON parsing script, the raw text route, and the limits.

## This is useful for

* Runbooks, READMEs and wikis kept as Markdown, where the section under a heading is the answer
* JSON exports from other systems, shaped into records with a short parsing script before indexing
* Logs, configs and source files that should be findable by an exact string
* Short text sent straight from your application through the raw text route

## What goes through this lane

Every extension under [Text and code](/guides/supported-file-types#text-and-code) on
Supported File Types: `.txt`, `.md`, `.json`, `.yaml`, `.yml`, and 151 source, markup and
config extensions. All are read as plain text. There is no syntax-aware parsing: a Python
file and a text file are chunked the same way, and an HTML file is indexed with its tags in
place.

Not indexed: compiled or binary variants (`.pyc`, `.pyo`, `.class`, `.o`, `.so`, `.dll`,
`.wasm`, `.jar`, `.svgz`), `.plist`, and files that usually hold secrets (`.env`, `.pem`,
`.key`, `.crt`). A file with a text extension whose first 8 KB contains a null byte is
treated as binary and skipped with zero chunks; the job does not fail. Extensionless files
from a connector are indexed when the storage reports a text content type.

## How text is chunked

#### Sections

A file of up to about 40,000 tokens is one section. Longer files are split at Markdown
headings and fenced code blocks when the file is Markdown, and at line boundaries
otherwise. A single line longer than the budget (a minified bundle, a one-line JSON blob)
is cut at whitespace first, then at `,` `;` `}` `]`, and only then at a character.

#### Summaries and passages

Each section gets a generated summary (two or three sentences) and a set of tags, and is
tiled into shorter passages that are embedded. A result contains the passage and the
summary and tags of its section. Passages do not overlap.

#### No page ranges

`page_start` and `page_end` are null for text and code. Page ranges exist for
[documents](/guides/file-types/documents). Encoding is UTF-8, with a fallback to
Latin-1; a UTF-8 byte order mark is kept as a character.

#### JSON parsing scripts

A `.json` file indexed without a `parsing_script` is read as raw text, including the
braces. With a `parsing_script`, a JavaScript function receives the parsed document and
returns the text to index, so chunks fall on records rather than on syntax. The script
runs in a sandbox with no network or filesystem access, must return a non-empty string,
and has 60 seconds. The document must be a single JSON value of at most 250 MB. JSON
Lines files are indexed as text. See [JSON Parsers](/guides/json-parsers).

## Raw text

`POST /v2/collections/{name}/index/text` indexes a string directly, up to 10 MB, with an
optional `filename`, `mask_pii` and `custom_metadata`. The file name is forced to end in
`.txt`. The document's identity is `upload://<filename>`, so sending the same name again
updates that document instead of creating a second one.

## Limits and billing

Files in this lane are limited to 250 MB. When the storage can report a size up front, an
oversized file is rejected before download; otherwise it is rejected after download. Text is
billed at 0.5 credits per 3,000 characters, rounded up per file. See
[File Size Limits](/guides/file-size-limits).

## Examples

#### A docs folder in Markdown

Goal: return the section that explains a procedure, from a `docs/` tree indexed from a
bucket.

**`request`**

```json title="request"
{
  "query": "rotate the signing key",
  "limit": 3
}
```

**`result (one item)`**

```json title="result (one item)"
{
  "text": "## Rotating the signing key\n\n1. Generate a new key pair with `make keys`...",
  "document": {
    "id": "…",
    "name": "docs/ops/auth.md"
  },
  "metadata": {
    "summary": "Steps to rotate the JWT signing key without downtime, including the dual-key window.",
    "file_tags": [
      "auth",
      "keys",
      "rotation",
      "ops"
    ]
  }
}
```

Markdown files are split at headings, so the result is the section under the matching
heading. `metadata.summary` describes the section.

#### A product catalogue in JSON

Goal: index a JSON export so each result is a product entry rather than a fragment of
syntax.

**`parsing_script`**

```javascript title="parsing_script"
export default function (doc) {
  return doc.products
    .map(p => `${p.sku} ${p.name}\n${p.description}\nCategory: ${p.category}`)
    .join("\n\n");
}
```

**`request`**

```json title="request"
{
  "query": "stainless 40 mm ball valve",
  "limit": 5,
  "filter": {
    "catalogue": {
      "$eq": "2026"
    }
  }
}
```

The script defines what one record looks like as text, so the chunker cuts on records.
The SKU stays in the text and matches an exact query.

#### Application logs

Goal: find an error across a month of logs and narrow it by day. A 200 MB log is under the
limit but is one document; split logs by day or by service before indexing and set the
day as `custom_metadata`.

**`request`**

```json title="request"
{
  "query": "connection pool exhausted",
  "limit": 10,
  "filter": {
    "day": {
      "$gte": "2026-09-15"
    }
  },
  "max_chunks_per_document": 3
}
```

Logs are cut at line boundaries, so a passage is a run of whole lines. Per-day files give
the `day` filter something to match.