File Types

Video

Captain indexes video as clips of up to 120 seconds. Each clip has a time range, a native embedding of the clip, a transcript slice and a written description of what is on screen. This page covers how a video is cut, what each clip contains, what a result contains, YouTube options, and the limits.

This is useful for

  • Product demos and webinars, where the reader wants the moment rather than the file
  • Training and onboarding video, found by what is on the slide as well as what is said
  • Screen recordings of bugs and walkthroughs, found by the text in the UI
  • Site and inspection footage filmed on a phone, with or without narration

What one video becomes

The file is normalised to a standard 720p MP4 and cut into contiguous clips of up to 120 seconds. Each clip gets a native embedding of the clip itself, a transcript slice and a written description, stored together on one record.

How a video is chunked

1

Clips with a time range

Clips are contiguous 120-second windows; the last one is shorter. Every clip carries location.start_seconds and location.end_seconds on the v3 query, or metadata.startSec and metadata.endSec on v2.

2

Transcript

Speech-to-text runs once over the whole file, so a sentence that crosses a clip boundary is transcribed correctly. Word timings are then used to give each clip the words spoken inside its window. Language is detected automatically; set transcription_language (for example es-US or pt-BR) to force one. There are no speaker labels.

3

Description

Each clip’s description has a category line and three to five sentences covering subjects, actions, objects and setting, with on-screen text transcribed verbatim (slide titles, captions, UI labels) and charts or diagrams read out. The description does not repeat the speech; spoken words are in the transcript.

4

Three signals on one record

The native embedding, the transcript slice and the description are stored on the same record. A query matches the picture by vector similarity and the words by text. The result text is [Video: file name], then the description, then Transcript: ….

5

Audio problems

A video whose audio track cannot be decoded is indexed video-only. Its speech is still transcribed, because the speech-to-text pass reads the source file rather than the normalised copy. A silent video has no transcript.

YouTube

POST /v2/collections/{collection}/index/youtube accepts up to 20 URLs. mode: "transcript" (the default) indexes the captions as text and bills as text. mode: "audio" or mode: "video" runs the media pipeline and bills per minute. on_missing_transcript (fail, audio or video) sets what happens to a video with no captions, and languages sets the caption language preference.

Limits and billing

Video is billed at 20 credits per minute, summed per job. Size limits depend on the route: 100 MB through the direct upload endpoint, 500 MB from the dashboard, and no upload limit from a cloud storage connector. Collections that contain media rerank by default; see Multimodal Search and File Size Limits.

Examples

Goal: return the clip where the presenter filters a report by region, with the offset to play from.

request
{
  "query": "filtering the report by region",
  "limit": 3
}
result (one item)
{
  "modality": "video",
  "text": "[Video: q3-demo.mp4]\nCategory: software demo, dashboard. A presenter opens a filter panel on the left of an analytics dashboard and selects 'EMEA'...\nTranscript: let's narrow this down to EMEA so you can see the regional split",
  "location": {
    "start_seconds": 480.0,
    "end_seconds": 600.0
  }
}

The description records the filter panel and the transcript records “EMEA”. Both are on the clip that starts at 480 seconds.

Goal: find the clip that shows a procedure’s slide, even though the presenter does not read the title aloud.

request
{
  "query": "lockout tagout procedure",
  "limit": 3,
  "filter": {
    "course": {
      "$eq": "site-induction"
    }
  }
}

On-screen text is transcribed into the description, so the slide title is searchable.

Goal: find a silent bug reproduction by the text of the error dialog.

request
{
  "query": "\"payment method declined\" dialog",
  "limit": 3
}

The description transcribes UI text. A silent clip has no transcript line.

See Supported File Types for the extension table and Audio for audio-only files.

© 2026 Captain