# Primate Vision API — full docs corpus (llms-full.txt)

> Every docs page in one file. Canonical per-page markdown lives at https://www.primateintelligence.ai/docs/<slug>.md. The OpenAPI 3.1 contract is the source of truth: https://api.primateintelligence.ai/v1/openapi.json


---

<!-- source: https://www.primateintelligence.ai/docs/quickstart.md -->

# Quickstart

> First analysis in three curl commands — no signup required.


Primate Vision answers questions about video. Upload a video (or point us at a URL), ask a question in plain English, and get a deterministic answer with confidence and clip timestamps — no hallucinations.

This page gets you from zero to a verified result in about a minute, **without creating an account**.

## 1. Get a key

One command. No email, no card:

```bash doc-test id=quickstart-sandbox
curl -s -X POST https://api.primateintelligence.ai/v1/sandbox
```

The response contains your `pv_test_` key plus a ready-to-analyze fixture video:

```json
{
  "object": "sandbox_key",
  "api_key": "pv_test_…",
  "mode": "test",
  "expires_at": "2026-07-16T00:00:00Z",
  "fixture_video_id": "video_…",
  "fixture_prompt": "Is there a person in this video?",
  "expected_answer": "yes"
}
```

Export it:

```bash
export PRIMATE_API_KEY="pv_test_…"   # from the response above
```

Test keys return **deterministic canned results** for the fixture corpus — perfect for CI. They can't spend credits or touch the GPU, and expire in 7 days. Graduating to a `pv_live_` key requires [signup](https://primateintelligence.ai/signup) (a billing gate, not an integration gate — your code doesn't change).

## 2. Create an analysis

Your sandbox account comes pre-seeded with a fixture video. Ask a question about it — `Prefer: wait` holds the connection until the result is ready (up to 120s), collapsing the poll loop:

```bash doc-test id=quickstart-analyze env=VIDEO_ID
curl -s -X POST https://api.primateintelligence.ai/v1/analyses \
  -H "Authorization: Bearer $PRIMATE_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait=60" \
  -d '{"video_id": "'"$VIDEO_ID"'", "prompt": "Is there a person in this video?"}'
```

## 3. Read the result

The response is the full analysis resource:

```json
{
  "id": "an_…",
  "object": "analysis",
  "status": "completed",
  "result": {
    "answer": "yes",
    "confidence": 0.93,
    "clips": [{ "start_s": 1, "end_s": 4, "confidence": 0.93 }],
    "query_type": "object"
  }
}
```

That's the whole loop: **video → question → answer**. `result.answer` is always `yes`, `no`, or `indeterminate`; `result.clips` tells you *where* in the video.

## The same thing in TypeScript

```typescript doc-test id=quickstart-ts sdk=ts
import Primate from '@primate-intelligence/sdk';

const client = new Primate(); // reads PRIMATE_API_KEY

const videos = await client.videos.list();          // fixture video is pre-seeded
const analysis = await client.analyses.createAndWait({
  video_id: videos.data[0].id,
  prompt: 'Is there a person in this video?',
});
console.log(analysis.result?.answer, analysis.result?.confidence);
```

## The same thing in Python

```python doc-test id=quickstart-py sdk=py
from primate_intelligence import Primate

client = Primate()  # reads PRIMATE_API_KEY

videos = client.videos.list()                       # fixture video is pre-seeded
analysis = client.analyses.create_and_wait(
    video_id=videos["data"][0]["id"],
    prompt="Is there a person in this video?",
)
print(analysis["result"]["answer"], analysis["result"]["confidence"])
```

## Next steps

- **[Upload your own video](/docs/guides/uploading)** — presigned upload, URL ingest, or the SDK helper
- **[Write better prompts](/docs/guides/prompts)** — what's answerable, structured queries, `unassessable_components`
- **[Stop polling](/docs/guides/waiting)** — `Prefer: wait` vs webhooks
- **[Building with an AI agent?](/docs/agents)** — the condensed machine-readable path
- **[API reference](/docs/reference)** — every endpoint, generated from the OpenAPI spec

---

<!-- source: https://www.primateintelligence.ai/docs/agents.md -->

# Quickstart for AI agents

> The zero-human-intervention integration path — canonical copy served by the API deploy artifact.


Machine-consumable reference for AI agents integrating with the Primate Intelligence Public API (`/v1`).

---

## Quick Start

### 1. Get a sandbox key (zero-human, instant)

```http
POST /v1/sandbox
```

Returns a `pv_test_` key, a `fixture_video_id`, and a `fixture_prompt`. No auth required. Provisioning is IP-capped per client (rolling 24h) — hitting the cap returns `429 sandbox_limit_exceeded` with `retry_after`.

**Response includes:**
- `livemode: false` — all sandbox responses are always test-mode
- `upgrade` — instructions for upgrading to a live key (see below)

### 2. Upgrade to a free live key (GitHub-verified)

Call `POST /v1/keys/upgrade` with your `pv_test_` key. Receives a real `pv_live_` key with a **6,000-second free credit grant**. No email, no card — but a **verified GitHub account is required** (one free grant per GitHub account, ever).

**Step A — first call returns the GitHub device-flow bootstrap:**

```http
POST /v1/keys/upgrade
Authorization: Bearer pv_test_<your-sandbox-key>
```

Returns `403 github_verification_required`. The error `details` carry everything needed:
`client_id`, `device_code_url` (`https://github.com/login/device/code`), and `access_token_url` (`https://github.com/login/oauth/access_token`).

**Step B — GitHub device flow (standard):**

1. `POST https://github.com/login/device/code` with `client_id` (and `Accept: application/json`) → `{device_code, user_code, verification_uri, interval}`
2. Show the user `verification_uri` + `user_code` (or open it in a browser); the user signs in to GitHub and enters the code
3. Poll `POST https://github.com/login/oauth/access_token` with `client_id`, `device_code`, `grant_type=urn:ietf:params:oauth:grant-type:device_code` every `interval` seconds until it returns `access_token`

No scopes are requested — the token only proves account identity.

**Step C — retry the upgrade with the token:**

```http
POST /v1/keys/upgrade
Authorization: Bearer pv_test_<your-sandbox-key>
Content-Type: application/json

{"github_token": "gho_..."}
```

The token is verified with GitHub and **discarded — never stored** (only the numeric GitHub account id is retained to enforce one-grant-per-account). A GitHub account that already claimed its grant gets `409 github_account_already_used` — use the billed claim flow (step 3) instead.

Response includes a `pv_live_` key and `tier: "free_grant"`. Free-grant keys:
- Run real GPU inference (`livemode: true`)
- Have no overage — when the grant is exhausted, analyses return `402 grant_exhausted` with a link to the claim flow for billed keys

### 3. Get a billed live key (device-code flow)

```http
POST /v1/keys/request
```

Returns `{device_code, claim_url, poll_interval}`. Direct the user to `claim_url`. Poll:

```http
GET /v1/keys/request/{device_code}
```

Returns `{status: "pending"}` until the user approves in the browser. On approval returns the billed `pv_live_` key ONCE (subsequent polls return `410 Gone`).

---

## The `livemode` Field

**Every public API resource includes a top-level `"livemode": boolean` field** (Stripe-style).

| Value | Meaning |
|-------|---------|
| `true` | Request used a `pv_live_` key — results are real GPU inference |
| `false` | Request used a `pv_test_` key or sandbox — results are deterministic canned fixtures |

**Agents SHOULD verify `livemode` matches their expectation before relaying results to users or downstream systems.** A result with `livemode: false` is a fixture/test result and must never be treated as evidence about real video content.

Example analysis response:
```json
{
  "object": "analysis",
  "livemode": true,
  "status": "completed",
  "result": { "answer": "yes", "confidence": 0.97, ... }
}
```

---

## Test Key Restrictions

Test (`pv_test_`) keys **only analyze the fixture video** seeded by `/v1/sandbox`. Attempting to analyze any other video returns:

```json
{
  "error": {
    "code": "test_key_fixture_only",
    "message": "Test keys can only analyze the provided fixture video. Create a live key to analyze your own uploads.",
    "status": 403
  }
}
```

This prevents fabricated results on real video content.

---

## Video Metadata

After `POST /v1/videos/{id}/complete`, the API probes the uploaded S3 object and populates:

| Field | Type | Description |
|-------|------|-------------|
| `duration_s` | number \| null | Duration in seconds |
| `width` | integer \| null | Frame width in pixels |
| `height` | integer \| null | Frame height in pixels |
| `fps` | number \| null | Frames per second |

Agents can use these fields to verify the right file arrived before submitting an analysis. Probe failures are non-fatal — fields remain `null` if probing fails.

---

## Free Re-runs (platform incidents)

When an analysis **fails because of a platform incident** (GPU outage, degraded inference), the failed analysis carries **`rerun_eligible: true`**. Call `POST /v1/analyses/{id}/rerun` to create a **fresh analysis at no charge** — same video, prompt/query, model, and options; new `an_` id; `usage` stays `null` on the re-run (never billed). One free re-run per failed analysis. Analyses that fail for non-platform reasons return `409 rerun_not_eligible`. If the re-run dispatch itself fails (503), your free re-run is NOT consumed — retry later.

---

## Demo Videos (public onboarding samples)

`GET /v1/demo-videos` (no auth) returns curated sample videos with **precomputed results** for example prompts — `{videos: [{id, display_name, duration_seconds, thumbnail_url, results_by_prompt}]}` where `results_by_prompt` maps prompt text → `{answer, confidence, query_type, video_url, computed_at}`. Use these to inspect real result shapes before uploading anything. Documented subset only — extra fields exist but may change.

---

## Video Playback & Result History (PRI-496)

**Source playback** — `GET /v1/videos/{id}` (and the list) populates `media: {url, expires_at}` on `ready` videos: a signed playback URL for the original source video, **fresh on every read with a 1-hour TTL**. When it expires, re-fetch the video for a new one — old videos always stay playable. `media` is `null` while the video is not ready or has no stored source object.

**Per-video result history** — `GET /v1/videos/{id}/analyses` lists every analysis run against a video (newest first), with the same list envelope and filters as `GET /v1/analyses` (`status`, `limit`, `starting_after`). It is the resource-nested spelling of `GET /v1/analyses?video_id={id}` — use whichever fits your client. Together with `Analysis.artifacts` (annotated result video, also sign-on-read) this gives full library semantics: list videos, replay sources, and rebuild each video's analysis history.

---

## Error Codes

| Code | HTTP | Retryable | Description |
|------|------|-----------|-------------|
| `test_key_fixture_only` | 403 | no | Test key tried to analyze a non-fixture video. Use a live key. |
| `grant_exhausted` | 402 | no | Free-grant credits depleted. Use the claim flow to get a billed key. |
| `insufficient_credits` | 402 | no | Live key ran out of credits. Add credits or upgrade. |
| `sandbox_limit_exceeded` | 429 | yes | Too many sandbox provisions from this IP. Retry after the window. |
| `upgrade_limit_exceeded` | 429 | yes | Too many free-grant upgrades from this IP today. |
| `invalid_api_key` | 401 | no | Key not found or malformed. |
| `key_expired` | 401 | no | Key has passed its `expires_at`. |

Full registry: `GET /v1/errors`

---

## Key Lifecycle

```
POST /v1/sandbox
  → pv_test_ key (livemode: false, fixture-only, 7-day TTL)
      │
      ▼
POST /v1/keys/upgrade   (auth: pv_test_ key + github_token from GitHub device flow)
  → pv_live_ key (livemode: true, tier: free_grant, 6,000s grant, 30-day idle expiry)
     (one free grant per GitHub account — duplicate → 409 github_account_already_used)
      │  (grant exhausted → 402 grant_exhausted)
      ▼
POST /v1/keys/request  →  GET /v1/keys/request/{code}  →  billed pv_live_ key
  (device-code claim flow — user approves in browser)
```

---

## What the Model Supports

Design prompts as targeted visual questions. The model natively handles:

| Type | Example | `answer` | `detected_count` | `clips` | Accuracy |
|------|---------|----------|-----------------|--------|---------|
| **Presence** | "Is there a person?" | yes/no/indeterminate | — | ✅ when yes | Stable |
| **Absence** | "Are there no dogs?" | yes/no | — | — | Stable |
| **Action** | "Is someone walking?" | yes/no/indeterminate | — | ✅ when yes | Beta |
| **Count — bare** | "How many people walking?" | — | ✅ integer | — | Stable |
| **Count — threshold** | "More than 2 people?" | yes/no | ✅ integer | — | Stable |
| **Compound** | "Person AND dog?" | yes/no | — | ✅ when yes | Stable |
| **Attribute** | "Is anyone in a red jacket?" | description | — | — | Stable |
| **Location** | "Where is the dog?" | description | — | — | Stable |
| **State** | "Is the door open?" | description | — | — | Stable |
| **Segment** | "Show me where the person is" | — | — | ✅ with masks | Stable |

> **Accuracy note:** Action queries (motion/activity detection) are **beta accuracy** — results may be less reliable than presence/counting queries on short or fast-moving clips.

**Not supported:** audio/sound, identity (who, not what), OCR/text in frame, subjective judgment, exhaustive count over long multi-hour footage.

Open-ended prompts ("Tell me what you see") return a description but have lower accuracy — prefer a targeted question.

Full guide: [Prompts & queries](https://primateintelligence.ai/docs/guides/prompts)

---

## Remote MCP Endpoint

Primate Intelligence exposes a hosted [Model Context Protocol](https://modelcontextprotocol.io) server at:

```
https://api.primateintelligence.ai/mcp
```

Transport: **Streamable HTTP** (MCP spec 2025-03-26+, stateless mode). No npx or local install required — any remote agent can connect directly.

### Authentication

Pass your Primate Vision API key as a Bearer token in every request:

```
Authorization: Bearer pv_live_<your-key>
```

Test keys (`pv_test_…`) also work and return deterministic fixture results. Get a free key:

```bash
# Instant sandbox key (no auth required)
curl -X POST https://api.primateintelligence.ai/v1/sandbox

# Upgrade to a live key with a 6,000-second free credit grant
# (requires a GitHub device-flow token — calling without one returns 403
#  github_verification_required with the client_id + URLs to complete it)
curl -X POST https://api.primateintelligence.ai/v1/keys/upgrade \
  -H "Authorization: Bearer pv_test_<your-sandbox-key>" \
  -H "Content-Type: application/json" \
  -d '{"github_token": "gho_..."}'
```

Requests without a valid key receive a 401 JSON-RPC error with provisioning guidance.

### Available Tools

The remote endpoint exposes the same 10 tools as the npx package:
`create_video_from_url`, `create_analysis`, `validate_analysis`, `create_analysis_batch`, `get_analysis`, `wait_for_analysis`, `list_models`, `get_usage`, `get_credits`, `get_test_fixture`.

- `validate_analysis` — free dry-run (`validate_only: true`): compiled query, assessability, and cost estimate before you spend credits.
- `create_analysis_batch` — 2–10 prompts on one video; the first is full price, each additional is billed at 50%.
- `get_credits` — balance plus the per-analysis transaction ledger (`GET /v1/credits`); prefer it over `get_usage` for auditing what each analysis cost.

Every tool declares an `outputSchema` (visible in `tools/list` and on the static server card at
`/.well-known/mcp/server-card.json`), generated from the same schemas that validate the public /v1
responses — and every result carries `structuredContent` conforming to it, alongside the
human-readable JSON text content.

**`wait_for_analysis` response envelope:** the tool returns `{ analysis, retry }` — the
[Analysis resource](https://primateintelligence.ai/docs/reference#analyses) unmodified under `analysis`,
plus `retry: null` when the analysis reached a terminal state, or
`retry: { reason: "timeout", note }` when the wait expired (call `wait_for_analysis` or
`get_analysis` again). Earlier versions merged an `_mcp_note` field into the analysis object on
timeout; that field is gone.

### Client Configuration Examples

**Claude Desktop / MCP config JSON:**
```json
{
  "mcpServers": {
    "primate-intelligence": {
      "type": "streamable-http",
      "url": "https://api.primateintelligence.ai/mcp",
      "headers": {
        "Authorization": "Bearer pv_live_<your-key>"
      }
    }
  }
}
```

**ChatGPT app directory / custom GPT:**
```
URL: https://api.primateintelligence.ai/mcp
Auth: Bearer token → your pv_live_ or pv_test_ key
```

**OpenAI Agents SDK (Python):**
```python
from agents.mcp import MCPServerStreamableHttp
server = MCPServerStreamableHttp(
    url="https://api.primateintelligence.ai/mcp",
    headers={"Authorization": "Bearer pv_live_<your-key>"},
)
```

### MCP Registry

The server is also listed in the MCP registry at `ai.primateintelligence/mcp` with both the npx stdio package and this remote entry. See `mcp/server.json` in the repository.

---

## Result Contract

```json
{
  "object": "analysis",
  "livemode": true,
  "status": "completed",
  "result": {
    "answer": "yes",
    "confidence": 0.97,
    "detected_count": 3,
    "clips": [{"start_s": 1.2, "end_s": 3.4, "confidence": 0.94}]
  },
  "usage": {
    "billed_seconds": 6,
    "credit_balance_after": 5994
  }
}
```

- `result.answer` ∈ `yes | no | indeterminate`. `indeterminate` = model couldn't commit; don't ship a decision.
- `result.detected_count` — populated for count-intent queries. Integer ≥ 0.
- `result.confidence` ∈ [0, 1]. Zero confidence with no positive detections returns `indeterminate`, not `no`. For count queries, `confidence` applies to the detected count itself — it is the model's confidence that `detected_count` is the correct number, not merely that something was detected. See the [Count queries](#count-queries--the-contract) section for full semantics.
- `result.clips[]` — present for presence/action/compound/segment when `answer: "yes"`. Null otherwise.
- `query.unassessable_components[]` — lists what couldn't be evaluated (e.g. audio). API answers what it can.
- `usage.billed_seconds` — seconds charged for this analysis.
- `usage.credit_balance_after` — the balance immediately after **this** analysis settled. Immutable point-in-time snapshot — it never changes as later analyses run. A `null` value (`snapshot_unavailable`) means the analysis settled before the snapshot feature shipped (migration 075) and the original balance is unknowable; treat it as missing data rather than zero.
- `livemode: true` = real GPU inference. Never relay `livemode: false` results as evidence about real content.
- `origin` ∈ `api | console | system` — how the analysis was created (public API, dashboard upload, or internal). **System-initiated analyses are never billed** — only `api`/`console` analyses reserve and settle credits.
- `narrative` (when created with `options: {narrative: true}`): `{status: "generating"|"ready"|"failed", entries: [{t_s, text}]}` — timestamped event sentences for the video. Generation runs asynchronously **after** the analysis completes: the first completed read may show `status: "generating"` with empty entries — poll the GET until `ready`. Included in the analysis price (no surcharge). Without the opt-in: `narrative: null`, always.
- `artifacts` (completed analyses with an annotated result video): `{annotated_video_url, expires_at}` — a **fresh 1-hour signed URL on every GET**; when it expires, re-fetch the analysis for a new one. `null` when no annotated video exists.

---

## Count queries — the contract

A count query ("how many X…", "more than N X?") succeeds like this:

```json doc-test id=count-contract-success
{
  "answer": "yes",
  "confidence": 0.94,
  "detected_count": 3,
  "clips": [{ "start_s": 1.2, "end_s": 3.4, "confidence": 0.94, "terms": { "person": 0.94 } }],
  "term_confidences": { "person": 0.94 },
  "query_type": "object",
  "video_duration_s": 6.0,
  "indeterminate_reason": null
}
```

And fails like this:

```json doc-test id=count-contract-failure
{
  "answer": "indeterminate",
  "confidence": 0,
  "detected_count": 0,
  "clips": [],
  "term_confidences": {},
  "query_type": "object",
  "video_duration_s": 6.0,
  "indeterminate_reason": "nothing_detected"
}
```

**Rules:**

- `detected_count` is only meaningful when `answer` is determinate (`yes` or `no`).
- `detected_count: 0` with `answer: "indeterminate"` means the pipeline found **nothing assessable** — NOT "zero occurrences".
- A true "zero occurrences" result is `answer: "no"`, `detected_count: 0`.
- **Never branch on `detected_count` without checking `answer` first.**

**Confidence semantics for count queries:** for count queries, `confidence` applies to the detected count itself — it is the model's confidence that `detected_count` is the correct number, not merely that something was detected. `{"confidence": 0.94, "detected_count": 3}` asserts "there are 3" at 94% confidence, not "there is at least one". Numerically it is derived from the per-term detection confidences underlying the count (max across detected clips, clamped to [0, 1]). When `answer` is `indeterminate`, `confidence` is always 0 (enforced by the API regardless of what inference returned); `indeterminate_reason: "nothing_detected"` additionally forces `detected_count` to 0 because no objects were seen at any confidence level. Treat `confidence ≥ 0.7` as reliable for production decisions, `0.4–0.69` as marginal (verify with a second query or tighter prompt), and `< 0.4` as unreliable. Example: `{"answer": "yes", "confidence": 0.94, "detected_count": 3}` means the model is 94% confident the count is exactly 3 — act on it; `{"answer": "indeterminate", "confidence": 0, "detected_count": 0, "indeterminate_reason": "nothing_detected"}` means no objects were detected at any confidence — do not infer "zero occurrences".

---

## Query-Type Maturity

| Query type | Maturity | Notes |
|---|---|---|
| **presence** (`object`) | GA | "Is there X in this video?" — most reliable query form. |
| **counting** (`object` + count intent) | GA | "How many X?" — returns `result.detected_count`. |
| **action** / **temporal** | Beta | "Does X happen during the first N seconds?" — duration-sensitive; result reliability depends on accurate video duration metadata. High `duration_mismatch` rate if source duration is missing. |
| **open-ended** | Rejected | Freeform prompts that don't resolve to a closed yes/no or count question are rejected immediately with `answer: "indeterminate"`, `indeterminate_reason: "unsupported_query_form"`, at **zero cost** — no credits billed. Rephrase as a presence or count query. |

**Agent guidance:**
- Prefer presence and counting queries for production pipelines.
- Action/temporal queries require the source video to have accurate `duration_s` metadata (populated after `POST /v1/videos/:id/complete`).
- If you receive `indeterminate_reason: "unsupported_query_form"`, rewrite the prompt before retrying — retrying the same open-ended form will always fail.

---

## Action pipeline verification

A ground-truth walking fixture is available for integration testing. Use it to verify the full pipeline (upload → analyze → result) against a known answer before deploying to production.

### Walking fixture

| Field | Value |
|-------|-------|
| **URL** | *(URL pending — clip `IMG_3281.MOV` not yet uploaded to CDN; see Slack thread for ETA)* |
| **CDN key** | `fixtures/walking-ground-truth.mov` on `d3silto12vjvss.cloudfront.net` (pending upload) |
| **Scene** | Outdoor walkway; several people walking. Clip is ~6–10 seconds. |
| **Ground truth** | At least 2 people visibly walking during the clip. |
| **Expected ideal API answer** | `POST /v1/analyses` with prompt `"Is someone walking?"` → `answer: "yes"`, `confidence ≥ 0.8`; with prompt `"How many people are walking?"` → `detected_count ≥ 2`, `confidence ≥ 0.7`. |

> **Note:** this fixture is for action-query validation. For basic presence testing, use the sandbox fixture (`POST /v1/sandbox` → `fixture_video_id`).

---

## Batch analyses & discounts

Run 2–10 prompts against the same video in a single request:

```http
POST /v1/analyses/batch
Authorization: Bearer pv_live_<key>

{
  "video_id": "video_01J...",
  "prompts": ["Is there a person?", "How many people are walking?"]
}
```

**Pricing rule:** the first prompt bills at full price; each additional prompt is discounted by `batch_discount_pct` (served by `GET /v1/credit-pricing`; currently **50%**, i.e. additional prompts bill at half price). The discount is config-driven — read it from the endpoint instead of hardcoding. Credits are reserved at the discounted rate, so the discount is observable directly in the ledger (`GET /v1/credits` will show a smaller `seconds_delta` for analysis 2+).

**Worked example:** 6-second video at 1¢/s:
- Prompt 1 (full price): 6s × 1¢ = **6¢**
- Prompt 2 (50% off): 3s × 1¢ = **3¢**
- Prompt 3 (50% off): 3s × 1¢ = **3¢**
- Total for 3 prompts: **12¢** (vs 18¢ if billed separately)

**Response shape:**
```json
{
  "object": "analysis_batch",
  "id": "batch_...",
  "video_id": "video_01J...",
  "analyses": [
    { "id": "analysis_01J...", "status": "queued", ... },
    { "id": "analysis_01J...", "status": "queued", ... }
  ],
  "pricing": {
    "full_price_prompts": 1,
    "discounted_prompts": 1,
    "discount_pct": 50
  }
}
```

Each analysis in `analyses[]` can be polled individually via `GET /v1/analyses/{id}`.

**Listing / filtering analyses:** `GET /v1/analyses` supports `status`, `video_id`, `model`, `created_after`, `created_before` plus cursor pagination (`limit`, `starting_after`). The `status` filter takes the public vocabulary: `queued | preparing | analyzing | rendering | completed | failed | canceled` (use `status=analyzing` for currently-running analyses; unknown values → 400 `validation_failed`).

**Dry-run (`validate_only`):** pass `"validate_only": true` to parse all prompts and get per-prompt cost estimates without reserving credits or creating jobs (HTTP 200):

```json
{
  "object": "analysis_batch_preview",
  "video_id": "video_01J...",
  "prompts": [
    { "index": 0, "query": {...}, "parse_mode": "heuristic", "assessable": true,
      "estimated_seconds": 10, "estimated_cost_usd": 0.10, "discount_pct": 0 },
    { "index": 1, "query": {...}, "parse_mode": "heuristic", "assessable": true,
      "estimated_seconds": 5,  "estimated_cost_usd": 0.05, "discount_pct": 50 }
  ],
  "pricing": {
    "full_price_prompts": 1, "discounted_prompts": 1, "discount_pct": 50,
    "estimated_total_seconds": 15, "estimated_total_cost_usd": 0.15
  }
}
```

Estimates are `null` when the video has no known duration (still processing or URL-sourced before probe completes).

> **Note:** sending a `prompts` array to `POST /v1/analyses` returns a validation error with a pointer to this endpoint.

---

## Streaming (real-time video over WebRTC)

`GET /v1/models` advertises streaming support per model — `darwin-1.3` carries `capabilities: {prompt, structured_query, narrative, streaming: true}`.

Streams analyze live video in real time — same prompt semantics and same result contract as file analyses, delivered per-frame over a signaling WebSocket.

### Lifecycle

```
POST /v1/streams → queued|ready → (WS join → offer/answer/ICE) → live → ended
```

1. `POST /v1/streams {prompt}` (secret key) → returns `signaling.url`, `ice_servers` (STUN + TURN with credentials), `limits`.
2. `POST /v1/client_tokens {scopes: ["streams:signal"], stream_id, ttl_s}` → `pvct_` token for the device. The signaling WS **never** accepts secret keys.
3. Connect `signaling.url?token=pvct_…`, send `join`, receive `ready` (or `queued {position}`), then standard WebRTC offer/answer + **bidirectional trickle ICE** (`ice {candidate}` messages flow both ways — the server trickles late-gathered srflx/relay candidates after its answer; keep consuming them).
4. `live` → `result {frame_num, detections}` per analyzed frame, `metering {elapsed_s, billed_s, session_remaining_s}` every 5s, `warning {remaining_s}` before credit exhaustion, `end {reason}`.
5. Mid-stream, send `{"type": "update_prompt", "prompt": "…"}` (client→server, ≤2000 chars) to change the question without reconnecting — the server recompiles it, applies it to the live engine, and confirms with `{"type": "prompt_updated"}`. Subsequent result frames echo the new prompt. Invalid prompts return `{"type": "error", "code": "validation_failed" | "parse_failed"}` and the old prompt stays active. On narrative-opted-in streams, expect `status` events (e.g. `recalculating`) as the engine rebuilds context after the update.

### Metering tick fields

Every 5s while live: `{type: "metering", elapsed_s, billed_s, session_remaining_s, balance_s}`.

- `session_remaining_s` — seconds remaining in **this session's** credit reservation (session cap), NOT your account credit balance. Account balance lives at `GET /v1/billing/credits` (`balance_seconds`).
- `balance_s` — **deprecated** alias of `session_remaining_s` (identical value). Removed **~2026-08-28**; migrate reads to `session_remaining_s`.
- `elapsed_s` / `billed_s` — live-clock seconds elapsed = billed (identical by construction; join/negotiation free).

### Result sampling (results are sampled, not per-frame)

Results are **sampled**, not emitted for every source frame — inference cadence is adaptive (roughly every 8th source frame under load). `frame_num` is the **source-frame index** the result was computed on (so gaps between consecutive `frame_num` values are normal), and `results_summary.result_frames` on the terminal resource counts the number of **result events emitted** — the two are different axes. Expect results at roughly 8–15/s depending on load; do not assume a fixed cadence.

### No-media warning (the server tells you when YOUR media is the problem)

If the WebRTC transport connects but **no decodable video frame arrives within 5 seconds**, the server pushes a `warning` event on the signaling WS (re-warned once at 15s, then quiet):

```json
{"type": "warning", "code": "no_media_frames", "transport_connected": true,
 "elapsed_s": 5, "packets_received": 312, "frames_decoded": 0, "hint": "…"}
```

Read `packets_received` to self-diagnose:
- `packets_received: 0` — your client is **not sending media** (track not attached, muted, or the capture source is dead).
- `packets_received > 0` with `frames_decoded: 0` — media is arriving but is **not decodable** (wrong codec — must be VP8 or H.264 — or the source produces no real frames, e.g. a canvas capture without an active draw loop).

If the session then ends without ever going live, the terminal resource records `end_reason: "media_timeout"` with the same counters in `failure_diagnostic`, and bills 0.

### Terminal results_summary

The ended stream resource carries `results_summary: {result_frames, frames, last_detections}`:

- `result_frames` — number of result events emitted over the session (sampled — see “Result sampling” above; this is NOT a source-frame count).
- `frames` — **deprecated** alias of `result_frames` (identical value). Removed **~2026-08-28**; migrate reads to `result_frames`.
- `last_detections` — the final `result.detections[]` rows, same contract as live results.

### Streaming result contract (identical to file analyses — transport never changes enums)

Each `result.detections[]` row:

- `answer`: `"yes" | "no" | "indeterminate"` — lowercase, same enum as file-API results
- `confidence`: 0..1; `term_confidences`: per-term map
- `prompt`: echoed **exactly as you submitted it** (byte-identical)
- `query_type`, `search_terms`, `prompt_intent`: same vocabulary as `POST /v1/parse`
- `frame_num`, `elapsed_s`, `session_id`, `session_fps`
- `timing`: server-side latency telemetry (per-stage breakdowns: inference, encode, per-hop p50s). Rich, production-grade, safe to log — field names may grow, existing names are stable.
- `narrative_update` (only when the stream was created with `options: {narrative: true}`): `{t_s, text}` — a new narrative sentence, emitted event-driven when the engine detects an appearance/disappearance/action, NOT per frame. Absent (not `null`) on frames without one. Terminal `results_summary.last_detections` never carries it.

### Status events (narrative-opted-in streams only, PRI-496)

Streams created with `options: {narrative: true}` also receive a `status` server→client event on the signaling WS:

```json
{"type": "status", "status": "prompt_context" | "combined_prompt" | "recalculating", "message": "…"}
```

Use these to order/annotate narrative entries (e.g. mark when the engine is recalculating context after a prompt update). The vocabulary is a **closed set** — exactly these three values; internal engine statuses never leak. Streams without the narrative opt-in never receive `status` events (their event vocabulary is unchanged).

### Session recordings (PRI-496)

Create the stream with **`recording: true`** (top-level boolean) to retrieve the server-side session recording afterwards:

1. The stream resource carries `recording: {status}` — `recording` while live, `available` once ended with a stored recording, `failed` if capture failed, `none` if nothing was stored. Streams without the opt-in have `recording: null`.
2. Once ended with `recording.status: "available"`, call **`GET /v1/streams/{id}/recording`** → `{url, expires_at, content_type: "video/h264", container: "h264-annex-b"}`. The URL is **signed fresh on every GET with a 1-hour TTL** — re-fetch for a new one; old recordings always stay retrievable.
3. The recording is a raw H.264 Annex-B elementary stream (the exact annotated frames the client saw). Lossless remux to MP4: `ffmpeg -i recording.h264 -c copy recording.mp4`.

### end_reason vocabulary (honest by construction)

| Reason | Meaning | Billed? |
|---|---|---|
| `completed` | Normal end after the session was live | live seconds |
| `canceled` | Ended before ever going live (client action) | 0 |
| `ice_failed` | WebRTC ICE never connected — see `failure_diagnostic` | 0 |
| `media_timeout` | Transport connected but no media/results flowed | 0 |
| `insufficient_credits` | Balance exhausted mid-stream | live seconds |
| `timeout` | `limits.max_session_s` cutoff reached | live seconds |
| `error` | Server-side failure | live seconds (0 if never live) |

**A stream that never went live is never `completed`** — the API enforces this. On `ice_failed`, the terminal resource carries `failure_diagnostic` (`{local_candidates, hint}`) — the server-side ICE candidate summary. If `local_candidates` shows only private addresses (`172.x`, `10.x`), the fault is server-side config; if it shows srflx/relay candidates, check your client's network path to the TURN servers in `ice_servers`.

### Client profiles that MUST work (and are CI-tested)

- Browser on home/office NAT (webcam) — the easy case
- **Datacenter/CI/edge-fleet clients: UDP-blocked, TURN-over-TCP:443 relay-only** — the demanding case. The server advertises a public host candidate and srflx/relay candidates (trickled when gathering outlasts the answer). Regression-tested every deploy with a relay-only aiortc client.

### Stream a file (regression harness recipe)

Re-running a known file through the live path is the natural streaming regression test. The examples repo ships `stream_file.py` (aiortc): loops a video file as the WebRTC source, applies your prompt, and audits the full session (candidates, states, results, billing) to JSON. See `python/streaming/` in the examples repo:

**https://github.com/Primate-Intelligence/primate-examples** (public — no auth needed)

### Billing

Billed per second of **live clock time** (join/queue/negotiation free), from the same credit ledger as uploads. Sessions that never go live bill exactly 0. Metering ticks arrive every 5s; `usage.billed_seconds` on the terminal resource is the reconciled charge.

---

## Pricing

Billing is metered in **source-clock video-seconds** — the duration of video analyzed, independent of resolution or fps. Current rates are discoverable (public, no auth):

```http
GET /v1/credit-pricing
```

Key fields:

| Field | Meaning |
|-------|---------|
| `price_per_second_cents` | Price per billed video-second, in cents |
| `signup_grant_seconds` | Free credit-seconds on signup / free-grant upgrade |
| `allowed_purchase_cents` | Preset top-up amounts |
| `batch_discount_pct` | Percent discount for each prompt after the first in `POST /v1/analyses/batch` |
| `batch_min_prompts` / `batch_max_prompts` | Allowed prompts-per-batch range |

**Cost of an analysis** = `usage.billed_seconds × price_per_second_cents`.

Worked example: a 6-second analysis at 1¢/second → `6 × 1 = 6` cents = **$0.06**.

Always read rates from the endpoint rather than hardcoding — pricing is config-driven and can change without an API version bump.

---

## API versioning

Every response — success, error, even 404 — carries an `X-Api-Version` header:

```
X-Api-Version: <package-version>+<git-sha7>
```

Example: `X-Api-Version: 0.1.0+a7c1993`. The value is computed once at boot and is stable for the lifetime of a deploy.

**Agent guidance:** if the value changes between two requests in the same session, a deploy happened mid-session. Re-check the [changelog](https://api.primateintelligence.ai/docs/changelog.md) before attributing new behaviour to a bug in your integration.

---

## Rate Limits

All public endpoints respond with `X-RateLimit-*` headers. Retry after `Retry-After` on `429`/`503`.

Sandbox/upgrade provisioning is additionally IP-rate-limited (3 sandbox provisions / IP / 24h; configurable global daily cap on free-grant upgrades).

---

## Changelog

All API behaviour changes are recorded in [`/docs/changelog.md`](https://api.primateintelligence.ai/docs/changelog.md), sorted newest-first. Subscribe via RSS: [`/docs/changelog.xml`](https://api.primateintelligence.ai/docs/changelog.xml).

This file and the changelog ship inside the API deploy artifact itself — `https://api.primateintelligence.ai/docs/agents.md` is the canonical copy, and `https://primateintelligence.ai/docs/agents.md` serves the same bytes (the website proxies the API). Spec, quickstart, and changelog therefore update atomically with the code they describe.

**Agent guidance:** check the changelog when `X-Api-Version` changes between requests in the same session — a deploy happened mid-session and a new feature or fix may affect your integration.


---

<!-- source: https://www.primateintelligence.ai/docs/guides/uploading.md -->

# Uploading video

> Three ways to get video in — presigned upload, URL ingest, SDK helper — and how to choose.


There are three paths to get video into Primate Vision. All of them end with a `video` resource in status `ready`, at which point you can create analyses against it.

**Formats:** `video/mp4` (H.264) and `video/quicktime`. **Max size:** 2 GiB. **Retention:** source videos are deleted 30 days after upload (see [security model](/docs/guides/security-model)).

## Choosing a path

| Path | Use when |
|---|---|
| **URL ingest** | The video is already at a public https URL (CDN, S3, YouTube-dl output) |
| **Presigned upload** | You have the bytes (server file, browser file input) — bytes go straight to S3, never through our API servers |
| **SDK `upload()` helper** | You use an SDK and want the presigned dance done for you |

## URL ingest

One call. The API fetches the video asynchronously:

```bash doc-test id=uploading-url skip-live
curl -s -X POST https://api.primateintelligence.ai/v1/videos \
  -H "Authorization: Bearer $PRIMATE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/clip.mp4"}'
```

The video starts `processing` and transitions to `ready` (or `failed` with `error.code` = `url_fetch_failed` / `url_forbidden` / `video_unreadable`). Poll `GET /v1/videos/{id}` or subscribe to the `video.ready` webhook.

URL rules (SSRF policy): `https` only, port 443, public hosts only, max 3 redirects, 2 GiB cap. Private/internal addresses are rejected with `url_forbidden`.

## Presigned upload

Three steps — create, PUT, complete:

```bash
# 1. Create — declares filename/type/size, returns a presigned S3 PUT URL
curl -s -X POST https://api.primateintelligence.ai/v1/videos \
  -H "Authorization: Bearer $PRIMATE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"filename": "clip.mp4", "content_type": "video/mp4", "size_bytes": 12345678}'

# 2. PUT the bytes directly to S3 (the upload.url + upload.headers from step 1)
curl -s -X PUT "$UPLOAD_URL" -H "Content-Type: video/mp4" --data-binary @clip.mp4

# 3. Tell the API the PUT finished
curl -s -X POST https://api.primateintelligence.ai/v1/videos/$VIDEO_ID/complete \
  -H "Authorization: Bearer $PRIMATE_API_KEY"
```

The presigned URL expires (see `upload.expires_at`, typically 1h). If the declared `size_bytes` doesn't match what landed in S3, `complete` fails with `upload_incomplete` — re-create the video.

## SDK helper

```typescript
import Primate from '@primate-intelligence/sdk';
import { readFile } from 'fs/promises';

const client = new Primate();
const video = await client.videos.upload(await readFile('clip.mp4'), {
  filename: 'clip.mp4',
  content_type: 'video/mp4',
});
```

```python
from primate_intelligence import Primate

client = Primate()
with open("clip.mp4", "rb") as f:
    video = client.videos.upload(f.read(), filename="clip.mp4")
```

Both run create → PUT → complete and return the video resource.

## Browser uploads

Never put a secret key in a browser. Mint an ephemeral [client token](/docs/guides/security-model#client-tokens) server-side with the `videos:write` scope, then run the presigned flow from the browser with the `pvct_` token. Working example: the [browser SPA sample](https://github.com/Primate-Intelligence/primate-examples).

## Lifecycle

```
awaiting_upload → uploading → processing → ready
                                        ↘ failed
```

`DELETE /v1/videos/{id}` removes the video (409 `resource_conflict` while analyses are still running against it). Deletion propagates from S3 + CDN within 72h; signed URLs expire within 1h regardless.

## Your video library (playback + history)

Uploaded videos stay usable as a library — three fields/endpoints together give full list-replay-history semantics:

- **`Video.media`** — ready videos carry `media: {url, expires_at}`, a signed playback URL for the original source video. Sign-on-read: generated **fresh on every GET with a 1-hour TTL** — re-fetch the video for a new URL after expiry; old videos always stay playable. `media` is `null` for non-ready videos and sandbox fixture videos.
- **`GET /v1/videos/{id}/analyses`** — every analysis ever run against a video, newest first, with the same list envelope + filters (`status`, `limit`, `starting_after`) as `GET /v1/analyses`. (Equivalent spelling: `GET /v1/analyses?video_id={id}`.)
- **`Analysis.artifacts`** — completed analyses with an annotated result video carry `annotated_video_url` (same sign-on-read semantics).

---

<!-- source: https://www.primateintelligence.ai/docs/guides/prompts.md -->

# Prompts & queries

> Free-text prompts vs structured queries, what's answerable, and how unassessable components behave.


Every analysis is a **question about a video**. You can ask it two ways: free text (`prompt`) or a pre-compiled structured query (`query`). Provide exactly one.

## What Primate can answer

The model supports ten distinct prompt types. The table below shows what to expect from each.

| Type | Example prompt | `answer` | `detected_count` | `clips` |
|------|---------------|----------|-----------------|---------|
| **Presence** | "Is there a person?" | yes / no / indeterminate | — | ✅ when yes |
| **Absence** | "Are there no dogs?" | yes / no | — | — |
| **Action** | "Is someone walking?" | yes / no | — | ✅ when yes |
| **Count — bare** | "How many people are in this video?" | — | ✅ integer | — |
| **Count — threshold** | "Are there more than 2 people?" | yes / no | ✅ integer | — |
| **Compound** | "Is there a person AND a dog?" | yes / no | — | ✅ when yes |
| **Attribute** | "Is anyone wearing a red jacket?" | description | — | — |
| **Location** | "Where is the dog?" | description | — | — |
| **State** | "Is the door open?" | description | — | — |
| **Segment** | "Show me where the person is" | — | — | ✅ with masks |

**Open-ended prompts** ("Tell me what you see") return a description but have lower accuracy than targeted queries. Prefer a specific question.

**Not supported:**
- Audio / sound ("Is there loud noise?")
- Identity ("Is this John?")
- OCR / text in frame ("What does the sign say?")
- Subjective judgment ("Does the room look nice?")
- Exhaustive counting over long footage ("Count every bird in this 2-hour archive") — count queries are accurate on a single clip or short segment, not long multi-hour recordings

---

## Free-text prompts

```json
{ "video_id": "video_…", "prompt": "Is there a person near the door?" }
```

The server compiles your prompt into a structured query (the `query` field on the analysis shows exactly what it understood, `parse_mode` shows how: `llm` or `heuristic`). Max 2000 characters.

### Prompt design tips

**Do this:**
- "Is there a person near the door?" — presence
- "Are there more than two people in frame?" — count + threshold, returns `yes`/`no` + `detected_count`
- "How many people are walking?" — bare count, returns `detected_count` + natural-language narrative
- "Does anyone fall down?" — action
- "Is there a person AND a dog?" — compound
- "Is anyone wearing a red jacket?" — attribute
- "Is the door open or closed?" — state
- "Where is the bicycle relative to the car?" — location

**Avoid this:**
- "Tell me everything that happens in the video" — open-ended; use a targeted question instead
- "Count every pedestrian across this 30-minute file" — count works best on clips, not long recordings
- "Is this suspicious activity?" — subjective; rephrase as a specific visual condition

**One question per analysis beats a paragraph.** If you need to answer "Is there a person?" AND "Are there more than 2 people?", submit two analyses — the answers are independent and the second one gives you `detected_count`.

---

## The answer contract

### `result.answer`

`result.answer` is one of `yes`, `no`, `indeterminate`:

| Value | Meaning |
|-------|---------|
| `yes` | The condition is present with meaningful confidence |
| `no` | The condition was not detected |
| `indeterminate` | The model could not commit either way — treat as "don't ship a decision on this" |

Count queries (bare "how many?" form) do not populate `result.answer`. Use `result.detected_count` instead.

### `result.detected_count`

Populated for `count`-intent analyses (when the prompt asks "how many?" or uses a threshold):

```json
{
  "result": {
    "answer": "yes",
    "confidence": 0.94,
    "detected_count": 3,
    "clips": [...]
  }
}
```

For count + threshold queries, `answer` tells you whether the threshold was met; `detected_count` tells you the actual number observed.

### `result.confidence`

`result.confidence` ∈ [0, 1]. Meaningful confidence is ≥ 0.5. A result with `confidence: 0` and no positive term detections returns `answer: "indeterminate"`, not `"no"`.

### `result.clips[]`

Clip objects carry `start_s`, `end_s`, `confidence`. Present for presence, action, compound, and segment queries when the answer is `yes`. Null or absent for description/count responses.

### `result.query_type`

How the question was classified: `object`, `action`, `compound`, `attribute`, or `open_ended`.

### `narrative` (opt-in)

Create the analysis with `options: {narrative: true}` and the completed analysis carries a top-level `narrative` object — timestamped event sentences describing what happened in the video:

```json
{"narrative": {"status": "ready", "entries": [{"t_s": 3.2, "text": "A person enters from the left."}]}}
```

Generation runs **asynchronously after the analysis completes**: the first completed read may show `status: "generating"` with empty entries — keep polling the GET until `ready` (or `failed`). Included in the analysis price, no surcharge. Without the opt-in, `narrative` is always `null`. (Streams have a live equivalent — see [Streaming § Live narrative](/docs/guides/streaming#live-narrative-opt-in).)

---

## Unassessable components

If part of your question can't be assessed, the API **answers what it can and tells you what it couldn't** — it never silently drops half your question:

```json
{
  "prompt": "Is there a person talking loudly?",
  "query": {
    "search_terms": ["person"],
    "unassessable_components": ["talking loudly (audio)"]
  },
  "result": { "answer": "yes", "confidence": 0.91 }
}
```

Check `query.unassessable_components` when your prompts mix visual and non-visual language.

---

## Structured queries

Skip the compiler and send the compiled form directly (`parse_mode: "client"`). Useful when you generate queries programmatically or need exact control:

```json
{
  "video_id": "video_…",
  "query": {
    "search_terms": ["person", "dog"],
    "conditions": [
      { "type": "presence", "target": "person", "negated": false },
      { "type": "presence", "target": "dog", "negated": false }
    ],
    "connective": "and",
    "query_type": "compound",
    "prompt_intent": "presence",
    "response_framing": "yes_no"
  }
}
```

Key structured query fields:

| Field | Values | Description |
|-------|--------|-------------|
| `query_type` | `object` `action` `compound` `open_ended` | Top-level classification |
| `prompt_intent` | `presence` `absence` `count` `action` `segment` | What the model optimizes for |
| `response_framing` | `yes_no` `description` `count` | Shape of the result |
| `count_terms` | string[] | What to count (for count intent) |
| `count_threshold` | integer \| null | Threshold for threshold comparison |
| `count_comparison` | `gt` `gte` `lt` `lte` `eq` \| null | How to compare against threshold |
| `quantifier` | `{operator, value}` \| null | Alternative to count_threshold |

If the body fails the CompiledQuery schema you get `400 query_invalid` with `param` naming the field. If the compiler can't parse a free-text prompt you get `422 parse_failed` — rephrase, or fall back to a structured query.

---

## Prompt errors

| Code | Meaning |
|------|---------|
| `prompt_empty` | Provide `prompt` or `query` |
| `prompt_too_long` | > 2000 chars |
| `query_invalid` | Structured query fails the schema (`param` names the field) |
| `parse_failed` | Compiler couldn't produce a query — retryable after rephrasing |

---

<!-- source: https://www.primateintelligence.ai/docs/guides/waiting.md -->

# Waiting for results

> Poll vs Prefer: wait vs webhooks — pick the right completion strategy.


Analyses run asynchronously. There are three ways to learn when one finishes; pick by context.

| Strategy | Use when |
|---|---|
| **`Prefer: wait`** | Interactive flows, scripts, first success experience — simplest |
| **Polling** | You want progress bars / queue position, or waits longer than 120s |
| **Webhooks** | Production pipelines, high volume, serverless — no held connections |

## `Prefer: wait` (sync sugar)

Add one header to the create and the API holds the connection until the analysis reaches a terminal state (or the wait cap):

```bash doc-test id=waiting-prefer env=VIDEO_ID
curl -s -X POST https://api.primateintelligence.ai/v1/analyses \
  -H "Authorization: Bearer $PRIMATE_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait=60" \
  -d '{"video_id": "'"$VIDEO_ID"'", "prompt": "Is there a person in this video?"}'
```

- Cap: **120 seconds**. If the analysis isn't terminal by then you get the current (running) resource back — fall through to polling.
- Test-mode analyses complete in seconds, so `Prefer: wait` virtually always returns the finished result.
- Under capacity brownouts `Prefer: wait` may be temporarily disabled (you get an immediate 202-style response with the running resource) — code must handle a non-terminal response either way.

The SDK helpers `createAndWait()` (TS) / `create_and_wait()` (Python) use this header, then fall back to polling automatically.

## Polling

```bash
curl -s https://api.primateintelligence.ai/v1/analyses/$ANALYSIS_ID \
  -H "Authorization: Bearer $PRIMATE_API_KEY"
```

While running, the resource carries honest progress signals:

```json
{
  "status": "queued",
  "queue_position": 3,
  "progress": null
}
```

`queue_position` and `estimated_start_s` are accurate to within ±50% — we publish transparency rather than an analysis-latency SLO.

**Backoff:** poll at 1–2s intervals for interactive flows; add jitter for fleets. Terminal states are `completed`, `failed`, `canceled` — anything else keeps polling.

## Webhooks

Register once, get an HTTPS POST on every terminal transition:

```bash
curl -s -X POST https://api.primateintelligence.ai/v1/webhook_endpoints \
  -H "Authorization: Bearer $PRIMATE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://yourapp.com/webhooks/primate", "events": ["analysis.completed", "analysis.failed"]}'
```

See the [webhooks guide](/docs/guides/webhooks) for verification, retries, and the #1 integration rule (the webhook is a notification, not the source of truth).

## Cancellation

```bash
curl -s -X POST https://api.primateintelligence.ai/v1/analyses/$ANALYSIS_ID/cancel \
  -H "Authorization: Bearer $PRIMATE_API_KEY"
```

Best-effort once the analysis is `analyzing`; a 409 `analysis_not_cancelable` means it already reached a terminal state. You are not billed for canceled work that never ran.

---

<!-- source: https://www.primateintelligence.ai/docs/guides/webhooks.md -->

# Webhooks

> Standard Webhooks signing, verification, retries, redelivery, and the integration rules that matter.


Webhook endpoints receive an HTTPS POST whenever a subscribed event fires. Event vocabulary (closed, v1): `video.ready`, `video.failed`, `analysis.completed`, `analysis.failed`, `analysis.canceled`, `stream.ended`.

## Rule #1: the webhook is a notification, not the source of truth

Payloads over 256 KiB are truncated (`data.truncated: true`). Delivery is **at-least-once with no ordering guarantee**. The reliable pattern is always:

1. Receive the event → verify the signature → dedupe on the `webhook-id` header
2. `GET` the resource by id for the authoritative state
3. Act on that

Never assume `analysis.completed` arrives before (or after) your poll sees `completed`.

## Register an endpoint

```bash
curl -s -X POST https://api.primateintelligence.ai/v1/webhook_endpoints \
  -H "Authorization: Bearer $PRIMATE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://yourapp.com/webhooks/primate", "events": ["analysis.completed", "analysis.failed"]}'
```

The response contains the signing secret (`whsec_…`) **exactly once**. Store it. Max 10 endpoints per account.

## Delivery contract

- Body: `{"id": "evt_…", "type": "analysis.completed", "created_at": "…", "data": {…full resource DTO…}}`
- Timeout **10s**; only 2xx counts as delivered (3xx = failure)
- Retry schedule: `1m, 5m, 30m, 2h, 8h, 24h` with jitter; then the event dead-letters (visible in the deliveries API for 30 days)
- 72h of 100% failure → the endpoint auto-disables + you get an email. Re-enable with `POST /v1/webhook_endpoints/{id}/enable`

## Verify signatures

Deliveries are signed per [Standard Webhooks](https://www.standardwebhooks.com). Three headers: `webhook-id`, `webhook-timestamp`, `webhook-signature` (`v1,<base64 HMAC-SHA256>`; multiple space-separated during rotation). Signed content is `` `${id}.${timestamp}.${rawBody}` ``; the HMAC key is the base64-decoded part of your `whsec_` secret. Reject skew > 5 minutes; compare constant-time.

Both SDKs ship verifiers:

```typescript
import { verifyWebhookRequest } from '@primate-intelligence/sdk';

app.post('/webhooks/primate', (req, reply) => {
  verifyWebhookRequest({
    secret: process.env.PRIMATE_WEBHOOK_SECRET!,
    headers: req.headers,
    rawBody: req.rawBody,          // MUST be the raw bytes, not re-serialized JSON
  });                              // throws on failure
  const event = JSON.parse(req.rawBody);
  if (alreadyProcessed(event.id)) return reply.code(204).send(); // dedupe on evt_ id
  // … fetch the resource, act …
  reply.code(204).send();
});
```

```python
from primate_intelligence import verify_webhook_request

@app.post("/webhooks/primate")
async def receive(request: Request):
    raw = await request.body()
    verify_webhook_request(
        secret=os.environ["PRIMATE_WEBHOOK_SECRET"],
        headers=dict(request.headers),
        raw_body=raw,              # raw bytes, not re-serialized JSON
    )                              # raises ValueError on failure
    event = json.loads(raw)
    ...
    return Response(status_code=204)
```

> Per-request `webhook` overrides on `POST /v1/analyses` / `POST /v1/streams` have no secret exchange and are delivered **unsigned** — register an endpoint when you need verification.

## Rotate secrets

```bash
curl -s -X POST https://api.primateintelligence.ai/v1/webhook_endpoints/$ID/rotate_secret \
  -H "Authorization: Bearer $PRIMATE_API_KEY"
```

Both old and new secrets sign deliveries for **24h** (the signature header carries two entries); verification passes when any signature matches *your* secret. Full key-compromise procedure: [security model](/docs/guides/security-model).

## Debug deliveries

```bash
# last 100 deliveries with response codes
curl -s https://api.primateintelligence.ai/v1/webhook_endpoints/$ID/deliveries \
  -H "Authorization: Bearer $PRIMATE_API_KEY"

# redeliver a failed/dead one now
curl -s -X POST https://api.primateintelligence.ai/v1/webhook_endpoints/$ID/deliveries/$DELIVERY_ID/redeliver \
  -H "Authorization: Bearer $PRIMATE_API_KEY"
```

## Local development

Use a tunnel (ngrok, cloudflared) so the API can reach your dev machine, or skip webhooks locally and use `Prefer: wait` — the two strategies are drop-in replacements for the completion signal.

---

<!-- source: https://www.primateintelligence.ai/docs/guides/errors-retries.md -->

# Errors & retries

> The error envelope, the closed registry, and exactly what to retry.


Every non-2xx response uses one envelope:

```json
{
  "error": {
    "code": "insufficient_credits",
    "message": "Insufficient credits. Add credits or enable auto-refill.",
    "status": 402,
    "param": null,
    "docs_url": "https://primateintelligence.ai/docs/errors#insufficient_credits",
    "request_id": "req_01H…"
  }
}
```

- **`code`** — stable, machine-readable, from a closed registry. Codes are append-only; meanings never change.
- **`docs_url`** — deep link to the [error registry page](/docs/errors) anchor for that code.
- **`request_id`** — include it in support requests; it indexes our logs.
- **`error.errors[]`** — on `validation_failed`, every violation is listed (param + message), not just the first.
- **`details`** — structured extras (e.g. `details.meter` on `quota_exceeded`).

## The registry is machine-readable

```bash doc-test id=errors-registry
curl -s https://api.primateintelligence.ai/v1/errors
```

Each entry carries two retry flags:

- **`retryable`** — safe to retry the same request unchanged (with backoff)
- **`idem`** — idempotent to retry with the same `Idempotency-Key`

The SDKs generate their retry policy from this table — if you build your own client, do the same and your retry logic can never drift from the API.

## What to retry

| Situation | Codes | Strategy |
|---|---|---|
| Rate limited | `rate_limit_exceeded`, `concurrency_limit_exceeded` | Wait `Retry-After` seconds, retry |
| Platform transient | `inference_unavailable`, `db_unavailable`, `internal_error`, `capacity_exhausted`, `idempotency_unavailable` | Exponential backoff + jitter, same `Idempotency-Key` |
| URL fetch flake | `url_fetch_failed` | Retry the create; check the source URL is reachable |
| Prompt didn't compile | `parse_failed` (422) | Rephrase the prompt or send a structured `query` |
| Everything else | `validation_failed`, `invalid_api_key`, `insufficient_credits`, … | **Do not retry unchanged** — fix the cause |

## Idempotency (never double-create)

Send `Idempotency-Key` (any string, 1–255 chars; UUIDv4 recommended) on `POST /v1/videos`, `/v1/analyses`, `/v1/webhook_endpoints`:

- Same key + same body within 24h → the stored response replays with `Idempotency-Replayed: true`
- Same key + **different** body → `409 idempotency_key_reused`
- Body comparison is canonical (JCS) — key order / whitespace / number formatting never false-positive

The SDKs auto-generate keys on create calls, which makes network-level retries safe by default.

## Resource-level errors

Two codes never appear as HTTP errors — only inside a failed resource's `error` field:

- **`inference_error`** — the model failed on this input; safe to create a new analysis
- **`stuck_timeout`** — the platform lost the job; not billed; resubmit

Check `analysis.error` / `video.error` when `status` is `failed`.

### Free re-runs after platform incidents

Failed analyses carry **`rerun_eligible: boolean`** (present only on `failed` status) — `true` when the failure occurred during a declared platform incident or was flagged by ops. When eligible, **`POST /v1/analyses/{id}/rerun`** creates a **fresh analysis at no charge** — same video, prompt, model, and options; new id; `usage` stays `null` (never billed). One free re-run per failed analysis; calling on an ineligible analysis returns `409 rerun_not_eligible`. Check `rerun_eligible` before writing your own retry — a free re-run beats paying for a resubmit.

## A worked retry loop (Python, no SDK)

```python
import time, requests

def create_analysis(body, key, attempts=4):
    idem = str(uuid.uuid4())
    for attempt in range(attempts):
        r = requests.post(
            "https://api.primateintelligence.ai/v1/analyses",
            json=body,
            headers={"Authorization": f"Bearer {key}", "Idempotency-Key": idem},
        )
        if r.ok:
            return r.json()
        err = r.json()["error"]
        registry = requests.get("https://api.primateintelligence.ai/v1/errors").json()
        flags = next(e for e in registry["data"] if e["code"] == err["code"])
        if not flags["retryable"] or attempt == attempts - 1:
            raise RuntimeError(f"{err['code']}: {err['message']} ({err['request_id']})")
        time.sleep(int(r.headers.get("Retry-After", 2 ** attempt)))
```

(Or just use the SDK — this is exactly what it does, with the registry vendored at build time.)

## See also

- [Error registry](/docs/errors) — every code, one anchor each (the `docs_url` targets)
- [Rate limits & quotas](/docs/guides/rate-limits)
- [Billing & credits](/docs/guides/billing) — handling `insufficient_credits` gracefully

---

<!-- source: https://www.primateintelligence.ai/docs/guides/rate-limits.md -->

# Rate limits & quotas

> Limit classes, headers, 429 handling, and how quotas differ from rate limits.


## Rate limits

Applied per API key, sliding window:

| Class | Limit | Covers |
|---|---|---|
| `reads` | 600/min | All GETs |
| `writes` | 300/min | POST/DELETE except analysis creation |
| `analyses.create` | 60/min | `POST /v1/analyses` |

Plan multipliers can raise these. Every response — including 429s — carries:

```
X-RateLimit-Limit: 600
X-RateLimit-Remaining: 597
X-RateLimit-Reset: 1751980860
```

A 429 adds `Retry-After` (integer seconds, ≥ 1). **Honor it** — the SDKs do automatically.

```json
{ "error": { "code": "rate_limit_exceeded", "status": 429, "message": "Rate limit exceeded. Back off per Retry-After." } }
```

## Concurrency limits

Separate from request rates: your plan caps **simultaneous running analyses**. Exceeding it returns `429 concurrency_limit_exceeded` — wait for an active analysis to finish rather than backing off blindly.

## Quotas (metered limits)

Quotas are about *how much* you process, not how fast you call. Exceeding a metered limit returns `429 quota_exceeded` with `details.meter` naming the meter. Check consumption any time:

```bash doc-test id=rate-limits-usage
curl -s https://api.primateintelligence.ai/v1/usage \
  -H "Authorization: Bearer $PRIMATE_API_KEY"
```

```json
{
  "meters": [
    { "meter": "credit_seconds", "unit": "seconds", "balance": 5990 },
    { "meter": "seconds_processed.period", "unit": "seconds", "used": 10, "resets_at": "2026-08-01T00:00:00Z" }
  ]
}
```

Running out of credits is `402 insufficient_credits` (not 429) — see [billing & credits](/docs/guides/billing).

## Capacity

Under platform-wide pressure new creates may get `503 capacity_exhausted` with `Retry-After` while in-flight work drains. Test-mode traffic is never shed (it doesn't touch the GPU). See [service expectations](/docs/guides/versioning#service-expectations).

## Client checklist

1. Honor `Retry-After` on every 429/503 — never hammer
2. Watch `X-RateLimit-Remaining` and pre-throttle when it gets low
3. Treat `concurrency_limit_exceeded` as "wait for completion", not "back off"
4. Treat `quota_exceeded`/`insufficient_credits` as account states — alert a human, don't retry
5. Single region (us-west-2) in v1 — no cross-region limit ambiguity

---

<!-- source: https://www.primateintelligence.ai/docs/guides/test-mode.md -->

# Test mode

> pv_test_ keys, deterministic fixtures, and CI verification without burning quota.


Every account has two key modes. They hit the same API surface with the same code — only the backend differs.

| | `pv_test_` | `pv_live_` |
|---|---|---|
| Getting one | `POST /v1/sandbox` — instant, anonymous | Signup + card |
| Results | Deterministic, canned, fixture corpus | Real GPU inference |
| Credits | Can never hold or spend credits | Metered |
| Speed | Analyses complete in seconds | Workload-shaped |
| Expiry | 7 days (sandbox) | Until revoked |

## Instant sandbox

```bash doc-test id=test-mode-sandbox
curl -s -X POST https://api.primateintelligence.ai/v1/sandbox
```

No email, no card, no human. The response includes the key, a pre-seeded fixture video id, the fixture prompt, and the expected answer. IP-limited (default 3 provisions per 24h → `429 sandbox_limit_exceeded`).

This is the zero-touch path for agents and CI: build and verify the *entire* integration before any human signs up. Graduating to live is a **billing gate, not an integration gate** — your code doesn't change.

## Deterministic fixtures

On test keys, analyses against the fixture corpus return canned results — same input, same output, every time. That makes them assertable in CI:

```bash doc-test id=test-mode-fixture
curl -s https://api.primateintelligence.ai/v1/test-fixture
```

```json
{
  "name": "presence",
  "test_video_url": "https://primateintelligence.ai/empty-state/forklift-demo.mp4",
  "test_prompt": "Is there a forklift in this video?",
  "expected_answer": "yes",
  "expected_confidence_min": 0.8
}
```

## CI verification pattern

```python
# conftest-friendly integration check (runs in seconds, costs nothing)
from primate_intelligence import Primate

def test_primate_integration():
    client = Primate()  # PRIMATE_API_KEY = a pv_test_ key in CI secrets
    videos = client.videos.list()
    analysis = client.analyses.create_and_wait(
        video_id=videos["data"][0]["id"],
        prompt="Is there a person in this video?",
        timeout_s=60,
    )
    assert analysis["result"]["answer"] == "yes"
    assert analysis["result"]["confidence"] >= 0.8
```

Every code sample in these docs runs in CI against this exact mechanism — docs that don't run don't ship.

## Limits of test mode

- Fixture corpus only — you can't upload arbitrary videos for real analysis on a test key (uploads work mechanically, but analysis results are canned)
- No live streaming GPU sessions
- `403 test_mode_only` marks operations unavailable in your key's mode

When you need real inference on your own footage: [sign up](https://primateintelligence.ai/signup), add a card, create a `pv_live_` key, change the env var. Done.

---

<!-- source: https://www.primateintelligence.ai/docs/guides/versioning.md -->

# Versioning & deprecation

> Compatibility guarantees, deprecation process, model sunsets, and service expectations.


## Compatibility contract

The v1 API is **additive-only**. Within v1 we may:

- Add new endpoints, optional request fields, response fields, enum values on *new* fields, and error codes (the registry is append-only)

We will never (without a major version):

- Remove or rename fields/endpoints, change field types or meanings, change error-code semantics, or tighten validation on existing fields

Write clients that **ignore unknown fields** — that's the only client-side requirement for staying compatible.

The gate is mechanical: every spec change runs `oasdiff breaking` in CI; a breaking diff fails the build unless it's an explicitly approved major-version migration.

## Deprecation process

When something must go:

1. It's marked `deprecated` in the OpenAPI spec + the [changelog](/docs/changelog) (typed entry: `breaking | additive | fix | deprecation`)
2. A **migration guide** ships with before/after code in curl/TS/Python, the deadline, and an automated check command
3. Sunset headers appear on responses where applicable
4. Breaking removals only ever land in a new major version

## Legacy `pk_` key sunset

API keys created before v1 used the `pk_` prefix. They remain valid during the launch migration so existing integrations do not break. New keys and rotations now create `pv_live_` / `pv_test_` keys only.

The `pk_` compatibility window is **12 months from GA announcement**. The exact sunset date starts when Matt approves the production GA cutover and will be published here, in the changelog, and in dashboard key-management copy. Before that date, rotate each legacy key from the dashboard or `POST /v1/api-keys/{id}/rotate`; the replacement key is drop-in for `Authorization: Bearer`.

After the sunset, `pk_` keys return `401 invalid_api_key`. No paid usage is charged by the act of rotating a key.

## Model versioning

Models are versioned resources (`GET /v1/models`) with lifecycle: `preview → stable → deprecated`, and a `sunset_at` date once deprecated.

- Omitting `model` on create uses the current default — fine for exploration
- **Production integrations should pin a model id** (e.g. `"model": "darwin-1.3"`) so behavior changes only when you choose
- `400 model_deprecated` past sunset tells you to re-pin; the changelog carries the migration entry

## SDK versioning

SDKs follow semver. An SDK major bump happens only with an API major. Minor/patch releases are additive (new endpoints, fixes) and safe to auto-update within `^`.

## Service expectations

Published targets (not a contractual SLA in v1):

- **Availability:** 99.9%/mo control plane; ≥ 99% completion success for admitted analyses
- **Latency:** p95 reads < 300ms; creates < 1.5s (excluding upload bytes); webhook dispatch p95 < 60s after terminal transition; test-mode analyses < 10s
- **Analysis latency is workload-shaped** — we publish honest transparency instead: `queue_position` + `estimated_start_s` accurate to ±50%
- Single region: us-west-2 (US). All processing and storage. EU is roadmap, not promise.

## Changelog & RSS

Every change lands in the [changelog](/docs/changelog) with a date and type tag. Subscribe: [RSS feed](/docs/changelog.xml). Agents: the changelog is also in `llms-full.txt`.

---

<!-- source: https://www.primateintelligence.ai/docs/guides/security-model.md -->

# Security model

> API keys, client tokens, scopes, data handling, and the key-compromise procedure.


## API keys

- Two prefixes: `pv_live_` (real processing, metered) and `pv_test_` ([test mode](/docs/guides/test-mode)). 256-bit CSPRNG entropy.
- Shown **exactly once** at creation/rotation. We store only a SHA-256 digest.
- Send via `Authorization: Bearer …` header only. Keys never belong in URLs, logs, or browser code.
- Env-var contract: `PRIMATE_API_KEY` — both SDKs and the MCP server read it. The MCP server never accepts keys as tool arguments (keys must not land in agent transcripts).
- Revocation propagates in ≤ 5 seconds.

## Client tokens (browser auth)

Secret keys must never reach a browser. For client-side flows, your server mints an **ephemeral client token**:

```bash
curl -s -X POST https://api.primateintelligence.ai/v1/client_tokens \
  -H "Authorization: Bearer $PRIMATE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"scopes": ["videos:write", "analyses:read"], "ttl_s": 300}'
```

- Prefix `pvct_`, TTL 60–900s (default 900), **never refreshable** — mint a new one when it expires (`401 token_expired`)
- Scopes (v1): `videos:write`, `analyses:read`, `analyses:write`, `streams:create`, `streams:signal` — a token's scopes must be a subset of the minting key's
- Optionally bind a token to one `video_id` or `stream_id` — the token then works only for that resource
- The streaming signaling WebSocket accepts **only** client tokens (or first-party session JWTs) — never secret keys

Pattern: a ≤ 20-line "token mint" route on your server, browser does the rest. Working example: the token-mint server sample in the [examples](https://github.com/Primate-Intelligence/primate-examples).

## Data handling

<!-- A12-FROZEN-SENTENCE: keep identical to ToS wording. Legal review pending (Phase-4 gate). -->
**Prompts and result metadata are retained to operate and improve the service; customer video content is never used to train models.**

The full contract:

| Data | Retention |
|---|---|
| Source videos | Deleted 30 days after upload; `DELETE /v1/videos/{id}` propagates from S3 + CDN within 72h (signed URLs expire ≤ 1h) |
| Result artifacts (annotated videos) | 30 days, same deletion path |
| Result JSON + analysis metadata | 13 months (billing/audit), then aggregated |
| Prompts | Retained per the sentence above; org-level opt-out available on request |
| Account deletion | Cascades videos/analyses/keys within 30 days; webhook endpoints immediately |

- Encryption: TLS ≥ 1.2 in transit; AES-256 at rest
- No customer video bytes in logs or analytics — enforced by a redaction test in CI
- Region: all processing/storage in us-west-2 (US)
- Audit log: key lifecycle + webhook changes retained 13 months

## URL ingest (SSRF policy)

`POST /v1/videos {url}` fetches are locked down: https only, port 443, public addresses only (RFC-1918/link-local/metadata ranges rejected), resolved IPs pinned (no TOCTOU), max 3 redirects each re-validated, 2 GiB streamed cap, content validated by magic bytes. Policy violations return `url_forbidden`; network failures `url_fetch_failed`.

## If a key is compromised

1. **Rotate immediately** — create a new key, revoke the old one in the [dashboard](https://primateintelligence.ai/dashboard/api-keys) (propagation ≤ 5s)
2. **Audit** — `GET /v1/analyses?created_after=…` for activity you don't recognize
3. **Contact support** with request ids for usage-forgiveness on abusive spend

We monitor per-key spend anomalies (> 5× 7-day baseline), page on them, and may auto-suspend a key pending confirmation — with an email + dashboard banner, never silently.

## Webhook signing

Deliveries are signed (Standard Webhooks, HMAC-SHA256, 5-minute replay window, 24h dual-secret rotation). Details: [webhooks guide](/docs/guides/webhooks).

---

<!-- source: https://www.primateintelligence.ai/docs/guides/billing.md -->

# Billing & credits

> Credit-seconds, grants, auto-refill, and handling insufficient_credits in code.


Primate Vision bills by **video-seconds processed**, prepaid as credits. 1 credit-second = 1 second of video analyzed. Current pricing is always at `GET /v1/credit-pricing` (public, no auth) and on the [pricing page](/pricing).

## How credits flow

- **Signup grant** — new accounts get free credit-seconds to evaluate with (currently 6,000s ≈ 100 minutes; expiring)
- **Card grant** — adding a card grants another tranche
- **Purchases** — buy credit blocks in the [dashboard](https://primateintelligence.ai/dashboard/billing); custom amounts supported
- **Auto-refill** — opt in to top up automatically when the balance crosses the threshold (currently 600s)

Test-mode keys never hold or spend credits — build and CI on `pv_test_` for free, forever.

## Check your balance

```bash doc-test id=billing-usage
curl -s https://api.primateintelligence.ai/v1/usage \
  -H "Authorization: Bearer $PRIMATE_API_KEY"
```

```json
{
  "meters": [
    { "meter": "credit_seconds", "unit": "seconds", "balance": 5990 },
    { "meter": "seconds_processed.period", "unit": "seconds", "used": 10, "resets_at": "2026-08-01T00:00:00Z" }
  ]
}
```

## What you're charged for

- **Analyses** — the duration of video actually analyzed, debited when the analysis completes. Failed platform-side work (`inference_error`, `stuck_timeout`) is **not billed**; resubmit freely.
- **Streams** — live sessions reserve credits up front and reconcile to actual connected seconds at end (`GET /v1/streams/{id}` shows the final usage). Mid-stream you get `warning` messages as the reservation runs low, then the session ends with `end_reason: "insufficient_credits"`.

## Handling `insufficient_credits` in code

The one billing error every integration must handle (`402`, **not retryable**):

```typescript doc-test id=billing-handling sdk=ts offline
import Primate, { PrimateError } from '@primate-intelligence/sdk';

const client = new Primate();
try {
  await client.analyses.create({ video_id: videoId, prompt: 'Is the loading dock clear?' });
} catch (err) {
  if (err instanceof PrimateError && err.code === 'insufficient_credits') {
    const { meters } = await client.usage.retrieve();
    const balance = meters.find((m) => m.meter === 'credit_seconds')?.balance ?? 0;
    notifyOwner(
      `Primate Vision balance is ${balance}s — top up or enable auto-refill: ` +
      `https://primateintelligence.ai/dashboard/billing`,
    );
    // Do NOT retry unchanged — queue the job for after refill instead.
  } else {
    throw err;
  }
}
```

The robust pattern:

1. Catch `insufficient_credits` → **stop submitting** (it will keep failing)
2. Read `GET /v1/usage` for the real balance
3. Surface a human-actionable message with the billing URL
4. If the account has auto-refill, retry after a short delay **once** — refill is triggered by the threshold crossing
5. Alert *before* you hit zero: poll `credit_seconds` and warn below your own threshold

## Free-credit expiry

Granted (free) credits expire; purchased credits don't. Expiry dates show in the dashboard. Expired grants simply vanish from `balance` — no negative surprises.

## Invoices & history

Purchase history and receipts live in the [dashboard billing page](https://primateintelligence.ai/dashboard/billing). Usage aggregates by day/analysis are on the [usage page](https://primateintelligence.ai/dashboard/usage).

---

<!-- source: https://www.primateintelligence.ai/docs/guides/streaming.md -->

# Streaming

> Real-time video analysis over WebRTC — create, signal, go live, get per-frame results.


Streams analyze live video in real time: point a camera (or any MediaStream) at the API and get per-frame detection results back over a WebRTC data channel, with the same prompt semantics as file analyses.

## Lifecycle

```
POST /v1/streams → queued|ready → (WS join → offer/answer/ICE) → live → ended
```

1. **Create** the stream server-side with your secret key:

```bash
curl -s -X POST https://api.primateintelligence.ai/v1/streams \
  -H "Authorization: Bearer $PRIMATE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Is there a person in the frame?", "options": {"narrative": true}, "recording": true}'
```

The response carries everything the client needs: `signaling.url` (a WebSocket), `ice_servers` (STUN/TURN with credentials), `limits` (`max_session_s`, `warn_at_remaining_s`), and `queue_position`/`estimated_start_s` when capacity is busy.

Two optional opt-ins at create time:

- **`options: {narrative: true}`** — [live narrative](#live-narrative-opt-in): result detections carry `narrative_update` sentences, and the WS delivers `status` ordering events. No surcharge.
- **`recording: true`** — [session recording](#session-recordings-opt-in): retrieve the annotated session video after the stream ends.

2. **Mint a client token** for the browser/device (the signaling WS **never** accepts secret keys):

```bash
curl -s -X POST https://api.primateintelligence.ai/v1/client_tokens \
  -H "Authorization: Bearer $PRIMATE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"scopes": ["streams:signal"], "stream_id": "'"$STREAM_ID"'", "ttl_s": 300}'
```

3. **Connect** from the client. With the TS SDK this is one call:

```typescript
import { connectStream } from '@primate-intelligence/sdk/browser';

const session = await connectStream({
  stream,                    // the Stream resource (relay it from your server)
  clientToken,               // pvct_… minted in step 2
  mediaStream: await navigator.mediaDevices.getUserMedia({ video: true }),
  onResult: (r) => console.log(r.frame_num, r.detections),
  onNarrative: (n) => log(`${n.t_s}s: ${n.text}`),        // narrative streams only
  onStatus: (s) => log(`engine: ${s.status}`),            // narrative streams only
  onWarning: (w) => (w.code ? diagnose(w) : showCountdown(w.remaining_s)),
  onEnd: (reason) => cleanup(reason),
});
// change the question mid-stream (server confirms with prompt_updated):
session.updatePrompt('Is there a forklift moving?');
// done:
session.end();
```

Under the hood: WS `join` → `ready` (or `queued {position}`) → WebRTC `offer`/`answer` + ICE trickle → media flows → `result` messages per analyzed frame.

## Signaling protocol (build your own client)

Messages you send: `join`, `offer {sdp}`, `ice {candidate}`, `update_prompt {prompt}`, `end`, `ping`.
Messages you receive: `queued {position, estimated_start_s}`, `ready`, `answer {sdp}`, `ice {candidate}`, `live`, `result {frame_num, detections}`, `prompt_updated`, `status {…}` (narrative streams only), `metering {…}`, `warning {…}`, `end {reason}`, `error {code, message}`, `pong`.

There are two kinds of `warning`: the credit-exhaustion countdown (`warning {remaining_s}`, no `code` field) and diagnostic warnings that carry a `code` field — see [No-media warning](#no-media-warning-the-server-tells-you-when-your-media-is-the-problem) below. Distinguish them by the presence of `code`.

A Python/aiortc backend example (robotics-style, no browser — including a "stream a file" recipe and a relay-only datacenter/CI profile) ships in the public [examples repo](https://github.com/Primate-Intelligence/primate-examples), alongside a self-contained browser webcam example.

**Trickle ICE is bidirectional**: after the `answer`, the server may keep sending `ice {candidate}` messages as late srflx/relay candidates finish gathering — keep consuming and adding them. Clients on UDP-blocked networks (datacenters, CI, strict NATs) complete via the TURN servers in `ice_servers`; TURN-over-TCP:443 is supported.

### Result contract (identical to file analyses)

Each `result.detections[]` row uses the same enums and casing as the file API: `answer` is lowercase `yes|no|indeterminate`, `confidence` is 0..1, and `prompt` is echoed exactly as you submitted it. Latency telemetry (per-stage breakdowns) lives under a `timing` object.

### Update the prompt mid-stream

Send `{"type": "update_prompt", "prompt": "…"}` (≤2000 chars) at any point while live — no reconnect, no new stream. The server recompiles the prompt, applies it to the live engine, and confirms with `{"type": "prompt_updated"}`; subsequent result frames echo the new prompt. An invalid prompt returns `{"type": "error", "code": "validation_failed" | "parse_failed"}` and the **old prompt stays active**. On narrative streams, expect a `recalculating` status event while the engine rebuilds context.

### Live narrative (opt-in)

Create the stream with `options: {narrative: true}` and two things change (nothing changes without the opt-in):

1. Result detections carry a **`narrative_update`** object — `{t_s, text}`, a new narrative sentence — whenever the engine detects an appearance/disappearance/action. Event-driven, NOT per frame: absent (not `null`) on frames without one.
2. The WS delivers additive **`status`** events for ordering/annotating narrative entries:

```json
{"type": "status", "status": "prompt_context" | "combined_prompt" | "recalculating", "message": "…"}
```

The vocabulary is a **closed set** — exactly these three values; internal engine statuses never leak. Typical use: mark when the engine is recalculating context right after an `update_prompt`. Included in the per-second price — no surcharge.

### Session recordings (opt-in)

Create the stream with **`recording: true`** (top-level boolean) to retrieve the server-side session recording afterwards:

1. The stream resource carries `recording: {status}` — `recording` while live, `available` once ended with a stored recording, `failed`, or `none`. Streams without the opt-in have `recording: null`.
2. Once `available`, call `GET /v1/streams/{id}/recording` → `{url, expires_at, content_type: "video/h264", container: "h264-annex-b"}`. The URL is **signed fresh on every GET, 1-hour TTL** — re-fetch for a new one; old recordings always stay retrievable.
3. The recording is a raw H.264 Annex-B elementary stream (the exact annotated frames the client saw). Lossless remux to MP4: `ffmpeg -i recording.h264 -c copy recording.mp4`.

### Result sampling — results are sampled, not per-frame

Results are **sampled**, not emitted for every source frame — inference cadence is adaptive (roughly every 8th source frame under load). Two different axes to keep straight:

- `frame_num` on each live `result` is the **source-frame index** the result was computed on. Gaps between consecutive `frame_num` values are normal and expected — don't treat them as dropped results.
- `results_summary.result_frames` on the ended stream resource counts the number of **result events emitted** over the session — NOT a source-frame count.

Expect roughly **8–15 results per second** depending on load; never assume a fixed cadence.

> **Alias note:** `results_summary.frames` is a supported alias of `results_summary.result_frames` (identical value), superseded by `result_frames`. Prefer `result_frames` in new code; existing reads of `frames` keep working.

### Metering ticks

While live, a `metering` message arrives every 5 seconds:

```json
{"type": "metering", "elapsed_s": 45, "billed_s": 45, "session_remaining_s": 555, "balance_s": 555}
```

- `session_remaining_s` — seconds remaining in **this session's** credit reservation (the session cap). This is NOT your account credit balance — that lives at `GET /v1/billing/credits` (`balance_seconds`).
- `balance_s` — supported alias of `session_remaining_s` (identical value), superseded by `session_remaining_s`. Prefer `session_remaining_s` in new code; existing reads of `balance_s` keep working.
- `elapsed_s` / `billed_s` — live-clock seconds elapsed = billed (identical by construction; join, queueing, and negotiation are free).

### No-media warning (the server tells you when your media is the problem)

If the WebRTC transport connects but **no decodable video frame arrives within 5 seconds**, the server pushes a diagnostic `warning` on the signaling WS (re-warned once at 15s, then quiet; canceled by the first decoded frame):

```json
{"type": "warning", "code": "no_media_frames", "transport_connected": true,
 "elapsed_s": 5, "packets_received": 312, "frames_decoded": 0, "hint": "…"}
```

Read `packets_received` to self-diagnose:

- `packets_received: 0` — your client is **not sending media** (track not attached, muted, or the capture source is dead).
- `packets_received > 0` with `frames_decoded: 0` — media is arriving but is **not decodable** (wrong codec — must be VP8 or H.264 — or the source produces no real frames, e.g. a canvas capture without an active draw loop).

If the session then ends without ever going live, the terminal resource records `end_reason: "media_timeout"` with the same media counters in `failure_diagnostic`, and bills 0.

### Honest end reasons

`end {reason}` and the terminal resource's `end_reason` distinguish real failures — a session that never went live is never `completed`:

| Reason | Meaning | Billed |
|---|---|---|
| `completed` | Normal end after live | live seconds |
| `canceled` | Ended before going live (client action) | 0 |
| `ice_failed` | ICE never connected — resource carries `failure_diagnostic` (server candidate summary) | 0 |
| `media_timeout` | Transport connected but no media flowed — `failure_diagnostic` carries the media counters | 0 |
| `insufficient_credits` | Balance exhausted mid-stream | live seconds |
| `timeout` | `limits.max_session_s` reached | live seconds |
| `error` | Server-side failure | live seconds (0 if never live) |

### failure_diagnostic — debugging a failed session from the API alone

When a session ends without ever going live on a transport failure, the terminal Stream resource carries a nullable `failure_diagnostic` object (and the session bills **0**):

- On `ice_failed` it holds the server-side ICE candidate summary:

```json
{"end_reason": "ice_failed",
 "failure_diagnostic": {
   "local_candidates": ["host 203.0.113.7", "srflx 203.0.113.7", "relay 198.51.100.20"],
   "hint": "Server advertised public host + srflx/relay candidates; check the client's path to the TURN servers in ice_servers."
 }}
```

  If `local_candidates` shows only private addresses (`172.x`, `10.x`), the fault is on our side — contact support. If it shows srflx/relay candidates, check your client's network path to the TURN servers in `ice_servers` (TURN-over-TCP:443 and `turns:` TLS relays are available for locked-down egress networks).

- On `media_timeout` it holds the media counters from the no-media warning (`packets_received`, `frames_decoded`) so you can tell "never sent media" apart from "sent undecodable media" after the fact.

`failure_diagnostic` is `null` on successful sessions and on ends unrelated to transport.

## Credits & limits

- One active (queued/ready/live) stream per account by default — a second create returns `409 stream_already_active`
- Sessions reserve credits up front; metering ticks every 5s (see [Metering ticks](#metering-ticks) for the field contract); you get `warning {remaining_s}` before funds run out, then `end {reason: "insufficient_credits"}`
- `limits.max_session_s` is the hard per-session cap from your plan
- Your account-wide balance is at `GET /v1/billing/credits` — metering ticks only report the session reservation
- After `ended`, `GET /v1/streams/{id}` shows `duration_s`, `usage` (reconciled to the second), and `results_summary` (`result_frames` = count of result events — see [Result sampling](#result-sampling--results-are-sampled-not-per-frame))

## End a stream

From the client: send `end` (or just `session.end()`). Server-side / cleanup:

```bash
curl -s -X POST https://api.primateintelligence.ai/v1/streams/$STREAM_ID/end \
  -H "Authorization: Bearer $PRIMATE_API_KEY"
```

Idempotent — safe to call on an already-ended stream. `stream.ended` fires as a [webhook](/docs/guides/webhooks) if subscribed.

---

<!-- source: https://www.primateintelligence.ai/docs/guides/build-from-scratch.md -->

# Tutorial: build from scratch

> Zero to a working video-question CLI in ~15 minutes — no account required to start.


We'll build `askvideo` — a Node CLI that takes a video URL and a question, and prints the answer. Zero to working in about 15 minutes, entirely on a free test key.

## Step 0: get a key (30 seconds)

```bash doc-test id=tutorial-sandbox
curl -s -X POST https://api.primateintelligence.ai/v1/sandbox
```

```bash
export PRIMATE_API_KEY="pv_test_…"   # from the response
```

## Step 1: project setup

```bash
mkdir askvideo && cd askvideo
npm init -y
```

## Step 2: the whole program

`askvideo.mjs`:

```javascript
const [url, ...promptParts] = process.argv.slice(2);
const prompt = promptParts.join(' ');
if (!url || !prompt) {
  console.error('usage: node askvideo.mjs <video-url> <question…>');
  process.exit(2);
}

const apiKey = process.env.PRIMATE_API_KEY;
const baseUrl = process.env.PRIMATE_BASE_URL ?? 'https://api.primateintelligence.ai';
if (!apiKey) {
  console.error('PRIMATE_API_KEY is required');
  process.exit(2);
}

async function api(path, options = {}) {
  const res = await fetch(`${baseUrl}${path}`, {
    ...options,
    headers: {
      Authorization: `Bearer ${apiKey}`,
      'Content-Type': 'application/json',
      ...(options.headers ?? {}),
    },
  });
  const body = await res.json();
  if (!res.ok) {
    const err = body.error ?? { code: 'request_failed', message: res.statusText };
    console.error(`${err.code}: ${err.message}`);
    if (err.docs_url) console.error(`docs: ${err.docs_url}`);
    process.exit(1);
  }
  return body;
}

async function createAndWait(videoId, question) {
  let analysis = await api('/v1/analyses', {
    method: 'POST',
    headers: { Prefer: 'wait=60' },
    body: JSON.stringify({ video_id: videoId, prompt: question }),
  });

  for (let i = 0; analysis.status !== 'completed' && i < 30; i += 1) {
    if (analysis.status === 'failed' || analysis.status === 'canceled') {
      throw new Error(`analysis_${analysis.status}: ${JSON.stringify(analysis.error)}`);
    }
    await new Promise(resolve => setTimeout(resolve, 1000));
    analysis = await api(`/v1/analyses/${analysis.id}`);
  }
  return analysis;
}

const video = await api('/v1/videos', {
  method: 'POST',
  body: JSON.stringify({ url }),
});
console.error(`video ${video.id} (${video.status})`);

const analysis = await createAndWait(video.id, prompt);
console.log(JSON.stringify(analysis.result, null, 2));
```

## Step 3: run it

Test keys analyze the fixture corpus deterministically, so use the official fixture:

```bash
FIXTURE=$(curl -s https://api.primateintelligence.ai/v1/test-fixture)
node askvideo.mjs \
  "$(echo $FIXTURE | python3 -c 'import json,sys; print(json.load(sys.stdin)["test_video_url"])')" \
  "Is there a person in this video?"
```

```json
{
  "answer": "yes",
  "confidence": 0.93,
  "clips": [{ "start_s": 1, "end_s": 4, "confidence": 0.93 }],
  "query_type": "object",
  "video_duration_s": 10
}
```

That's a working integration. Everything below is production hardening.

## Step 4: make it CI-verifiable

`test.mjs`:

```javascript
const baseUrl = process.env.PRIMATE_BASE_URL ?? 'https://api.primateintelligence.ai';
const headers = {
  Authorization: `Bearer ${process.env.PRIMATE_API_KEY}`,
  'Content-Type': 'application/json',
};
const fixture = await fetch(`${baseUrl}/v1/test-fixture`).then(r => r.json());
const video = await fetch(`${baseUrl}/v1/videos`, {
  method: 'POST',
  headers,
  body: JSON.stringify({ url: fixture.test_video_url }),
}).then(r => r.json());
const analysis = await fetch(`${baseUrl}/v1/analyses`, {
  method: 'POST',
  headers: { ...headers, Prefer: 'wait=60' },
  body: JSON.stringify({ video_id: video.id, prompt: fixture.test_prompt }),
}).then(r => r.json());

const ok =
  analysis.result.answer.toLowerCase() === fixture.expected_answer.toLowerCase() &&
  analysis.result.confidence >= fixture.expected_confidence_min;
console.log(ok ? 'INTEGRATION OK' : 'INTEGRATION BROKEN');
process.exit(ok ? 0 : 1);
```

Run that in CI with a `pv_test_` key in your secrets — it completes in seconds and costs nothing.

## Step 5: production hardening checklist

- **Error handling** — retry only codes marked retryable in the [error registry](/docs/guides/errors-retries); never retry [`insufficient_credits`](/docs/guides/billing)
- **Stop polling** — swap `createAndWait` for [webhooks](/docs/guides/webhooks) at volume
- **Pin a model** — pass `model: 'darwin-1.3'` so behavior changes only when you choose ([versioning](/docs/guides/versioning))
- **Your own videos** — swap URL ingest for [presigned upload](/docs/guides/uploading) when the bytes live with you
- **Go live** — [sign up](https://primateintelligence.ai/signup), create a `pv_live_` key, change the env var. No code changes.

## Where to next

- [Sample matrix](https://github.com/Primate-Intelligence/primate-examples) — Python, webhook receivers, browser SPA, streaming, token-mint server
- [Agent quickstart](/docs/agents) — the condensed machine-readable version of everything you just did
- [API reference](/docs/reference)

---

<!-- source: https://www.primateintelligence.ai/docs/supported-actions.md -->

# Supported actions

> What Darwin-preview-1.3B can and cannot detect — the honest capability list, including the full action vocabulary and known gaps.


This page is the honest capability list for **Darwin-preview-1.3B** (status: preview). It documents what the model can reliably detect, what it detects with reduced accuracy, and what it cannot detect at all — so you can design prompts that work instead of discovering the limits by trial and error.

**Preview means early access**: detection quality and the set of reliably detected actions are still evolving, and results may improve or change between checkpoint updates. The API contract (request/response shapes, billing) is stable.

If an analysis returns `answer: "indeterminate"` with `indeterminate_reason: "unsupported_query_form"`, or an action prompt produces a nonsensical match, this page explains why — and what to ask instead.

## Summary: what works, by prompt type

| Prompt type | Example | Reliability |
|---|---|---|
| **Presence** | "Is there a person?" | Stable — most reliable query form |
| **Absence** | "Are there no dogs?" | Stable |
| **Count** (bare + threshold) | "How many people?" / "More than 2 people?" | Stable on clips; not long multi-hour footage |
| **Compound** | "Is there a person AND a dog?" | Stable |
| **Attribute** | "Is anyone wearing a red jacket?" | Stable |
| **Location** | "Where is the dog?" | Stable |
| **State** | "Is the door open?" | Stable |
| **Segment** | "Show me where the person is" | Stable (returns masks) |
| **Action** | "Is someone walking?" | **Beta** — see the vocabulary below |
| **Open-ended** | "Tell me what you see" | Rejected at zero cost (`unsupported_query_form`) |

Presence, counting, attribute, location, state, and segmentation queries recognize the visual world broadly — common objects, people, animals, vehicles, scenes. They are not limited to a fixed list. **Action queries are different**: they run against a fixed action vocabulary, and that is where the honest limits live.

## How action detection works — and why the vocabulary matters

When you ask an action question ("Is someone running?", "Does anyone fall down?"), the model matches what it sees against a **fixed vocabulary of 532 action classes** (Kinetics-derived). Two consequences:

1. **In-vocabulary actions** can be detected, at beta accuracy. Short clips, fast motion, and unusual camera angles reduce reliability.
2. **Out-of-vocabulary actions cannot be detected.** The model has no embedding for the action, so it falls back to the nearest in-vocabulary neighbour — which can produce confidently wrong or nonsensical matches. If the action you care about is not in the list below, do not ship a decision on the result.

When the system can tell at submit time that a requested action is outside the vocabulary, the API flags it (see [Unsupported action signal](#unsupported-action-signal) below). When it cannot tell, you may still get a low-quality nearest-neighbour result — treat unexpected action answers with suspicion and check this list.

## Known gaps (confirmed, tracked)

These are actions users commonly ask about that the current vocabulary does **not** cover:

- **Hand gestures** — thumbs up, thumbs down, OK sign, peace sign. (Exception: `waving hand`, `finger snapping`, `clapping`-family classes exist.) Gesture prompts currently produce unreliable nearest-neighbour matches.
- **Most vehicle actions** — the vocabulary contains `driving car`, `driving tractor`, `motorcycling`, `riding scooter`, `riding a bike`, `riding mountain bike`, and `pushing car`. Essentially everything else is missing: parking, reversing, lane changes, braking, turning, overtaking, collisions/crashes, honking, loading/unloading, towing. Treat vehicle-action prompts (beyond basic "driving") as **not detected**.
- **Everyday ambient actions phrased generically** — "walking" and "running" as bare actions are matched via the model's broader action understanding but the vocabulary's walking/running entries are context-specific (`walking the dog`, `running on treadmill`, `jogging`). Generic phrasings usually work but are beta-accuracy; be specific where you can.

Vocabulary expansion (a ~5,000-action taxonomy with synonym grouping) is on the roadmap; this page tracks the current shipped reality.

## What is never supported (any prompt type)

- **Audio / sound** — "Is there loud noise?", "What did they say?"
- **Identity** — "Is this John?" (who, not what)
- **OCR / reading text in frame** — "What does the sign say?"
- **Subjective judgment** — "Does this look dangerous?", "Is the room clean?" (subjective terms are surfaced in `query.unassessable_components`)
- **Exhaustive counting over long footage** — counts are accurate on a clip or short segment, not a multi-hour archive

## Unsupported action signal

When prompt compilation determines the request cannot be scored, the API responds honestly instead of burning credits:

- **Open-ended prompts** are rejected at submit time, at zero cost: the analysis completes immediately with `answer: "indeterminate"`, `indeterminate_reason: "unsupported_query_form"`, and `confidence: 0`.
- Analyses whose result cannot be trusted carry an `indeterminate_reason` (`low_confidence`, `nothing_detected`, `unsupported_query_form`, `duration_mismatch`) — [full semantics in the prompts guide](/docs/guides/prompts).
- Responses in the unsupported path link back to this page so agents and users can see the capability list without guessing.

## The full action vocabulary (532 classes)

The complete, current list. If an action question maps to one of these, it is detectable (beta accuracy). If it does not, rephrase — often a **presence** question ("Is there a person on a ladder?") is more reliable than an action question ("Is someone climbing?").

**A** — `abseiling` · `acting in play` · `adjusting glasses` · `air drumming` · `answering questions` · `applauding` · `applying cream` · `archery` · `arm wrestling` · `arranging flowers` · `assembling computer` · `auctioning`

**B** — `baby waking up` · `baking cookies` · `balloon blowing` · `bandaging` · `barbequing` · `bartending` · `base jumping` · `bathing dog` · `beatboxing` · `bee keeping` · `belly dancing` · `bench pressing` · `bending back` · `bending metal` · `biking through mud` · `blasting sand` · `blowing glass` · `blowing leaves` · `blowing nose` · `blowing out candles` · `bobsledding` · `bookbinding` · `bouncing on trampoline` · `bowling` · `braiding hair` · `breading or breadcrumbing` · `breakdancing` · `brush painting` · `brushing hair` · `brushing teeth` · `building cabinet` · `building shed` · `bungee jumping` · `busking`

**C** — `canoeing or kayaking` · `capoeira` · `carrying baby` · `cartwheeling` · `carving pumpkin` · `catching fish` · `catching or throwing baseball` · `catching or throwing frisbee` · `catching or throwing softball` · `celebrating` · `changing oil` · `changing wheel` · `checking tires` · `cheerleading` · `chopping wood` · `clapping` · `clay pottery making` · `clean and jerk` · `cleaning floor` · `cleaning gutters` · `cleaning pool` · `cleaning shoes` · `cleaning toilet` · `cleaning windows` · `climbing a rope` · `climbing ladder` · `climbing tree` · `closing something` · `contact juggling` · `cooking chicken` · `cooking egg` · `cooking on campfire` · `cooking sausages` · `counting money` · `country line dancing` · `covering something` · `cracking neck` · `crawling baby` · `crossing river` · `crying` · `curling hair` · `cutting nails` · `cutting pineapple` · `cutting watermelon`

**D** — `dancing ballet` · `dancing charleston` · `dancing gangnam style` · `dancing macarena` · `deadlifting` · `decorating the christmas tree` · `digging` · `dining` · `disc golfing` · `diving cliff` · `dodgeball` · `doing aerobics` · `doing laundry` · `doing nails` · `drawing` · `dribbling basketball` · `drinking` · `drinking beer` · `drinking shots` · `driving car` · `driving tractor` · `drop kicking` · `drumming fingers` · `dunking basketball` · `dying hair`

**E** — `eating burger` · `eating cake` · `eating carrots` · `eating chips` · `eating doughnuts` · `eating hotdog` · `eating ice cream` · `eating spaghetti` · `eating watermelon` · `egg hunting` · `exercising arm` · `exercising with an exercise ball` · `extinguishing fire`

**F** — `faceplanting` · `feeding birds` · `feeding fish` · `feeding goats` · `felling tree` · `filling eyebrows` · `finger snapping` · `fixing hair` · `flipping pancake` · `fly tying` · `flying kite` · `folding clothes` · `folding napkins` · `folding paper` · `folding something` · `front raises` · `frying vegetables`

**G** — `gargling` · `getting a haircut` · `getting a tattoo` · `giving or receiving award` · `golf chipping` · `golf driving` · `golf putting` · `grinding meat` · `grooming dog` · `grooming horse` · `gymnastics tumbling`

**H** — `hammer throw` · `hand washing clothes` · `head stand` · `headbanging` · `headbutting` · `high jump` · `high kick` · `hitting baseball` · `hockey stop` · `holding something` · `holding something in front of something` · `holding something next to something` · `holding something over something` · `hoverboarding` · `hugging` · `hula hooping` · `hurdling` · `hurling sport`

**I** — `ice climbing` · `ice fishing` · `ice skating` · `ironing` · `ironing clothes`

**J** — `javelin throw` · `jetskiing` · `jogging` · `juggling balls` · `juggling fire` · `juggling soccer ball` · `jumping bicycle` · `jumping into pool` · `jumping jacks` · `jumping sofa` · `jumpstyle dancing`

**K** — `kicking field goal` · `kicking soccer ball` · `kissing` · `kitesurfing` · `knitting` · `krumping`

**L** — `laughing` · `laying bricks` · `laying stone` · `letting something roll down a surface` · `licking` · `lifting a surface with something on it` · `lifting hat` · `lifting something` · `lifting something up completely` · `lighting candle` · `lighting fire` · `lock picking` · `long jump` · `looking at phone` · `lunge`

**M** — `making a cake` · `making a sandwich` · `making bed` · `making jewelry` · `making pizza` · `making snowman` · `making sushi` · `making tea` · `marching` · `massaging back` · `massaging feet` · `massaging legs` · `massaging persons head` · `milking cow` · `mopping floor` · `motorcycling` · `moving child` · `moving furniture` · `moving something across a surface` · `moving something away from something` · `moving something closer to something` · `moving something down` · `moving something up` · `mowing lawn` · `mushroom foraging`

**N** — `news anchoring`

**O** — `opening bottle` · `opening door` · `opening present` · `opening something`

**P** — `packing` · `paragliding` · `parasailing` · `parkour` · `passing American football in game` · `passing American football not in game` · `peeling apple` · `peeling banana` · `peeling potatoes` · `petting animal not cat` · `petting cat` · `picking fruit` · `picking something up` · `pillow fight` · `pinching` · `plastering` · `playing accordion` · `playing badminton` · `playing bagpipes` · `playing basketball` · `playing bass guitar` · `playing cards` · `playing cello` · `playing chess` · `playing clarinet` · `playing controller` · `playing cricket` · `playing cymbals` · `playing didgeridoo` · `playing drums` · `playing flute` · `playing guitar` · `playing handball` · `playing harmonica` · `playing harp` · `playing ice hockey` · `playing keyboard` · `playing kickball` · `playing laser tag` · `playing monopoly` · `playing organ` · `playing paintball` · `playing piano` · `playing poker` · `playing recorder` · `playing saxophone` · `playing squash or racquetball` · `playing tennis` · `playing trombone` · `playing trumpet` · `playing ukulele` · `playing violin` · `playing volleyball` · `playing xylophone` · `plumbing` · `poaching eggs` · `poking something so it falls over` · `poking something so it slightly moves` · `pole vault` · `polishing furniture` · `polishing metal` · `popping balloons` · `pouring something into something` · `pouring something onto something` · `pretending to be a statue` · `pretending to pick something up` · `pretending to put something next to something` · `pretending to take something from somewhere` · `pretending to throw something` · `pull ups` · `pulling something from behind of something` · `pulling something from left to right` · `pulling something from right to left` · `pulling something onto something` · `pulling something out of something` · `pulling two ends of something so that it gets stretched` · `pumping fist` · `pumping gas` · `punching bag` · `punching person boxing` · `push up` · `pushing car` · `pushing something from left to right` · `pushing something from right to left` · `pushing something off of something` · `pushing something onto something` · `pushing something so it spins` · `pushing something so that it almost falls off but doesnt` · `pushing something so that it falls off the table` · `pushing something with something` · `pushing wheelchair` · `putting makeup on someone` · `putting on eyeliner` · `putting on foundation` · `putting on lipstick` · `putting on mascara` · `putting on shoes` · `putting something and something on the table` · `putting something behind something` · `putting something in front of something` · `putting something into something` · `putting something next to something` · `putting something on a flat surface without letting it roll` · `putting something on a surface` · `putting something on the edge of something so it is not supported and falls down` · `putting something onto a slanted surface but it doesnt glide down` · `putting something onto something` · `putting something similar to other things that are already on the table` · `putting something that cant roll onto a slanted surface so it slides down` · `putting something underneath something` · `putting something upright on the table`

**R** — `reading book` · `reading newspaper` · `recording music` · `removing something spreading it apart from something` · `riding a bike` · `riding camel` · `riding elephant` · `riding mechanical bull` · `riding mountain bike` · `riding mule` · `riding or walking with horse` · `riding scooter` · `riding snow blower` · `riding unicycle` · `ripping paper` · `robot dancing` · `rock climbing` · `rock scissors paper` · `roller skating` · `rolling eyes` · `rolling pastry` · `rolling something on a flat surface` · `rope pushdown` · `rowing boat` · `running on treadmill`

**S** — `sailing` · `salsa dancing` · `sanding floor` · `scrambling eggs` · `scuba diving` · `setting table` · `sewing` · `shaking hands` · `shaking head` · `shaking something` · `sharpening knives` · `sharpening pencil` · `shaving head` · `shaving legs` · `shearing sheep` · `shooting basketball` · `shooting goal soccer` · `shopping` · `shot put` · `shoveling snow` · `showing something behind something` · `showing something next to something` · `showing something on top of something` · `showing something to the camera` · `shredding paper` · `shuffling cards` · `side kick` · `sign language interpreting` · `singing` · `situp` · `skateboarding` · `ski ballet` · `ski jumping` · `skiing crosscountry` · `skiing mono` · `skiing slalom` · `skipping rope` · `skipping stone` · `skydiving` · `slacklining` · `slapping` · `sled dog racing` · `smoking` · `smoking hookah` · `snatch weight lifting` · `sneezing` · `sniffing` · `snorkeling` · `snowboarding` · `snowkiting` · `snowmobiling` · `somersaulting` · `spinning poi` · `spinning something so it continues spinning` · `spinning something that quickly stops spinning` · `spraying` · `springboard diving` · `squat` · `squeezing something` · `stacking something` · `stamping something` · `standing on hands` · `sticking something on something` · `sticking tongue out` · `stirring` · `stomping grapes` · `stretching arm` · `stretching leg` · `strumming guitar` · `stuffing something into something` · `surfing crowd` · `surfing water` · `sweeping floor` · `swimming backstroke` · `swimming breast stroke` · `swimming butterfly stroke` · `swing dancing` · `swinging legs` · `swinging on something` · `sword fighting` · `sword swallowing`

**T** — `tackling football` · `tai chi` · `taking a shower` · `taking photo` · `taking something from somewhere` · `taking something out of something` · `tango dancing` · `tap dancing` · `tapping guitar` · `tapping pen` · `tasting beer` · `tasting food` · `tasting wine` · `texting` · `throwing axe` · `throwing ball` · `throwing discus` · `throwing knife` · `throwing something` · `throwing something against something` · `throwing something in the air and catching it` · `throwing something in the air and letting it fall` · `throwing something onto a surface` · `tickling` · `tipping something over` · `tipping something with something in it over so everything falls out` · `tobogganing` · `tossing coin` · `tossing salad` · `training dog` · `trapezing` · `trimming or shaving beard` · `trimming trees` · `triple jump` · `tumbling something downwards` · `turning something upside down` · `turning the camera left while filming something` · `turning the camera right while filming something` · `tying bow tie` · `tying knot not on a+rope` · `tying shoe laces`

**U** — `uncovering something` · `unfolding something` · `using a power drill` · `using a wrench` · `using computer` · `using remote controller not gaming` · `using segway`

**V** — `vault` · `visiting museum`

**W** — `walking the dog` · `washing dishes` · `washing feet` · `washing hair` · `washing hands` · `water skiing` · `water sliding` · `watering plants` · `waving hand` · `waxing back` · `waxing chest` · `waxing eyebrows` · `waxing legs` · `weaving basket` · `welding` · `whistling` · `windsurfing` · `wiping something off of something` · `wrapping present` · `wrestling` · `writing`

**Y** — `yawning` · `yoga`

**Z** — `zumba`

---

*This list reflects the deployed vocabulary and updates when the model checkpoint updates. Last verified: 2026-08-01.*

---

<!-- source: https://www.primateintelligence.ai/docs/errors.md -->

# Error registry

> Every error code with retry semantics — the docs_url targets.


Every error code the API can return, with retry semantics. This registry is closed and append-only — codes are never removed or repurposed within v1. Machine-readable at `GET https://api.primateintelligence.ai/v1/errors`.

**retryable** = safe to retry the same request unchanged (with backoff). **idem** = idempotent to retry with the same `Idempotency-Key`.

## HTTP errors

### `invalid_api_key`

<a id="invalid_api_key"></a>**HTTP 401** · retryable: **no** · idem: **no**

API key is invalid. Check the key, env var, and whitespace.

### `key_revoked`

<a id="key_revoked"></a>**HTTP 401** · retryable: **no** · idem: **no**

This API key has been revoked. Rotate or create a new key.

### `key_expired`

<a id="key_expired"></a>**HTTP 401** · retryable: **no** · idem: **no**

This API key has expired. Create a new key.

### `token_expired`

<a id="token_expired"></a>**HTTP 401** · retryable: **no** · idem: **no**

Client token has expired. Mint a new token from your server (client tokens are never refreshable).

### `insufficient_scope`

<a id="insufficient_scope"></a>**HTTP 403** · retryable: **no** · idem: **no**

The key does not carry the scope required for this operation.

### `account_restricted`

<a id="account_restricted"></a>**HTTP 403** · retryable: **no** · idem: **no**

Account is restricted. Complete signup/verification.

### `test_mode_only`

<a id="test_mode_only"></a>**HTTP 403** · retryable: **no** · idem: **no**

Operation not available in this key mode.

### `test_key_fixture_only`

<a id="test_key_fixture_only"></a>**HTTP 403** · retryable: **no** · idem: **no**

Test keys can only analyze the provided fixture video. Create a live key to analyze your own uploads.

### `validation_failed`

<a id="validation_failed"></a>**HTTP 400** · retryable: **no** · idem: **no**

Request validation failed. All violations are listed in error.errors[].

### `unsupported_media_type`

<a id="unsupported_media_type"></a>**HTTP 415** · retryable: **no** · idem: **no**

Send video/mp4 or video/quicktime with a correct Content-Type.

### `payload_too_large`

<a id="payload_too_large"></a>**HTTP 413** · retryable: **no** · idem: **no**

Payload exceeds the 100MB multipart cap. Use the presigned upload path.

### `resource_not_found`

<a id="resource_not_found"></a>**HTTP 404** · retryable: **no** · idem: **no**

No such resource. IDs are account-scoped; check the ID and prefix.

### `resource_conflict`

<a id="resource_conflict"></a>**HTTP 409** · retryable: **no** · idem: **no**

Invalid state transition. GET the resource for its current state.

### `idempotency_key_reused`

<a id="idempotency_key_reused"></a>**HTTP 409** · retryable: **no** · idem: **no**

Idempotency-Key was reused with a different request body.

### `idempotency_unavailable`

<a id="idempotency_unavailable"></a>**HTTP 409** · retryable: **yes** · idem: **yes**

Idempotency store unavailable. Retry with the same key after Retry-After.

### `stream_already_active`

<a id="stream_already_active"></a>**HTTP 409** · retryable: **no** · idem: **no**

An active stream already exists on this account. End it (POST /v1/streams/{id}/end) or wait for it to finish.

### `upload_incomplete`

<a id="upload_incomplete"></a>**HTTP 400** · retryable: **no** · idem: **no**

Upload incomplete or size mismatch. Re-create the video and complete within 24h.

### `video_unreadable`

<a id="video_unreadable"></a>**HTTP 400** · retryable: **no** · idem: **no**

Container/codec unsupported. Re-encode to H.264 MP4.

### `video_too_large`

<a id="video_too_large"></a>**HTTP 400** · retryable: **no** · idem: **no**

Video exceeds the 2 GiB limit.

### `video_too_long`

<a id="video_too_long"></a>**HTTP 400** · retryable: **no** · idem: **no**

Video exceeds the plan duration limit. Upgrade or trim.

### `url_fetch_failed`

<a id="url_fetch_failed"></a>**HTTP 400** · retryable: **yes** · idem: **no**

Source URL unreachable or timed out.

### `url_forbidden`

<a id="url_forbidden"></a>**HTTP 400** · retryable: **no** · idem: **no**

URL scheme or host blocked by the ingest policy.

### `prompt_empty`

<a id="prompt_empty"></a>**HTTP 400** · retryable: **no** · idem: **no**

Provide prompt or query.

### `prompt_too_long`

<a id="prompt_too_long"></a>**HTTP 400** · retryable: **no** · idem: **no**

Prompt exceeds 2000 characters.

### `query_invalid`

<a id="query_invalid"></a>**HTTP 400** · retryable: **no** · idem: **no**

Body fails the CompiledQuery schema; param names the field.

### `parse_failed`

<a id="parse_failed"></a>**HTTP 422** · retryable: **yes** · idem: **no**

The compiler could not produce a query. Rephrase or send a structured query.

### `model_not_found`

<a id="model_not_found"></a>**HTTP 400** · retryable: **no** · idem: **no**

Unknown model. See GET /v1/models.

### `model_deprecated`

<a id="model_deprecated"></a>**HTTP 400** · retryable: **no** · idem: **no**

Model is deprecated. Pin a supported version; see the changelog.

### `analysis_not_cancelable`

<a id="analysis_not_cancelable"></a>**HTTP 409** · retryable: **no** · idem: **no**

Analysis is already in a terminal state.

### `rerun_not_eligible`

<a id="rerun_not_eligible"></a>**HTTP 409** · retryable: **no** · idem: **no**

This analysis is not eligible for a free re-run. Eligible: failed during a platform incident (rerun_eligible: true on the failed analysis) and not already re-run.

### `rate_limit_exceeded`

<a id="rate_limit_exceeded"></a>**HTTP 429** · retryable: **yes** · idem: **yes**

Rate limit exceeded. Back off per Retry-After.

### `concurrency_limit_exceeded`

<a id="concurrency_limit_exceeded"></a>**HTTP 429** · retryable: **yes** · idem: **yes**

Concurrent analysis limit reached. Wait for an active analysis to finish.

### `quota_exceeded`

<a id="quota_exceeded"></a>**HTTP 429** · retryable: **no** · idem: **no**

Quota exceeded for a metered limit. See details.meter.

### `insufficient_credits`

<a id="insufficient_credits"></a>**HTTP 402** · retryable: **no** · idem: **no**

Insufficient credits. Add credits or enable auto-refill.

### `grant_exhausted`

<a id="grant_exhausted"></a>**HTTP 402** · retryable: **no** · idem: **no**

Free grant exhausted. Claim a full account via POST /v1/keys/request to continue.

### `capacity_exhausted`

<a id="capacity_exhausted"></a>**HTTP 503** · retryable: **yes** · idem: **yes**

Platform queue is full. Honor Retry-After.

### `sandbox_limit_exceeded`

<a id="sandbox_limit_exceeded"></a>**HTTP 429** · retryable: **yes** · idem: **yes**

Sandbox provisioning limit reached for this IP. Retry after the window resets, or sign up for a live key.

### `upgrade_limit_exceeded`

<a id="upgrade_limit_exceeded"></a>**HTTP 429** · retryable: **no** · idem: **no**

Too many upgrade attempts from this IP. Retry after the window resets.

### `github_verification_required`

<a id="github_verification_required"></a>**HTTP 403** · retryable: **no** · idem: **no**

A verified GitHub account is required for the free grant. Complete the GitHub device flow and retry with github_token.

### `github_token_invalid`

<a id="github_token_invalid"></a>**HTTP 401** · retryable: **no** · idem: **no**

The github_token could not be verified with GitHub. Complete the device flow again and retry.

### `github_account_already_used`

<a id="github_account_already_used"></a>**HTTP 409** · retryable: **no** · idem: **no**

This GitHub account has already claimed a free grant. One free grant per GitHub account, ever.

### `device_code_expired`

<a id="device_code_expired"></a>**HTTP 410** · retryable: **no** · idem: **no**

Device code has expired or been consumed. Start a new request.

### `inference_unavailable`

<a id="inference_unavailable"></a>**HTTP 503** · retryable: **yes** · idem: **yes**

Transient inference backend outage. Retry with backoff.

### `upstream_error`

<a id="upstream_error"></a>**HTTP 502** · retryable: **no** · idem: **no**

Inference upstream returned an error. Message is sanitized — no internal paths or stack traces.

### `db_unavailable`

<a id="db_unavailable"></a>**HTTP 503** · retryable: **yes** · idem: **yes**

Transient database outage. Retry with backoff.

### `internal_error`

<a id="internal_error"></a>**HTTP 500** · retryable: **yes** · idem: **yes**

Internal error. Includes request_id; contact support if persistent.

### `webhook_endpoint_limit`

<a id="webhook_endpoint_limit"></a>**HTTP 400** · retryable: **no** · idem: **no**

At most 10 webhook endpoints per account.

## Resource-level errors

These never appear as HTTP responses — only in `analysis.error.code` / `video.error.code` on failed resources.

### `inference_error`

<a id="inference_error"></a>Model failed on this input. Safe to create a new analysis; persistent failures on one video → support with request_id.

### `stuck_timeout`

<a id="stuck_timeout"></a>The platform lost the job past the sweep window. Not billed; resubmit.



---

<!-- source: https://www.primateintelligence.ai/docs/changelog.md -->

# Changelog

> Dated API changes, newest-first — canonical copy served by the API deploy artifact. Subscribe via RSS at /docs/changelog.xml.


Sorted newest-first. Agent-readable summary of shipped changes, broken by date.
Full field contracts: [openapi.json](./openapi.json) · [agents.md](./agents.md)

---

## 2026-07-31 — docs/SDK/MCP — DX artifact chain for the PRI-496 public capabilities (PRI-530)

- **Docs**: the mid-stream `update_prompt` → `prompt_updated` protocol is now fully documented (agents.md streaming lifecycle step 5 + openapi POST /v1/streams description), alongside the `status` event vocabulary (`prompt_context | combined_prompt | recalculating`) and the narrative opt-ins. `/llms.txt` gains a Live-narrative section and the client-token (pvct_) line.
- **TypeScript SDK `@primate-intelligence/sdk@0.3.0`**: real `Narrative` type (was a `{text?}` stub), `StreamCreateParams.options.narrative` + `recording`, `Stream.recording`, `streams.recording()`, `videos.listAnalyses()`, `analyses.rerun()`, `Analysis.rerun_eligible`, `Video.media`, `StreamDetection.narrative_update`, `connectStream` `onNarrative`/`onStatus` callbacks + `StreamStatusMessage`.
- **Python SDK `primate-intelligence==0.3.0`**: `streams.recording()`, `videos.list_analyses()`, `analyses.rerun()`; docstrings for the narrative opt-ins and the update_prompt protocol.
- **MCP server 0.3.0** (hosted `/mcp` + `@primate-intelligence/mcp`): 13 tools — adds `get_video` (status + media playback URL), `list_video_analyses`, `rerun_analysis`; `create_analysis` gains `narrative: true`. Output schemas corrected (`narrative {status, entries}`, `Video.media`).
- **Samples**: S6 (TS) and S7 (Python/aiortc) now exercise narrative, status events, mid-stream `update_prompt`, and (S6) session-recording retrieval.
- All shapes verified against the live dev API (recorded narrative stream with status events + update_prompt, 2026-07-31). No API behavior change — documentation/tooling parity only.

---

## 2026-07-31 — internal — web-app library routes accept public ids (PRI-496 Lane B)

- Internal-only (first-party web app; NOT part of the published contract): `GET /v1/user-videos` list rows now carry `public_id` (`video_…`) + `status`; the `/v1/user-videos/:id…` routes accept either the internal UUID or the public `video_` id; `PUT …/results` resolves public `an_` analysis ids to the internal job FK. Enables the web app's cutover to public `POST /v1/analyses` re-runs. Public OpenAPI document unchanged.

---

## 2026-07-31 — fix — narrative-opted-in public streams now activate the engine's narrative path (PRI-510/PRI-496)

- Public streams created with `options: {narrative: true}` now send the engine-config serialization (`narrative_enabled: true`) to the inference engine — previously the engine's narrative path never activated for public streams, so `narrative_update` objects (and the new `status` events) could never appear even with the feature enabled. Non-opted-in streams are unchanged. Public prompt echo now strips ALL internal serializations (byte-identical up to the first `|||`).

---

## 2026-07-31 — fix — stream recording key stored verbatim from the sidecar (PRI-496)

- The public recording linkage now stores the sidecar-reported S3 key **verbatim** (`{env}/recordings/…`) instead of stripping the env prefix. The sidecar's env label is the authoritative namespace of the object it wrote; the API's `NODE_ENV`-derived prefix disagrees on dev (Railway builds run `NODE_ENV=production`), which made dev recording URLs point at `prod/`. Caught in dev verification; prod behavior unchanged (labels agree there).

---

## 2026-07-31 — additive — free re-runs for platform-incident failures (PRI-493) + demo videos documented (PRI-496)

- **Failed analyses now carry `rerun_eligible: boolean`** — `true` when the failure occurred during a declared platform incident (or ops flagged it). Field is present only on `failed` analyses; absent elsewhere. Additive.
- **New `POST /v1/analyses/{id}/rerun`** — creates a **fresh analysis at no charge** from an eligible failed one (same video, prompt/query, model, options; new id; `usage` stays null — never billed). One free re-run per failed analysis; ineligible → `409 rerun_not_eligible` (new registry code). If the re-run's dispatch fails (503) the free re-run is NOT consumed. Normal capacity/brownout gates still apply.
- **`GET /v1/demo-videos` is now documented** in openapi.json/agents.md as a public (unauthenticated) onboarding surface: curated sample videos with precomputed results per example prompt. Documented subset is the stable contract.
- Decisions recorded for the remaining small gaps: stream limits already ship in the `POST /v1/streams` response (`limits`); `/v1/feedback` and `/v1/prompts` remain console-internal.

---

## 2026-07-31 — additive — stream session recordings + public status events (PRI-496)

- **`POST /v1/streams` accepts `recording: true`** (optional boolean, default false): opt into retrieval of the server-side session recording. The stream then carries a new nullable `recording: {status}` object (`recording` · `available` · `failed` · `none`); streams that don't opt in keep `recording: null` — no existing field changed.
- **New `GET /v1/streams/{id}/recording`** — returns `{object: "stream_recording", stream_id, url, expires_at, content_type, container}` with a **signed playback URL, fresh on every GET, 1-hour TTL** (sign-on-read, same semantics as `Analysis.artifacts`). The recording is a raw H.264 Annex-B elementary stream — lossless remux to MP4 with `ffmpeg -i in.h264 -c copy out.mp4`. 404 when recording wasn't enabled; 409 while the stream is active, when capture failed, or when nothing was stored.
- **New additive `status` server→client event on the `/v1/streams/{id}/signal` WS**: `{type: "status", status: "prompt_context"|"combined_prompt"|"recalculating", message?}` — the narrative-ordering vocabulary, forwarded ONLY for streams that opted into `options: {narrative: true}` (and only while the narrative feature flag is enabled). Closed set at the edge; internal serializations are stripped from `message`. Existing vocabulary (`ready/queued/answer/ice/live/result/prompt_updated/metering/warning/end/error/pong`) is untouched; non-opted-in streams see zero change.

---

## 2026-07-31 — additive — video library parity: source playback URL + per-video analysis history (PRI-496)

- Fixture videos (the sandbox plane's canned sample) return `media: null` — they are pseudo-objects with no real S3 source; a signed URL would 404.
- **`Video.media` is now populated** on `ready` videos: `{url, expires_at}` — a signed playback URL for the original source video, generated **fresh on every read** with a 1-hour TTL (sign-on-read, same semantics as `Analysis.artifacts`). Re-fetch the video for a new URL after expiry; old videos always stay playable. `media` is `null` for non-ready videos and rows without a stored source object. New nullable field — additive, no existing field changed.
- **New `GET /v1/videos/{id}/analyses`** — lists every analysis run against a video (newest first), same list envelope + filters (`status`, `limit`, `starting_after`) as `GET /v1/analyses`. Resource-nested spelling of `GET /v1/analyses?video_id={id}`; both work. This plus `media` plus `artifacts` gives the public plane full video-library semantics (list, replay, per-video result history).

---

## 2026-07-30 — breaking (gated) — GitHub OAuth gate on the free grant upgrade (PRI-479)

- **`POST /v1/keys/upgrade` now requires a verified GitHub account** when the gate is enabled (it is, on dev + prod). Calling without a `github_token` returns **`403 github_verification_required`** whose `details` carry the full device-flow bootstrap: `client_id`, `device_code_url`, `access_token_url`. Complete GitHub's standard device flow (no scopes requested — identity only) and retry with `{"github_token": "gho_..."}`.
- **One free grant per GitHub account, ever.** The numeric GitHub account id is stamped on the user row (DB unique index); a GitHub account that already claimed its grant gets **`409 github_account_already_used`**. The OAuth token itself is verified with GitHub and discarded — never stored.
- New error codes in the §15 registry: `github_verification_required` (403), `github_token_invalid` (401), `github_account_already_used` (409).
- Why: the upgrade was IP-rate-limited only — a key-churning agent rotating IPs could harvest unlimited 6,000s GPU grants. GitHub's own abuse controls now bound free grants at one per real account; IP caps remain as secondary defense.
- Sandbox provisioning (`POST /v1/sandbox`) and the billed device-code claim flow (`POST /v1/keys/request`) are unchanged.

---

## 2026-07-30 — fix — internal/fixture accounts excluded from product metrics (PRI-472)

- **New `users.is_internal` flag** (migration 036: `boolean NOT NULL DEFAULT false` + partial index + pre-filtered `analytics_users` view). Backfilled for the Anthropic directory-review fixture, all loadtest accounts, and E2E fixtures.
- **All admin analytics endpoints now exclude internal accounts**: `/v1/admin/analytics/growth` (totals, signups, activation, W1 retention, MRR/ARR/paying users, North-Star minutes), `/v1/admin/pmf` (signups_total, activation funnel, repeat-user retention), `/v1/admin/analytics/credits` (card attach rate, refill conversion, API activation), `/v1/admin/analytics/verticals` (user vertical/replacing breakdowns).
- **Clerk `user.created` webhook** now propagates `public_metadata.internal_fixture: true` → `users.is_internal`, so future fixture accounts are excluded at creation.
- Historical note: metric values step down slightly vs. previous reports (fixture activity was previously counted — e.g. 105 loadtest users on prod). This is a correction, not a regression.
- Admin response fields only — no public API schema change.

---

## 2026-07-30 — additive — API host agent onramp: GET / index + GET /robots.txt (PRI-492)

- **`GET /` now returns a JSON index** instead of a 404 error envelope: `{name, description, openapi, docs, llms, agents, changelog, mcp, sandbox, health}` — an agent probing the API host root gets pointers to every machine-readable surface (OpenAPI spec, llms.txt, agents.md, MCP endpoint, one-POST sandbox key).
- **`GET /robots.txt` now returns 200 text/plain** (allow all) instead of the 404 JSON envelope, and points crawlers at `/llms.txt`, `/v1/openapi.json`, and `/docs/agents.md`.
- Both endpoints are unauthenticated. No existing route changed; no schema change.

---

## 2026-07-30 — fix — GET /v1/models: darwin-1.3 capabilities.streaming corrected to true (PRI-496)

- The model catalog still said `capabilities: {streaming: false}` from before real-time streams shipped. Streaming has been live for weeks (`POST /v1/streams`); the flag now says so. Data correction only — no schema change.

---

## 2026-07-30 — additive — Analysis.artifacts now populated: annotated result video URL (PRI-496)

- **`GET /v1/analyses/{id}` (and list) now populates `artifacts`** for completed analyses that produced an annotated result video: `{annotated_video_url, expires_at}` per the existing `ArtifactsSchema`. The URL is a **fresh 1-hour signed CDN link generated on every read** — when it expires, re-fetch the analysis for a new one (old results always stay retrievable). Analyses without an annotated video (and all non-completed statuses) continue to return `artifacts: null`.
- The schema stub has been in the contract since P3 — this makes it live. Zero breaking changes; agents that treated `artifacts` as always-null should start reading it.

---

## 2026-07-30 — additive — narrative over the public API: options.narrative on analyses + streams (PRI-510)

- **`POST /v1/analyses` (and `/batch`) `options.narrative: true` is now honored.** The completed analysis carries a populated `narrative` object per the existing `NarrativeSchema`: `{status: "generating"|"ready"|"failed", entries: [{t_s, text}]}` — timestamped event sentences generated after completion (poll `GET /v1/analyses/{id}`; the `analysis.completed` webhook may still show `status: "generating"`). Omitted/false preserves the previous behavior exactly (`options: {narrative: false}`, `narrative: null`). Narrative is **included in the analysis price** — no surcharge, no usage-shape change.
- **`POST /v1/streams` accepts `options: {narrative: boolean}`** (previously rejected as an unknown field). Opted-in streams receive a `narrative_update` object (`{t_s, text}`) on result-frame detections whenever the engine produces a new sentence — event-driven, not per frame; the key is absent on frames without one. Disabled/omitted streams never see it. Terminal `results_summary.last_detections` continues to strip it (frozen shape).
- **Test mode (`pv_test_`)**: opted-in canned analyses return a deterministic canned narrative (`status: "ready"`) — fixture plane, zero quota.
- The schema fields (`options.narrative`, nullable `narrative`, `NarrativeSchema`) were already in the contract as forward-compatible stubs — this release makes them live. Zero breaking changes (oasdiff clean).

---

## 2026-07-30 — internal — inference-WS replacement-storm alerts now reach Sentry (PRI-418); load-test metric renamed to metadata_ratio (PRI-360)

- **Replacement-storm detector alerts to Sentry** (PRI-418): when inference-WS replacements exceed the threshold (default 10/60s), the existing error-level structured log is now also sent via `Sentry.captureMessage` at error level, so storms like the 2026-07-02 duplicate-sidecar incident page instead of only landing in stdout. Cooldown unchanged (max one alert per 5 min). No public API contract change.
- **Load-test harness: `metadata_ratio` metric added** (PRI-360): the k6 shared metrics library gains a `metadata_ratio` Rate (fraction of incoming sidecar WS messages that are `metadata` frames), wired into `09-stream-baseline.js`. Replaces the informal "ack ratio" name — the counted frames are metadata, not acks. Tooling-only; no runtime behaviour change.

---

## 2026-07-29 — additive — batch discount is config-driven and served by GET /v1/credit-pricing (PRI-508)

- **`GET /v1/credit-pricing` now returns `batch_discount_pct`, `batch_min_prompts`, and `batch_max_prompts`.** The `POST /v1/analyses/batch` discount (each prompt after the first) was a hardcoded constant that could drift from the documented "50%"; it now lives in the same `credit_price_config` row as every other price knob and is served publicly here. **Read the discount from this endpoint instead of hardcoding it** — a pricing change is a config update, not an API version bump. Current values match shipped behaviour exactly: `batch_discount_pct: 50`, `batch_min_prompts: 2`, `batch_max_prompts: 10`.
- **`discount_pct` in batch responses** (`analysis_batch.pricing`, `analysis_batch_preview.prompts[].discount_pct`) now reflects the configured value (still `50` today). The preview per-prompt field is documented as an integer rather than the literal set `0 | 50` — `0` still means the full-price first prompt.
- No behaviour change at current config: batch pricing, bounds, and copy are byte-identical until the config row changes.

---

## 2026-07-29 — fix — /v1/test-fixture video URL follows the app.* replatform; anonymous sandbox IP cap now per-client

- **`GET /v1/test-fixture` `test_video_url` updated** to `https://app.primateintelligence.ai/empty-state/forklift-demo.mp4`. The web replatform moved the product SPA (which serves the fixture file) to `app.primateintelligence.ai`; the old apex path stopped resolving when the apex became the marketing site. If you pinned the old URL in CI, update it — the file bytes are unchanged. The canonical pinned demo asset `https://primateintelligence.ai/demos/empty-state/forklift-demo.mp4` is unaffected.
- **`POST /v1/sandbox` IP cap fixed to key on the real client IP.** Previously the daily provision cap could key on a shared proxy hop, which made the limit far stricter than documented for users behind the same edge. No contract change — same `sandbox_limit_exceeded` error, same documented 3/day per-IP semantics, now enforced per actual client.

---

## 2026-07-29 — docs-only — POST /v1/parse documented in openapi.json; GET /v1/analyses status-filter vocabulary documented

- **`POST /v1/parse` is now in the OpenAPI document** (PRI-496 P0). The endpoint has been live for months (stateless prompt compiler: `{prompt}` → `{compiled_query, parse_mode: "haiku"|"heuristic"}`; auth optional; 30 req/min per IP; errors `empty_prompt` 400 / `parse_failed` 422 / `parse_unavailable` 503) but was undocumented — for agents, undocumented is nonexistent. No behavior change. Also added to the `/llms.txt` key-endpoints list.
- **`GET /v1/analyses` `status` filter vocabulary documented**: `queued | preparing | analyzing | rendering | completed | failed | canceled` (§13.2). The filter has always worked (unknown values → 400 `validation_failed`); its accepted values were just not written down. Documented in the parameter description — the machine schema stays `type: string`, so nothing tightens contractually. Use `status=analyzing` for currently-running analyses.
- Zero contract changes — the public surface is frozen; this is documentation of already-shipped behavior.

---

## 2026-07-28 — additive — SDK 0.2.0 (TS + Python) and MCP 0.2.0: full parity with the July API rounds

- **TypeScript SDK `@primate-intelligence/sdk@0.2.0`** (breaking — type corrections): `AnalysisUsage` fixed to the real contract `{billed_seconds, credit_balance_after}`; `StreamEndReason` fixed (adds `canceled`/`ice_failed`/`media_timeout`, drops never-returned `client_disconnect`); `video_duration_s` nullable; `Analysis` gains `livemode` + `origin`. New: `client.credits.retrieve()` (`GET /v1/credits`), `client.creditPricing.retrieve()`, `analyses.validateOnly()`, `analyses.createBatch()`/`validateBatch()`, `detected_count` + `indeterminate_reason` on results, typed streaming contract in `/browser` (`session_remaining_s` metering, `no_media_frames` warning union, typed detections, `failure_diagnostic`, typed `results_summary`). Migrate off the `balance_s`/`results_summary.frames` aliases before their ~2026-08-28 removal.
- **Python SDK `primate-intelligence==0.2.0`** (additive): `client.credits` + `client.credit_pricing` resources, `analyses.validate_only()`, `analyses.create_batch()`/`validate_batch()`, docstrings updated to the real result/streaming contracts.
- **MCP 0.2.0 (hosted `/mcp` + npm `@primate-intelligence/mcp`)**: three new tools — `validate_analysis` (free dry-run with cost estimate), `create_analysis_batch` (batch discount), `get_credits` (transaction ledger). `get_analysis`/`wait_for_analysis` descriptions now document `detected_count`, `indeterminate_reason`, nullable duration, and the usage snapshot. The npm package catches up with the hosted server: every tool declares `outputSchema` and returns `structuredContent`; `wait_for_analysis` returns the `{analysis, retry}` envelope (the legacy `_mcp_note` field merged into the analysis resource is gone from the npm package too).

---

## 2026-07-28 — failure_diagnostic hint wording: customer-appropriate, no internal references

- **`failure_diagnostic.hint` (on `end_reason: "ice_failed"`) reworded.** The hint previously referenced an internal server environment variable — meaningless (and confusing) to API consumers. It now states plainly which side the fault is likely on and what to do: when the server did not advertise a publicly reachable network candidate → *“server-side configuration issue — nothing to change on your end; retry, and contact support if it persists”*; when the server did advertise public candidates → *“check your network allows UDP or TCP/443 outbound and that your client uses the TURN servers in `ice_servers`”*. Field shape unchanged (`{local_candidates, hint}`); only the hint text changed. Same sweep reworded the transient status messages sent on the signaling WS (`inference_unavailable`, capacity/restart rejections) to remove internal jargon.

---

## 2026-07-28 — Metering tick + results_summary field renames (deprecation aliases until ~2026-08-28)

- **Metering tick: `balance_s` renamed to `session_remaining_s`** (Streaming DX Finding 2). The value was always the seconds remaining in **this session's** credit reservation (the session cap) — the old name read like account credit balance, which lives at `GET /v1/billing/credits` (`balance_seconds`). Every tick now carries BOTH `session_remaining_s` (canonical) and `balance_s` (deprecated alias, identical value). **`balance_s` will be removed ~2026-08-28** — migrate reads to `session_remaining_s`.
- **Terminal `results_summary`: `frames` renamed to `result_frames`** (Streaming DX Finding 3). Results are **sampled**, not per-source-frame (adaptive inference cadence, roughly every 8th source frame under load), so the count is the number of result **events emitted** — `result_frames` says that; `frames` implied a source-frame count. Both fields present (identical value); **`frames` will be removed ~2026-08-28** — migrate reads to `result_frames`.
- **Docs: result-sampling semantics now documented** in agents.md — `frame_num` on live results is the *source-frame index* (gaps between consecutive values are normal); `results_summary.result_frames` counts result events; expect roughly 8–15 results/s depending on load.
- No behavior changes — both renames are additive aliases for 30 days, then the deprecated names drop.

---

## 2026-07-28 — No-media warning: the server now tells you when your media is the problem

- **New signaling-WS event: `warning {code: "no_media_frames"}`** — when the WebRTC transport connects but no decodable video frame arrives within 5 seconds, the server pushes `{type:"warning", code:"no_media_frames", transport_connected:true, elapsed_s, packets_received, frames_decoded, hint}` (re-warned once at 15s, then quiet; canceled by the first decoded frame). Previously the server stayed silent for the full media-timeout window (~50s) with zero client-visible signal. `packets_received` distinguishes the two failure classes: `0` = your client is not sending media (track dead/muted); `>0` with `frames_decoded: 0` = media arrives but is not decodable (wrong codec, or a source producing no real frames — e.g. canvas capture without a draw loop).
- **`end_reason: "media_timeout"` now carries the media counters in `failure_diagnostic`** when a connected-but-no-media session ends without ever going live. Billed 0, as before.
- Existing `warning {remaining_s}` credit-exhaustion event is unchanged (distinguish by the `code` field — credit warnings have no `code`).

---

## 2026-07-28 — TURN-over-TLS (`turns:`) relay advertised for TLS-only egress networks

- **ICE server list now always includes a `turns:<host>:443?transport=tcp` relay** in session credentials. Twilio's NTS token response carries `turn:` relays (UDP/TCP 3478, TCP 443) but no `turns:` URL, so clients behind TLS-only firewalls (datacenter/corporate egress allowing only TLS on 443) could never reach a relay and sessions failed with `ice_failed`. The API now synthesizes the `turns:` entry from the existing relay host, reusing its ephemeral credentials. No-op when the provider already returns a `turns:` URL or in STUN-only fallback. No request/response field changes — the `ice_servers` array simply gains one entry.

---

## 2026-07-28 — MCP tools declare outputSchema + structuredContent; wait_for_analysis timeout shape cleanup

- **All 7 hosted MCP tools now declare an `outputSchema`** (in `tools/list` on `POST /mcp` and on the static server card at `/.well-known/mcp/server-card.json`). Schemas are generated from the same Zod DTOs that validate the public /v1 responses (`src/dto/schemas.ts`), so they cannot drift from the real contract. Per the MCP spec, every tool result now also carries `structuredContent` conforming to its declared schema (the SDK validates each result at runtime); the human-readable JSON text content is unchanged.
- **`wait_for_analysis` response shape changed** (privacy-audit finding F-1): the tool now returns `{ analysis, retry }` — the analysis resource unmodified under `analysis`, and `retry: null` on terminal states or `retry: { reason: "timeout", note }` when the wait expired. The old behavior merged an undocumented `_mcp_note` field into the analysis object on timeout, mutating a documented resource shape. Agents reading the analysis should use `result.analysis` (or `structuredContent.analysis`).
- **`query` field description** on analysis outputs now states explicitly that it is the compiled, deterministic interpretation of the caller's own prompt — a transparency feature (privacy-audit finding F-2, documentation only; no data change).
- REST API surface unchanged — this release only affects the MCP endpoint metadata and the MCP-layer `wait_for_analysis` envelope.

---

## 2026-07-28 — OpenAI apps domain verification + explicit destructiveHint on read-only MCP tools

- **New public endpoint: `GET /.well-known/openai-apps-challenge`** — returns the OpenAI apps domain-verification challenge token (`OPENAI_APPS_CHALLENGE_TOKEN` env var) as raw `text/plain`; 404 when unset. Unauthenticated by design — OpenAI's plugin/apps portal fetches it to verify domain ownership (PRI-475). Not part of the product API surface.
- **MCP tool annotations: explicit `destructiveHint: false`** added to all five read-only tools (`get_analysis`, `wait_for_analysis`, `list_models`, `get_usage`, `get_test_fixture`) in both the live `tools/list` response and the static server card at `/.well-known/mcp/server-card.json`. Previously the hint was omitted (spec default is `true`), which OpenAI's tool scanner flagged. No changes to tool names, descriptions, or input schemas.

---

## 2026-07-28 — Streaming DX: server ICE fix + trickle, honest end reasons, unified result contract, public examples repo

### Transport (the critical fix)
- **Server-side ICE candidates are now reachable from anywhere** — the streaming server advertises its public address as a host candidate and **trickles late-gathered srflx/relay candidates** to the client as `ice` messages after the answer SDP (previously candidates that outlasted the bounded gathering window were silently dropped, so relay-only clients — datacenters, CI, UDP-blocked networks, strict NATs — stalled in `ice: checking` until timeout while browser clients survived via peer-reflexive discovery). Keep consuming `ice {candidate}` messages after `answer`.
- **Relay-only regression client in CI** — every deploy is tested with an aiortc client that offers only TURN relay candidates (TURN-over-TCP:443), the datacenter/edge-fleet profile.

### Honest `end_reason` (breaking-adjacent: two new enum values)
- **New values: `ice_failed`, `media_timeout`** on `stream.end_reason` (WS `end {reason}` and the terminal Stream resource). A session that never went live is **never** `completed` — enforced server-side.
- **New nullable field: `failure_diagnostic`** on the Stream resource — on `ice_failed`, carries `{local_candidates, hint}` (the server-side candidate summary) so transport failures are debuggable from the API alone.
- Billing unchanged: never-live sessions bill exactly 0 (join/negotiation/queue free).

### Streaming result contract unified with file analyses (breaking for undocumented internals)
- `result.detections[].answer` is now **lowercase `yes|no|indeterminate`** — the same enum as file-API results (was `"Yes"/"No"`, an internal casing leak; enums never vary by transport).
- `result.detections[].prompt` now echoes **exactly what you submitted** — the internal `|||COMPILED_QUERY:<base64>` serialization no longer leaks into the public field.
- Latency telemetry (`inference_ms`, `server_elapsed_ms`, per-hop p50s, per-stage `latency` breakdown) is grouped under a documented **`timing`** object. Internal scheduling fields (`queue_fill`, `queue_length`, `is_warm`, `video_enabled`) are no longer emitted.
- Same normalization applies to `results_summary.last_detections` on the terminal resource.

### Docs & examples
- **Public examples repo**: https://github.com/Primate-Intelligence/primate-examples — Python streaming client (aiortc, the "stream a file" recipe), browser webcam example, relay-only CI harness. The old private-repo examples link in the streaming guide is gone.
- **agents.md gains a full Streaming section** — lifecycle, signaling protocol incl. server trickle, the streaming result contract, the honest end_reason vocabulary, the datacenter client profile, and billing semantics.

---

## 2026-07-27 — PRI-482 round 8: one-clock docs — canonical docs served by the API, website proxies them, hardened doc-sync CI gate, count-confidence semantics clarified

### Documentation architecture (the "separate clocks" fix)
- **Single source of truth** — `docs/agents.md` and `docs/changelog.md` ship inside the API deploy artifact and are served at `https://api.primateintelligence.ai/docs/agents.md` / `/docs/changelog.md`. The website (`https://primateintelligence.ai/docs/agents.md` and `/docs/changelog.md`) now **proxies these API routes** instead of serving a hand-copied duplicate. Spec, quickstart, and changelog update atomically with the code they describe — there is no copy step left to go stale.
- **RSS moves to the API** — `GET /docs/changelog.xml` is generated from `docs/changelog.md` at request time, so the feed can never drift from the changelog it is derived from.
- **Hardened doc-sync CI gate** — a diff touching `src/routes/**`, `src/dto/paths.ts`, or billing/pricing services (`credit-ledger-service`, `billing-service`, `usage-meter-service`, `quota-service`) now fails CI unless **both** a docs surface (`docs/agents.md` or `docs/openapi.json`) **and** `docs/changelog.md` move in the same diff. Previously route changes required only one doc surface and billing changes required none.
- **Nightly drift monitor** — an external probe diffs the website-served docs against the API canonical and alerts on any byte mismatch (belt-and-suspenders against pipeline-level failures CI can't see).

### Contract clarification
- **Count-query confidence** — agents.md (Result Contract + Count queries sections) and `llms.txt` now state explicitly: for count queries, `confidence` applies to the detected count itself — it is the model's confidence that `detected_count` is the correct number, not merely that something was detected.

---

## 2026-07-27 — PRI-482 round 7: batch validate_only, enriched batch pricing, llms.txt batch listing, docs-atomicity CI gate, result-contract confidence cross-reference

### New capabilities
- **`POST /v1/analyses/batch` with `validate_only: true`** — dry-run for batch: parses all prompts, returns `analysis_batch_preview` (HTTP 200) with per-prompt `query`, `parse_mode`, `assessable`, `estimated_seconds`, `estimated_cost_usd`, and `discount_pct`; plus batch-level totals `estimated_total_seconds` and `estimated_total_cost_usd`. No credits reserved, no jobs created. Estimates are `null` when the video has no known duration.
- **Enriched batch pricing on real create** — `POST /v1/analyses/batch` response `pricing` block now includes `estimated_total_seconds` and `estimated_total_cost_usd` (null when duration unknown), so callers can see the expected cost without a separate `validate_only` round-trip.

### Docs & CI
- **`llms.txt` batch endpoint** — `POST /v1/analyses/batch` added to the `## Key endpoints` section.
- **Docs-atomicity CI gate** — `scripts/changelog-gate.ts` now enforces a second rule: any diff touching `src/routes/**` or `src/dto/paths.ts` must also update `docs/agents.md` or `docs/openapi.json`. Bypass with `[skip-docs-gate]` in a commit message.
- **Result Contract confidence cross-reference** — `result.confidence` bullet in agents.md "Result Contract" section now notes that for count queries confidence reflects detection certainty (not count certainty), with a link to the Count queries section.
- **agents.md batch section** — added `validate_only` documentation with worked JSON example.
- **OpenAPI** — `validate_only` added to `CreateAnalysisBatchRequest`; `AnalysisBatchPreview` response schema added; `AnalysisBatch.pricing` updated with estimated fields.

---

## 2026-07-27 — PRI-482 round 6: batch discounts real, confidence semantics, key lifecycle in OpenAPI, 404 envelope, ledger source_type, historical balance freeze

### New endpoints
- **`POST /v1/analyses/batch`** — submit 2–10 prompts against the same video in one request. First prompt billed at full price; each additional prompt billed at **50%**. Credits reserved per-prompt at the discounted rate (observable directly in `GET /v1/credits` ledger). Response includes `analyses[]` array + `pricing: {full_price_prompts, discounted_prompts, discount_pct: 50}`. Registered in OpenAPI spec and documented in agents.md "Batch analyses & discounts" section.
- **`POST /v1/keys/upgrade`** and **`POST /v1/keys/request` / `GET /v1/keys/request/{code}`** now registered in `docs/openapi.json`. These routes existed but were absent from the spec — they are the first two calls any agent makes.

### Error handling
- **404 catch-all** — unknown routes previously returned the framework-default `{"message":"Route ... not found"}`. They now return the standard typed error envelope (`code: "resource_not_found"`, same shape as all other errors).
- **`POST /v1/analyses` with `prompts` array** now returns a helpful validation error pointing to `POST /v1/analyses/batch` instead of a generic "Unknown field".

### Billing fixes
- **`source_type: "analysis"` for analysis debits** — credit ledger rows for public-API analyses were labelled `source_type: "upload"`. They are now labelled `source_type: "analysis"` (new DB constraint value). Migration 077 backfills existing reservation and transaction rows in dev. Agents reading `GET /v1/credits` will see `source_type: "analysis"` on analysis debits going forward.
- **Historical `credit_balance_after` frozen** — analysis rows settled before migration 075 had a null `balance_after_seconds` snapshot, causing `credit_balance_after` to drift as later analyses ran. Migration 078 stamps those rows with the current account balance, freezing the value. `credit_balance_after: null` (`snapshot_unavailable`) now only appears when the account balance itself is unavailable.

### Docs & contract
- **Confidence semantics for count queries** — added a paragraph to agents.md "Count queries — the contract" section: `confidence` is the aggregate detection confidence (max of per-term values, [0,1]); forced to 0 when `answer: "indeterminate"`; `≥0.7` reliable, `0.4–0.69` marginal, `<0.4` unreliable.
- **Changelog link in agents.md** — added a dedicated "Changelog" section linking `/docs/changelog.md`.
- **`snapshot_unavailable` semantics** — `usage.credit_balance_after: null` documented in agents.md Result Contract.
- **Walking fixture section** added to agents.md "Action pipeline verification" (clip URL pending upload).

### Migrations applied to DEV
- `077_analysis_source_type.sql` — adds `'analysis'` to `credit_reservations.source_type` constraint; backfills existing rows.
- `078_freeze_historical_balance_after.sql` — stamps null `balance_after_seconds` on settled reservations with current account balance.

---

## 2026-07-27 — PRI-480 round 5: agent DX — versioned responses, immutable billing snapshot, truthful origin, pricing discovery

### Breaking changes
- **`analysis.origin` enum changed: `user` → `api`, new value `console`.** The round-2 entry below introduced `origin: "user" | "system"` derived at read time from `source_type` — that heuristic misclassified console (dashboard) uploads as `system` and recorded nothing. `origin` is now stamped truthfully at job creation: `api` (public API), `console` (dashboard upload), `system` (internal — auto-probe, fixture seeding, run_prompt). Legacy rows without the column fall back to the heuristic (`public_api*` → `api`, `upload` → `console`, else `system`). The field shipped yesterday with near-zero consumers.

### New fields & headers
- **`X-Api-Version` response header** — every response (including errors) carries `<package-version>+<git-sha>`. If the value changes mid-session, a deploy happened — re-check this changelog before assuming a bug.
- **`usage.credit_balance_after` is now immutable.** Previously it re-read the live account balance at GET time, so an old analysis's value mutated as later analyses ran. It is now a point-in-time snapshot stamped at settlement (`credit_reservations.balance_after_seconds`). Analyses settled before this deploy fall back to the live balance (previous behaviour).

### Billing invariant
- **System-initiated analyses are never billed.** `origin: "system"` jobs (e.g. internal run_prompt) never reserve or settle credits — enforced by a CI test auditing every `reserveCredits` call site.

### Docs & discoverability
- **`GET /v1/credit-pricing`** (public, no auth) is now in the OpenAPI spec, the [agents.md](./agents.md) Pricing section, and `/llms.txt`. Returns `price_per_second_cents`, `signup_grant_seconds`, `card_grant_seconds` — cost of an analysis = `usage.billed_seconds × price_per_second_cents`.
- **Count-query contract** section added to [agents.md](./agents.md): `detected_count` is only meaningful when `answer` is determinate; success/failure examples are schema-validated in CI.
- **Changelog is now CI-enforced** — any push/PR diff touching `src/**` must also update this file with a well-formed topmost `## YYYY-MM-DD — <title>` entry (`scripts/changelog-gate.ts` + `changelog-gate.yml`).
- **Late note (shipped earlier today in commit df7f0ce without an entry):** `GET /docs/agents.md` and `GET /docs/changelog.md` are now served publicly — `/llms.txt` referenced both but they 404'd.
- Changelog re-sorted newest-first (the 2026-07-24/22/18 entries were filed below 2026-07-09).

---

## 2026-07-27 — PRI-480 round 2: result invariants, synchronous metadata, billed_seconds, origin

### New fields
- **`analysis.origin`** — `"user"` (submitted via the public API) or `"system"` (generated internally, e.g. auto-probe or fixture seeding). Agents should normally only act on `origin: "user"` analyses.

### Behaviour fixes
- **Result consistency invariants.** A completed analysis returning `detected_count: 5` + `answer: "indeterminate"` + `indeterminate_reason: "nothing_detected"` is now repaired at the API boundary: (a) `nothing_detected` forces `detected_count` to 0, (b) `indeterminate` forces `confidence` to 0, (c) determinate `yes`/`no` answers never carry an `indeterminate_reason`.
- **Video metadata populated synchronously on `/complete`.** `POST /v1/videos/:id/complete` now awaits the ffprobe enrichment (15 s timeout) before building the response, so `duration_s`, `width`, `height`, and `fps` are present in the response body and in immediate `GET /v1/videos/:id` reads. On timeout the endpoint returns the row as-is and enrichment continues in the background.
- **`billed_seconds` on list analyses now correct.** The `GET /v1/analyses` list was querying `credit_reservations.job_id` (a column that does not exist). Fixed to join via `jobs.reservation_id → credit_reservations.id`, matching the single-GET path. `usage.billed_seconds` is now populated on all settled list items.

### New endpoints
- **`GET /llms.txt`** — plain-text LLM index following the llms.txt convention. No auth required. Lists the API base URL, `/docs`, `/openapi.json`, and changelog. Lets AI coding assistants discover the API surface without a browser.

### Docs
- **Query-type maturity note** added to [agents.md](./agents.md): presence and counting queries are mature/GA; action and temporal queries are beta (duration-sensitive); open-ended prompts are rejected with `unsupported_query_form` rather than billed.

---

## 2026-07-27 — PRI-480: API hardening, credits, and fixture parity

### Breaking changes
None.

### New fields
- **`result.video_duration_s`** — now `number | null` (was always `number`). Returns `null` when the source video duration could not be determined (short uploads, metadata-less files). Agents must null-check this field.
- **`result.indeterminate_reason`** — why the answer is `indeterminate`. Enum:
  - `low_confidence` — model saw something but confidence was below threshold; try a more specific prompt.
  - `nothing_detected` — all confidence scores were 0; no subject detected in any frame.
  - `unsupported_query_form` — open-ended or freeform query; rephrase as a closed yes/no or count question.
  - `duration_mismatch` — inference result duration disagrees with source video duration (>20%); result is untrusted.
- **`result.detected_count`** — integer count for count-intent queries ("how many X…"). `null` on yes/no queries.

### New endpoints
- **`GET /v1/credits`** — returns `{ balance_seconds, grant_seconds, used_seconds, transactions: [...] }`. Per-analysis transaction history. Lets agents audit every charge independently. No parameters needed; key-scoped.

### New parameters
- **`POST /v1/analyses` → `validate_only: boolean`** — parse + estimate cost, no GPU, no credits deducted. Returns `AnalysisPreview` with `estimated_cost_seconds`, `query_type`, and `answerability` signal. Use before committing to an analysis.

### Behaviour fixes
- **Action pipeline duration now deterministic.** The action query path (`how many X…`) was returning GPU wall-clock time as `video_duration_s` instead of source video duration. For a 6s video this produced 40–75s billing. Root cause: `duration_seconds` was absent from the action result dict in the inference server; the API fell back to GPU wall-clock. Fixed in inference I-1 (inference commit `4e0825a`).
- **`detected_count` now populates for action queries.** The count was computed by inference but not forwarded in the callback payload. Fixed in inference I-2.
- **`indeterminate_reason` forwarded from inference.** When inference explicitly sets a reason, the API now respects it rather than re-classifying.
- **Duration invariant cross-check.** If inference returns a duration >20% different from the admitted source duration, the API flags the result `indeterminate` with `indeterminate_reason: duration_mismatch` rather than silently returning wrong timing.
- **Open-ended query gate.** Submitting a query the model cannot answer (open-ended form) now returns `answer: indeterminate`, `indeterminate_reason: unsupported_query_form` immediately, at zero cost, instead of burning GPU credits and returning an unreliable result.
- **Video metadata probed at upload-complete.** `duration_seconds`, `width`, `height`, `fps` are populated on the video object immediately after `POST /v1/videos/:id/complete`. Fire-and-forget ffprobe enrichment runs after response is sent for non-faststart files.

### Test fixture update
`GET /v1/test-fixture` now returns a `fixtures[]` array with two entries:
- `presence` — forklift yes/no (existing fixture, unchanged)
- `action` — forklift count query; assert `detected_count >= 1`

Legacy flat fields (`test_video_url`, `test_prompt`, `expected_answer`, `expected_confidence_min`) preserved for backwards compatibility.

---

## 2026-07-24 — PRI-477: billing cap, detected_count, error envelope

### New fields
- **`result.detected_count`** — integer count on count-intent queries (object path).
- **`analyses.list` response** — includes `billed_seconds` per analysis.
- **Error envelope** — all errors now return `{ error: { code, message, request_id } }` (was bare `{ error, message }`).

### Behaviour fixes
- **Billing cap** — per-analysis billing is capped at the source video duration. A slow GPU run (warmup, recompile) cannot overbill beyond the video length.
- **`answer: indeterminate`** — when confidence is 0 and all term confidences are 0 the answer is now `indeterminate` instead of `no`. A `no` that can't be substantiated is indistinguishable from a model that saw nothing.
- **`usage` deprecated** — the `GET /v1/usage` response is kept but `usage` inline on analysis objects has moved to `GET /v1/credits`.

---

## 2026-07-22 — PRI-476: inline usage, url-ingest probe, liveProgress

### New fields
- **`analysis.usage`** — inline `{ billed_seconds, credit_balance_after }` on completed analyses. Agents no longer need a second GET /v1/usage round-trip.
- **URL-ingest video probe** — `POST /v1/videos` with `source_url` now runs the same metadata probe that upload-complete uses.

---

## 2026-07-18 — Public API v1 GA

Initial public release of the REST API with:
- `POST /v1/sandbox` — instant `pv_test_` key, no email, no card
- `POST /v1/keys/upgrade` — upgrade to `pv_live_` + 6,000s free grant
- `POST /v1/videos` — presigned upload or URL ingest
- `POST /v1/analyses` — submit prompt + video → analysis
- `GET /v1/analyses/:id` — poll or use `Prefer: wait=60`
- `GET /v1/analyses` — paginated list
- `GET /v1/credits` — balance + transactions
- `GET /v1/test-fixture` — stable CI fixture

---

## 2026-07-12 — Credit notifications, ToS consent, CORS

### New features
- **Expiring-credits reminder email** — sent once per credit lot, 7 days before expiry. Requires opt-in notification prefs.
- **Low-balance email** — fires once per episode (was: every 24h).

### Behaviour fixes
- **Admin API-key mint/rotate** now stamps implied Terms of Service acceptance.
- **Admins exempt from ToS gate** on stream and upload endpoints.
- **CORS preflight** now allows `X-Primate-Client` header (dashboard regression fix).

---

## 2026-07-09 — PRI-440/439: P11 hardening, pricing analytics, SDK 0.1.1

### New features
- **Brownout ladder + status page** — spend anomaly auto-suspend with configurable ladder (`BROWNOUT_STATE`). Public status reflected at `/v1/status`.
- **SSRF IP-pinning** — all outbound URL fetches (e.g., source_url ingest) are blocked from internal RFC-1918 and link-local ranges.
- **Key-hygiene redaction** — raw key values are redacted from server logs after mint.
- **C1 runtime measurement** — per-analysis GPU wall-clock recorded for pricing analytics (Phase 5).

### SDK
- **`PrimateError.rawMessage`** — verbatim server message, for UI display. TS SDK + MCP bumped to 0.1.1 via npm OIDC Trusted Publishing.
- **Python SDK** — PyPI Trusted Publishing workflow added (tag `sdk-py-v*`).

---

<!-- Backfilled 2026-07-27 (PRI-482 r8) from the website changelog when docs collapsed to one clock. -->

## 2026-07-09 — `additive` — SDKs, MCP server, agent artifacts

- Official SDKs: TypeScript (`@primate-intelligence/sdk`) and Python (`primate-intelligence`) — typed resources, registry-driven retries, auto idempotency keys, `createAndWait()`, webhook verification, pagination iterators, browser `connectStream()`.
- MCP server (`@primate-intelligence/mcp`) with seven tools mirroring the spec.
- `llms.txt` / `llms-full.txt` + `/.well-known/llms.txt`; all docs pages served as raw markdown at `/docs/*.md`.
- New documentation site: quickstart, agent quickstart, ten guides, Scalar API reference, error-registry anchors.

## 2026-07-09 — `deprecation` — Legacy `pk_` API key prefix sunset prepared

- Existing `pk_` keys remain valid through the launch migration.
- New keys and rotations create `pv_live_` / `pv_test_` keys only.
- The 12-month `pk_` sunset clock starts at production GA announcement after Matt approves the cutover; the exact date will be published in the versioning guide and dashboard before enforcement.

## 2026-07-08 — `additive` — Billing: credit purchases + auto-refill

- `GET /v1/credit-pricing` (public): price per second, grant sizes, purchase options.
- Dashboard credit purchases, saved payment methods, and threshold-triggered auto-refill.
- `GET /v1/billing/usage-summary` for dashboard usage aggregation.

## 2026-07-07 — `additive` — Webhooks + usage meters

- `webhook_endpoints` resource: CRUD, Standard-Webhooks signing, delivery worker with `1m…24h` retry schedule, deliveries API, redelivery, secret rotation with 24h dual-secret overlap, auto-disable after 72h of failures.
- Event vocabulary: `video.ready`, `video.failed`, `analysis.completed`, `analysis.failed`, `analysis.canceled`, `stream.ended`.
- Per-request `webhook` overrides on `POST /v1/analyses` / `POST /v1/streams` (unsigned).
- `GET /v1/usage` generic meters shape (`credit_seconds` balance + period consumption).

## 2026-07-06 — `additive` — Streams public API

- `streams` resource (§4.8): create → signal (WS) → WebRTC → live per-frame results → end. Queueing with honest `queue_position`/`estimated_start_s`, per-plan session caps, 5s metering ticks, credit reservation reconciled to the second.
- `stream_already_active` (409) error code registered.

## 2026-07-05 — `additive` — Client tokens

- `POST /v1/client_tokens`: ephemeral browser-safe `pvct_` tokens (60–900s TTL, scope subsets, optional video/stream binding). Signaling WS accepts client tokens only — never secret keys.
- `token_expired` (401) error code registered.

## 2026-07-04 — `additive` — Instant sandbox + auth cutoff

- Anonymous `POST /v1/sandbox` issues instant `pv_test_` keys (IP-limited, 7-day expiry, fixture corpus, deterministic results) with a pre-seeded fixture video.
- Metered credit enforcement on the live plane; open signup.
- `sandbox_limit_exceeded` (429) error code registered.

## 2026-07-03 — `additive` — Public core: videos + analyses

- `videos` resource: presigned upload (create → PUT → complete), SSRF-guarded URL ingest, cursor-paginated lists, delete with 409 protection.
- `analyses` resource: free-text prompts or structured queries, `Prefer: wait` sync sugar (cap 120s), live progress + queue position, cancel, deterministic `result` contract (`answer`/`confidence`/`clips`).
- `GET /v1/models`, `GET /v1/errors` (machine-readable registry), idempotency keys (JCS canonical body comparison, 24h replay window).

## 2026-07-02 — `additive` — OpenAPI spec

- `GET /v1/openapi.json` — OpenAPI 3.1, generated from the DTO registry; CI drift + `oasdiff breaking` gates. The spec is the source of truth for docs, SDKs, and this changelog's compatibility promises.


---

<!-- source: https://www.primateintelligence.ai/docs/reference.md -->

# API reference

> Public API v1 for Primate Vision — video understanding for the agentic web. P1 (PRI-430) publishes resource schemas; endpoints ship from P3 onward.

Generated from the OpenAPI 3.1 contract (version `1.0.0-alpha`). The machine-readable spec is the source of truth: [https://api.primateintelligence.ai/v1/openapi.json](https://api.primateintelligence.ai/v1/openapi.json).

- Base URL: `https://api.primateintelligence.ai`
- Auth: `Authorization: Bearer $PRIMATE_API_KEY` (see [Security model](/docs/guides/security-model))
- Errors: every non-2xx response carries the standard envelope — see the [Error registry](/docs/errors)

## Provisioning & keys

### `POST /v1/sandbox`

<a id="post-v1-sandbox"></a>**Provision an instant sandbox (pv_test_) key — no auth, no card**

Anonymous zero-touch provisioning (A1). Issues a fixture-only test-mode key: analyses return deterministic canned results, never touch the GPU, and burn no credits. IP-limited; 7-day expiry. Live keys require signup (billing gate, not an integration gate).

**Responses**

- `201` — Sandbox key issued. The raw key appears only in this response. → [SandboxKey](#schema-sandboxkey)
- `429` — sandbox_limit_exceeded — IP daily cap reached. Retry-After set. → [ErrorEnvelope](#schema-errorenvelope)
- `503` — db_unavailable — transient; retry with backoff. → [ErrorEnvelope](#schema-errorenvelope)

### `POST /v1/keys/upgrade`

<a id="post-v1-keys-upgrade"></a>**Upgrade a test key to a live free-grant key (6,000s credit)**

Stage 1 of the key lifecycle. Call with a `pv_test_` key in Authorization; receive a `pv_live_` key with `tier: free_grant` and a 6,000-second credit grant. No email or card required. **GitHub-gated (PRI-479):** calling without a `github_token` returns `403 github_verification_required` whose `details` carry the OAuth `client_id` and device-flow URLs — complete GitHub's device flow and retry with `{"github_token": "..."}`. One free grant per GitHub account, ever (duplicate → `409 github_account_already_used`). IP caps (3 / IP / 24h) remain as secondary defense.

**Request body** — [UpgradeKeyRequest](#schema-upgradekeyrequest)

**Responses**

- `201` — Live key issued. The raw `api_key` appears only in this response. → [UpgradeKeyResponse](#schema-upgradekeyresponse)
- `401` — github_token_invalid — the supplied github_token failed verification with GitHub. → [ErrorEnvelope](#schema-errorenvelope)
- `403` — test_mode_only — must call with a pv_test_ key. github_verification_required — GitHub gate enabled and no github_token supplied; `details` carry the device-flow bootstrap (client_id, URLs). → [ErrorEnvelope](#schema-errorenvelope)
- `409` — github_account_already_used — this GitHub account has already claimed its free grant. → [ErrorEnvelope](#schema-errorenvelope)
- `429` — upgrade_limit_exceeded — per-IP or global daily cap reached. → [ErrorEnvelope](#schema-errorenvelope)

### `POST /v1/keys/request`

<a id="post-v1-keys-request"></a>**Start a device-code claim flow for a full billed key**

Stage 2 of the key lifecycle. Optional auth (test or live key accepted, or none). Returns a `device_code` and `claim_url`. Direct the user to `claim_url` to sign in; poll `GET /v1/keys/request/{code}` until status changes from `pending`. Key is delivered once in the claim response; subsequent polls return 410.

**Responses**

- `202` — Device code issued. → [DeviceCodeRequest](#schema-devicecoderequest)
- `503` — db_unavailable → [ErrorEnvelope](#schema-errorenvelope)

### `GET /v1/keys/request/{code}`

<a id="get-v1-keys-request-code"></a>**Poll a device-code claim request**

Returns `{status: "pending"}` until the user approves in the browser. After approval, returns `status: "consumed"` (key was delivered in the claim response). Expired or already-consumed codes return 410.

**Parameters**

- `code` (path, string) *(required)* — The `device_code` from POST /v1/keys/request.

**Responses**

- `200` — Current status of the device code. → [DeviceCodeStatus](#schema-devicecodestatus)
- `404` — resource_not_found — code not found. → [ErrorEnvelope](#schema-errorenvelope)
- `410` — device_code_expired — code consumed or expired. Start a new request. → [ErrorEnvelope](#schema-errorenvelope)

## Videos

### `GET /v1/videos`

<a id="get-v1-videos"></a>**List videos (cursor pagination, created_at desc)**

**Parameters**

- `limit` (query, integer)
- `starting_after` (query, string) — Cursor — mutually exclusive with ending_before.
- `ending_before` (query, string)
- `status` (query, string)
- `created_after` (query, string (date-time))
- `created_before` (query, string (date-time))

**Responses**

- `200` — List envelope (§13.4). → [VideoList](#schema-videolist)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `POST /v1/videos`

<a id="post-v1-videos"></a>**Create a video (presigned upload or URL ingest)**

**Parameters**

- `Idempotency-Key` (header, string) — §16.1: first request wins; 24h replay window; same key + different body → 409 idempotency_key_reused.

**Request body** — [CreateVideoRequest](#schema-createvideorequest)

**Responses**

- `201` — Video created. Presigned mode → status awaiting_upload with upload instructions; URL mode → status processing (async fetch). → [Video](#schema-video)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `POST /v1/videos/{id}/complete`

<a id="post-v1-videos-id-complete"></a>**Signal that the presigned S3 PUT finished**

**Parameters**

- `id` (path, string) *(required)*

**Responses**

- `200` — Size verified (±5% of declaration); video is ready. → [Video](#schema-video)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `GET /v1/videos/{id}`

<a id="get-v1-videos-id"></a>**Fetch a video**

**Parameters**

- `id` (path, string) *(required)*

**Responses**

- `200` — The video. → [Video](#schema-video)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `DELETE /v1/videos/{id}`

<a id="delete-v1-videos-id"></a>**Delete a video (409 while analyses are non-terminal on it)**

**Parameters**

- `id` (path, string) *(required)*

**Responses**

- `200` — Deleted. → [DeletedVideo](#schema-deletedvideo)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `GET /v1/videos/{id}/analyses`

<a id="get-v1-videos-id-analyses"></a>**List analyses for a video (result history)**

PRI-496: resource-nested spelling of GET /v1/analyses?video_id={id}. Same list envelope, same filters (status, limit, starting_after), ordered created_at desc. Use it to rebuild a video's full analysis/session history.

**Parameters**

- `id` (path, string) *(required)*
- `limit` (query, integer)
- `starting_after` (query, string) — Cursor — mutually exclusive with ending_before.
- `ending_before` (query, string)
- `status` (query, string)
- `created_after` (query, string (date-time))
- `created_before` (query, string (date-time))

**Responses**

- `200` — List envelope of analyses referencing this video (§13.4). → [AnalysisList](#schema-analysislist)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `GET /v1/demo-videos`

<a id="get-v1-demo-videos"></a>**Public sample videos with precomputed results (no auth)**

PRI-496 GAP-8: onboarding samples — curated videos with precomputed answers for a set of example prompts. Unauthenticated. Use these to see real result shapes before uploading your own video. The documented subset above is the stable contract; extra fields may appear and change without notice.

**Responses**

- `200` — Sample videos. → [DemoVideoList](#schema-demovideolist)

## Analyses

### `GET /v1/analyses`

<a id="get-v1-analyses"></a>**List analyses (filters: status, video_id, model, created_*)**

**Parameters**

- `limit` (query, integer)
- `starting_after` (query, string) — Cursor — mutually exclusive with ending_before.
- `ending_before` (query, string)
- `created_after` (query, string (date-time))
- `created_before` (query, string (date-time))
- `status` (query, string) — Filter by public analysis status (§13.2 vocabulary): queued | preparing | analyzing | rendering | completed | failed | canceled. E.g. status=analyzing returns currently-running analyses. Unknown values → 400 validation_failed.
- `video_id` (query, string)
- `model` (query, string)

**Responses**

- `200` — List envelope (§13.4). → [AnalysisList](#schema-analysislist)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `POST /v1/analyses`

<a id="post-v1-analyses"></a>**Create an analysis (video × prompt × model)**

Async by default: 202 + poll. `Prefer: wait=<s>` (≤120) blocks and returns the terminal resource when it finishes in time. Test-mode keys (pv_test_) return deterministic canned results with no GPU time and no credit burn. Set `validate_only: true` to dry-run a prompt — returns an AnalysisPreview with cost/assessability and creates nothing.

**Parameters**

- `Idempotency-Key` (header, string) — §16.1: first request wins; 24h replay window; same key + different body → 409 idempotency_key_reused.
- `Prefer` (header, string) — wait=<seconds> (cap 120)

**Request body** — [CreateAnalysisRequest](#schema-createanalysisrequest)

**Responses**

- `200` — Terminal resource (Prefer: wait satisfied, or test mode) — OR an AnalysisPreview when validate_only:true was sent. Discriminate on the `object` field. → Analysis | AnalysisPreview
- `202` — Queued. Response echoes the compiled query and honest queue_position. → [Analysis](#schema-analysis)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `402` — insufficient_credits (details.meter = credit_seconds) → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)
- `503` — inference_unavailable / capacity_exhausted (with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `GET /v1/analyses/{id}`

<a id="get-v1-analyses-id"></a>**Fetch an analysis (live progress while running)**

**Parameters**

- `id` (path, string) *(required)*

**Responses**

- `200` — The analysis. progress uses the canonical stage set queued→preparing→analyzing→rendering. → [Analysis](#schema-analysis)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `POST /v1/analyses/{id}/rerun`

<a id="post-v1-analyses-id-rerun"></a>**Free re-run of a failed analysis (never billed)**

PRI-493: when a failed analysis carries rerun_eligible: true (it failed during a platform incident, or was flagged by ops), POST here creates a FRESH analysis — same video, prompt/query, model, and options — at no charge (usage stays null on the re-run). One free re-run per failed analysis. 409 rerun_not_eligible otherwise. Normal capacity gates still apply.

**Parameters**

- `id` (path, string) *(required)*

**Responses**

- `202` — Fresh analysis queued (new id). Never billed. → [Analysis](#schema-analysis)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)
- `503` — inference_unavailable / capacity_exhausted (with Retry-After) — the free re-run is NOT consumed on platform failure. → [ErrorEnvelope](#schema-errorenvelope)

### `POST /v1/analyses/{id}/cancel`

<a id="post-v1-analyses-id-cancel"></a>**Cancel an analysis (best-effort once analyzing)**

**Parameters**

- `id` (path, string) *(required)*

**Responses**

- `200` — Canceled (or best-effort race documented in §14.2). → [Analysis](#schema-analysis)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `POST /v1/analyses/batch`

<a id="post-v1-analyses-batch"></a>**Submit 2–10 prompts against the same video (batch discount)**

Pricing: first prompt billed at full price; each additional prompt billed at 50% (a 50% discount — config-driven; read the live value from GET /v1/credit-pricing batch_discount_pct). Credits are reserved per-prompt at the discounted rate so the ledger shows the discount directly. Returns an `analysis_batch` containing all analysis DTOs plus a `pricing` summary. Each analysis can be polled individually via `GET /v1/analyses/{id}`. Pass `validate_only: true` for a cost-estimate dry-run (HTTP 200, no credits reserved).

**Request body** — [CreateAnalysisBatchRequest](#schema-createanalysisbatchrequest)

**Responses**

- `200` — Dry-run preview (validate_only: true). Prompts parsed; cost estimated; no credits reserved, no jobs created. → [AnalysisBatchPreview](#schema-analysisbatchpreview)
- `202` — Batch queued. All analyses dispatched; poll each `id` individually. → [AnalysisBatch](#schema-analysisbatch)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `402` — insufficient_credits / grant_exhausted → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)
- `503` — capacity_exhausted / inference_unavailable → [ErrorEnvelope](#schema-errorenvelope)

## Streams

### `GET /v1/streams`

<a id="get-v1-streams"></a>**List streams (cursor pagination, created_at desc)**

**Parameters**

- `limit` (query, integer)
- `starting_after` (query, string) — Cursor — mutually exclusive with ending_before.
- `ending_before` (query, string)
- `status` (query, string)
- `created_after` (query, string (date-time))
- `created_before` (query, string (date-time))
- `status` (query, `"queued"` · `"ready"` · `"live"` · `"ended"`)

**Responses**

- `200` — List envelope (§13.4). → [StreamList](#schema-streamlist)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `POST /v1/streams`

<a id="post-v1-streams"></a>**Create a real-time analysis stream**

Returns the signaling WS URL, ephemeral ICE servers, and session limits. Signaling protocol v1: join → ready|queued(position) → offer/answer/ice → live; mid-stream update_prompt {prompt} (client→server; server confirms with prompt_updated), metering (5s ticks), warning, end {reason}. WS auth: ?token= accepts a client token (pvct_) or first-party session JWT — never a secret key. Billed per second of live clock time (join/negotiation free) from the same credit ledger as uploads. options.narrative: true adds (a) a narrative_update object ({t_s, text}) to result-frame detections whenever the engine emits a new sentence — event-driven, absent otherwise — and (b) an additive server→client status event {type: "status", status: "prompt_context"|"combined_prompt"|"recalculating", message?} for ordering/annotating narrative entries (closed vocabulary; non-opted-in streams never receive status events). No extra cost.

**Parameters**

- `Idempotency-Key` (header, string) — §16.1: first request wins; 24h replay window; same key + different body → 409 idempotency_key_reused.

**Request body** — [CreateStreamRequest](#schema-createstreamrequest)

**Responses**

- `201` — Stream created (queued or ready). Response carries signaling.url + ice_servers + limits. → [Stream](#schema-stream)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `402` — insufficient_credits — positive balance required. → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — stream_already_active — one queued|ready|live stream per account (also the un-keyed-retry guard, §31-F2) / idempotency_key_reused. → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)
- `503` — capacity_exhausted (with Retry-After) — slot queue beyond the hard cap. → [ErrorEnvelope](#schema-errorenvelope)

### `GET /v1/streams/{id}`

<a id="get-v1-streams-id"></a>**Fetch a stream (usage + results summary once ended)**

**Parameters**

- `id` (path, string) *(required)*

**Responses**

- `200` — The stream. Terminal streams carry usage {billed_seconds, credit_balance_after} + duration_s + results_summary. → [Stream](#schema-stream)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `GET /v1/streams/{id}/recording`

<a id="get-v1-streams-id-recording"></a>**Fetch the session recording (signed URL, sign-on-read)**

PRI-496: returns a signed playback URL for the server-side session recording. Requires the stream to have been created with recording: true and to have ended with a stored recording. 404 resource_not_found when recording was not enabled; 409 resource_conflict while the stream is still active, when capture failed, or when no recording was stored. URL TTL 1h — re-fetch for a fresh one.

**Parameters**

- `id` (path, string) *(required)*

**Responses**

- `200` — Signed recording URL. → [StreamRecording](#schema-streamrecording)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `POST /v1/streams/{id}/end`

<a id="post-v1-streams-id-end"></a>**End a stream (idempotent)**

Graceful end: settles the credit reservation for the billed live-clock seconds (releases it entirely when the stream never went live). Ending an already-ended stream returns the terminal resource unchanged.

**Parameters**

- `id` (path, string) *(required)*

**Responses**

- `200` — The ended stream with final usage. → [Stream](#schema-stream)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

## Webhook endpoints

### `GET /v1/webhook_endpoints`

<a id="get-v1-webhook-endpoints"></a>**List webhook endpoints (secrets never included)**

**Responses**

- `200` — List envelope. → [WebhookEndpointList](#schema-webhookendpointlist)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `POST /v1/webhook_endpoints`

<a id="post-v1-webhook-endpoints"></a>**Create a webhook endpoint (returns the one-time whsec_ secret)**

Standard Webhooks signing (webhook-id / webhook-timestamp / webhook-signature). The secret appears ONLY in this response. Delivery contract: 10s timeout, retries 1m/5m/30m/2h/8h/24h with jitter, at-least-once, no ordering — dedupe on the evt_ id. Endpoints failing 100% for 72h are auto-disabled with an email notice. At most 10 endpoints per account.

**Parameters**

- `Idempotency-Key` (header, string) — §16.1: first request wins; 24h replay window; same key + different body → 409 idempotency_key_reused.

**Request body** — [CreateWebhookEndpointRequest](#schema-createwebhookendpointrequest)

**Responses**

- `201` — Endpoint created. `secret` is returned exactly once. → [WebhookEndpoint](#schema-webhookendpoint)
- `400` — validation_failed / webhook_endpoint_limit (10 per account). → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `GET /v1/webhook_endpoints/{id}`

<a id="get-v1-webhook-endpoints-id"></a>**Fetch a webhook endpoint (secret never included)**

**Parameters**

- `id` (path, string) *(required)* — Webhook endpoint id (we_…).

**Responses**

- `200` — The endpoint. → [WebhookEndpoint](#schema-webhookendpoint)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `DELETE /v1/webhook_endpoints/{id}`

<a id="delete-v1-webhook-endpoints-id"></a>**Delete a webhook endpoint (effective immediately, §17)**

**Parameters**

- `id` (path, string) *(required)* — Webhook endpoint id (we_…).

**Responses**

- `200` — {id, object, deleted: true}.
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `POST /v1/webhook_endpoints/{id}/rotate_secret`

<a id="post-v1-webhook-endpoints-id-rotate-secret"></a>**Rotate the signing secret (24h dual-secret overlap)**

Returns the NEW secret exactly once. The previous secret keeps signing deliveries for 24h so receivers can migrate without dropped verifications — the signature header carries one signature per active secret.

**Parameters**

- `id` (path, string) *(required)* — Webhook endpoint id (we_…).

**Responses**

- `200` — Endpoint with the new one-time `secret`. → [WebhookEndpoint](#schema-webhookendpoint)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `POST /v1/webhook_endpoints/{id}/enable`

<a id="post-v1-webhook-endpoints-id-enable"></a>**Re-enable a disabled endpoint (clears auto-disable state)**

**Parameters**

- `id` (path, string) *(required)* — Webhook endpoint id (we_…).

**Responses**

- `200` — The re-enabled endpoint. → [WebhookEndpoint](#schema-webhookendpoint)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `GET /v1/webhook_endpoints/{id}/deliveries`

<a id="get-v1-webhook-endpoints-id-deliveries"></a>**List recent deliveries (last 100, with response codes)**

Dead-lettered deliveries stay visible for 30 days. Deliveries carry the evt_ id (stable across retries), attempt count, last response status/error, and next retry time.

**Parameters**

- `id` (path, string) *(required)* — Webhook endpoint id (we_…).

**Responses**

- `200` — List envelope (newest first). → [WebhookDeliveryList](#schema-webhookdeliverylist)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

### `POST /v1/webhook_endpoints/{id}/deliveries/{delivery_id}/redeliver`

<a id="post-v1-webhook-endpoints-id-deliveries-delivery-id-redeliver"></a>**Redeliver a delivery now (works on failed/dead rows)**

**Parameters**

- `id` (path, string) *(required)* — Webhook endpoint id (we_…).
- `delivery_id` (path, string) *(required)* — Delivery id (whd_…).

**Responses**

- `202` — Requeued for immediate delivery. → [WebhookDelivery](#schema-webhookdelivery)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

## Client tokens

### `POST /v1/client_tokens`

<a id="post-v1-client-tokens"></a>**Mint an ephemeral browser-safe client token (pvct_)**

Server-side token exchange (§4.9): authenticate with a secret key (or first-party session), receive a short-lived scoped pvct_ bearer for the browser. The browser then calls Primate directly — your secret key never leaves your server. Tokens are revoked implicitly when the minting key is revoked (≤5s). Rate class: 120/min per key. Client tokens can never mint tokens.

**Request body** — [CreateClientTokenRequest](#schema-createclienttokenrequest)

**Responses**

- `201` — Token minted. The raw token appears only in this response. → [ClientToken](#schema-clienttoken)
- `400` — validation_failed — scopes/ttl_s/binding violations, all listed at once. → [ErrorEnvelope](#schema-errorenvelope)
- `403` — insufficient_scope — requested scopes exceed the minting key’s, or the caller is itself a client token. → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found — video_id binding does not resolve to an owned video. → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded — client_tokens.create is 120/min per key. → [ErrorEnvelope](#schema-errorenvelope)

## Billing

### `GET /v1/credits`

<a id="get-v1-credits"></a>**Credit balance and transaction history**

Agent-facing billing truth (PRI-480 A-5): remaining balance in billable seconds plus a paginated ledger. Check this before submitting work to avoid a 402. Requires the billing:read scope.

**Parameters**

- `limit` (query, integer)
- `before` (query, string)

**Responses**

- `200` — Balance plus one page of ledger entries. → [CreditsResponse](#schema-creditsresponse)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

## Usage

### `GET /v1/usage`

<a id="get-v1-usage"></a>**Usage meters (credit balance, period consumption)**

**Responses**

- `200` — Meters + legacy fields. Same numbers as the dashboard (shared service, reconciles with the credit ledger to the second). → [UsageResponse](#schema-usageresponse)
- `400` — validation_failed / url_forbidden / prompt errors (§15) → [ErrorEnvelope](#schema-errorenvelope)
- `401` — invalid_api_key / key_revoked / key_expired → [ErrorEnvelope](#schema-errorenvelope)
- `404` — resource_not_found (account-scoped; no existence oracle) → [ErrorEnvelope](#schema-errorenvelope)
- `409` — resource_conflict / idempotency_key_reused / idempotency_unavailable → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded / concurrency_limit_exceeded / quota_exceeded (always with Retry-After) → [ErrorEnvelope](#schema-errorenvelope)

## Meta & platform

### `GET /v1/models`

<a id="get-v1-models"></a>**List available models**

**Responses**

- `200` — Model catalog. Requests may say model:"latest"; responses always name the concrete version. → object

### `GET /v1/credit-pricing`

<a id="get-v1-credit-pricing"></a>**Current credit pricing (public, no auth)**

**Responses**

- `200` — Pricing config. Billing unit is source-clock video-seconds; analysis cost = usage.billed_seconds × price_per_second_cents. → [CreditPricing](#schema-creditpricing)

### `POST /v1/parse`

<a id="post-v1-parse"></a>**Compile a free-text prompt into a CompiledQuery (no analysis created)**

Stateless prompt compiler. Auth optional (anonymous allowed); rate-limited to 30 req/min per IP. Never touches the GPU and never burns credits — use POST /v1/analyses with validate_only:true when you also want cost/assessability estimates.

**Request body** — [ParseRequest](#schema-parserequest)

**Responses**

- `200` — The compiled query and which parser produced it. → [ParseResponse](#schema-parseresponse)
- `400` — empty_prompt — prompt missing, not a string, or blank. → [ErrorEnvelope](#schema-errorenvelope)
- `422` — parse_failed — prompt could not be compiled into a query. → [ErrorEnvelope](#schema-errorenvelope)
- `429` — rate_limit_exceeded (30 req/min per IP). → [ErrorEnvelope](#schema-errorenvelope)
- `503` — parse_unavailable — parse service temporarily unavailable. → [ErrorEnvelope](#schema-errorenvelope)

### `GET /v1/errors`

<a id="get-v1-errors"></a>**Machine-readable error-code registry (§15)**

**Responses**

- `200` — Closed registry — codes are append-only; SDK retry policy generates from retryable/idem flags. → object

## Object schemas

Every named object in the contract. Field types link to their schema.

### VideoStatus

<a id="schema-videostatus"></a>Video lifecycle state (§14.1).

Enum: `"awaiting_upload"` · `"processing"` · `"ready"` · `"failed"`

### VideoUpload

<a id="schema-videoupload"></a>Presigned upload instructions. Present only while awaiting_upload. URL TTL 1h.

- `method` — `"PUT"` *(required)*
- `url` — string (uri) *(required)*
- `headers` — object *(required)*
- `expires_at` — string (date-time) *(required)*

### VideoMedia

<a id="schema-videomedia"></a>Signed 1h playback URL for the source video (sign-on-read — re-fetch for a fresh URL). Null while not ready or when no stored source object exists.

- `url` — string (uri) *(required)*
- `expires_at` — string (date-time) *(required)*

### ErrorObject

<a id="schema-errorobject"></a>- `code` — string *(required)*
- `message` — string *(required)*

### Video

<a id="schema-video"></a>A source video (§13.1).

- `id` — string *(required)* — Video resource id (prefixed ULID). Immutable.
- `object` — `"video"` *(required)*
- `livemode` — boolean *(required)* — true when the request used a live key; false for test keys and sandbox. Stripe-style.
- `status` — [VideoStatus](#schema-videostatus) *(required)* — Video lifecycle state (§14.1).
- `filename` — string,null *(required)*
- `size_bytes` — integer,null *(required)*
- `duration_s` — number,null *(required)*
- `width` — integer,null *(required)*
- `height` — integer,null *(required)*
- `fps` — number,null *(required)*
- `content_type` — `"video/mp4"` · `"video/quicktime"` *(required)*
- `source` — `"upload"` · `"url"` *(required)*
- `upload` — [VideoUpload](#schema-videoupload) *(required)* — Presigned upload instructions. Present only while awaiting_upload. URL TTL 1h.
- `media` — [VideoMedia](#schema-videomedia) — Signed 1h playback URL for the source video (sign-on-read — re-fetch for a fresh URL). Null while not ready or when no stored source object exists.
- `error` — [ErrorObject](#schema-errorobject) *(required)*
- `metadata` — object — Free-form key/value metadata. ≤16 keys, key ≤40 chars [a-zA-Z0-9_.-], value ≤500 chars.
- `expires_at` — string,null (date-time) *(required)*
- `created_at` — string (date-time) *(required)*

### AnalysisStatus

<a id="schema-analysisstatus"></a>Analysis lifecycle state (§14.2). Clients MUST tolerate unknown future non-terminal values.

Enum: `"queued"` · `"preparing"` · `"analyzing"` · `"rendering"` · `"completed"` · `"failed"` · `"canceled"`

### Progress

<a id="schema-progress"></a>Non-terminal only. percent is monotonic non-decreasing per analysis.

- `stage` — string *(required)*
- `percent` — number *(required)*

### Clip

<a id="schema-clip"></a>- `start_s` — number *(required)*
- `end_s` — number *(required)*
- `confidence` — number *(required)*
- `terms` — object *(required)*

### Result

<a id="schema-result"></a>Analysis result — the product (§13.3). Reserved (always absent in v1): masks, bounding_boxes, events.

- `answer` — `"yes"` · `"no"` · `"indeterminate"` *(required)*
- `confidence` — number *(required)*
- `clips` — array<[Clip](#schema-clip)> *(required)*
- `term_confidences` — object *(required)*
- `query_type` — `"object"` · `"action"` · `"compound"` · `"attribute"` · `"open_ended"` *(required)*
- `video_duration_s` — number,null *(required)*
- `detected_count` — integer,null
- `indeterminate_reason` — `"low_confidence"` · `"nothing_detected"` · `"unsupported_query_form"` · `"duration_mismatch"`

### Narrative

<a id="schema-narrative"></a>- `status` — `"generating"` · `"ready"` · `"failed"` *(required)*
- `entries` — array<object> *(required)*

### Artifacts

<a id="schema-artifacts"></a>Signed URL TTL 1h; re-fetch GET analysis for a fresh URL.

- `annotated_video_url` — string (uri) *(required)*
- `expires_at` — string (date-time) *(required)*

### AnalysisUsage

<a id="schema-analysisusage"></a>Terminal only. fps-independent clock seconds (pricing doc). credit_balance_after is the balance immediately after THIS analysis settled — an immutable point-in-time snapshot; it does not change as later analyses run. Null when the account row is unavailable (e.g. free-grant exhausted before settlement).

- `billed_seconds` — integer *(required)*
- `credit_balance_after` — integer,null *(required)*

### Analysis

<a id="schema-analysis"></a>A video analysis (§13.2).

- `id` — string *(required)* — Analysis resource id (prefixed ULID).
- `object` — `"analysis"` *(required)*
- `livemode` — boolean *(required)* — true when the request used a live key; false for test keys and sandbox. Stripe-style.
- `origin` — `"api"` · `"console"` · `"system"` *(required)* — `api` — submitted via the public API; `console` — uploaded via the dashboard; `system` — generated internally (e.g. auto-probe, fixture seeding, run_prompt). System analyses are never billed. Agents should normally only process api/console-origin analyses. (Renamed 2026-07-27: former value `user` is now `api`.)
- `status` — [AnalysisStatus](#schema-analysisstatus) *(required)* — Analysis lifecycle state (§14.2). Clients MUST tolerate unknown future non-terminal values.
- `video_id` — string *(required)* — Video resource id (prefixed ULID). Immutable.
- `prompt` — string,null *(required)*
- `query` — object *(required)* — CompiledQuery (schema at /v1/schemas/compiled-query.json). Always populated in responses.
- `parse_mode` — `"llm"` · `"heuristic"` · `"client"` *(required)*
- `model` — string *(required)*
- `options` — object *(required)*
- `progress` — [Progress](#schema-progress) *(required)* — Non-terminal only. percent is monotonic non-decreasing per analysis.
- `queue_position` — integer,null *(required)*
- `result` — [Result](#schema-result) *(required)* — Analysis result — the product (§13.3). Reserved (always absent in v1): masks, bounding_boxes, events.
- `narrative` — [Narrative](#schema-narrative) *(required)*
- `artifacts` — [Artifacts](#schema-artifacts) *(required)* — Signed URL TTL 1h; re-fetch GET analysis for a fresh URL.
- `error` — [ErrorObject](#schema-errorobject) *(required)*
- `usage` — [AnalysisUsage](#schema-analysisusage) *(required)* — Terminal only. fps-independent clock seconds (pricing doc). credit_balance_after is the balance immediately after THIS analysis settled — an immutable point-in-time snapshot; it does not change as later analyses run. Null when the account row is unavailable (e.g. free-grant exhausted before settlement).
- `rerun_eligible` — boolean — PRI-493: present on failed analyses — true when this analysis qualifies for ONE free re-run (it failed during a platform incident, or was flagged by ops). POST /v1/analyses/{id}/rerun creates a fresh analysis at no charge. Absent on non-failed analyses.
- `metadata` — object — Free-form key/value metadata. ≤16 keys, key ≤40 chars [a-zA-Z0-9_.-], value ≤500 chars.
- `created_at` — string (date-time) *(required)*
- `started_at` — string,null (date-time) *(required)*
- `completed_at` — string,null (date-time) *(required)*

### StreamStatus

<a id="schema-streamstatus"></a>Stream lifecycle state (§4.8): queued → ready → live → ended. Terminal immutable.

Enum: `"queued"` · `"ready"` · `"live"` · `"ended"`

### StreamEndReason

<a id="schema-streamendreason"></a>Why the stream ended. completed — normal end after live. canceled — ended before going live (client action). insufficient_credits — balance exhausted. error — server-side failure. timeout — max_session_s cutoff. ice_failed — WebRTC ICE never connected (see failure_diagnostic). media_timeout — connected but no media/results within the first-frame window. A stream that never went live is never `completed`.

Enum: `"completed"` · `"canceled"` · `"insufficient_credits"` · `"error"` · `"timeout"` · `"ice_failed"` · `"media_timeout"`

### StreamSignaling

<a id="schema-streamsignaling"></a>Present while the stream is joinable; null once ended.

- `url` — string *(required)* — Versioned signaling WS. Auth: ?token= accepts a client token (pvct_) or first-party Clerk JWT — NEVER a secret key.
- `expires_at` — string (date-time) *(required)*

### StreamLimits

<a id="schema-streamlimits"></a>- `max_session_s` — integer *(required)* — Plan entitlement (max_session_s) — hard session cutoff.
- `warn_at_remaining_s` — integer *(required)* — The `warning {remaining_s}` threshold surfaced mid-stream.

### StreamRecordingState

<a id="schema-streamrecordingstate"></a>PRI-496: recording state — null unless the stream opted in with recording: true at create.

- `status` — `"recording"` · `"available"` · `"failed"` · `"none"` *(required)*

### StreamUsage

<a id="schema-streamusage"></a>Terminal only. Same ledger as uploads — one balance.

- `billed_seconds` — integer *(required)* — LIVE-clock seconds (join/negotiation free). fps-independent by construction.
- `credit_balance_after` — integer,null *(required)*

### Stream

<a id="schema-stream"></a>A real-time analysis stream (§4.8).

- `id` — string *(required)* — Stream resource id (prefixed ULID). P6 §4.8.
- `object` — `"stream"` *(required)*
- `status` — [StreamStatus](#schema-streamstatus) *(required)* — Stream lifecycle state (§4.8): queued → ready → live → ended. Terminal immutable.
- `end_reason` — [StreamEndReason](#schema-streamendreason) *(required)* — Why the stream ended. completed — normal end after live. canceled — ended before going live (client action). insufficient_credits — balance exhausted. error — server-side failure. timeout — max_session_s cutoff. ice_failed — WebRTC ICE never connected (see failure_diagnostic). media_timeout — connected but no media/results within the first-frame window. A stream that never went live is never `completed`.
- `prompt` — string,null *(required)*
- `query` — object,null *(required)*
- `model` — string *(required)*
- `queue_position` — integer,null *(required)*
- `estimated_start_s` — integer,null *(required)*
- `signaling` — [StreamSignaling](#schema-streamsignaling) *(required)* — Present while the stream is joinable; null once ended.
- `ice_servers` — array,null *(required)*
- `limits` — [StreamLimits](#schema-streamlimits) *(required)*
- `duration_s` — integer,null *(required)* — Live-clock duration. Terminal only.
- `results_summary` — object,null *(required)* — Terminal summary: {result_frames, frames (deprecated alias of result_frames — removed ~2026-08-28), last_detections}. result_frames counts result EVENTS emitted (results are sampled, not per-source-frame). Detection rows use the same enums/casing as the file-API result contract (answer: yes|no|indeterminate, lowercase; prompt echoed exactly as submitted).
- `failure_diagnostic` — object,null *(required)* — Present when end_reason is ice_failed (and some media_timeout cases): ICE candidate summary from the server side — {local_candidates: string[], hint}. Use it to distinguish "server advertised only private/host candidates" from "client could not reach relay".
- `recording` — [StreamRecordingState](#schema-streamrecordingstate) — PRI-496: recording state — null unless the stream opted in with recording: true at create.
- `usage` — [StreamUsage](#schema-streamusage) *(required)* — Terminal only. Same ledger as uploads — one balance.
- `metadata` — object — Free-form key/value metadata. ≤16 keys, key ≤40 chars [a-zA-Z0-9_.-], value ≤500 chars.
- `created_at` — string (date-time) *(required)*
- `live_started_at` — string,null (date-time) *(required)*
- `ended_at` — string,null (date-time) *(required)*

### WebhookEventType

<a id="schema-webhookeventtype"></a>Closed v1 event vocabulary (§4.5). Payload = the full resource DTO.

Enum: `"video.ready"` · `"video.failed"` · `"analysis.completed"` · `"analysis.failed"` · `"analysis.canceled"` · `"stream.ended"`

### WebhookEndpoint

<a id="schema-webhookendpoint"></a>A customer webhook endpoint (§4.5). Standard Webhooks signing.

- `id` — string *(required)* — Webhook endpoint id (prefixed ULID).
- `object` — `"webhook_endpoint"` *(required)*
- `url` — string (uri) *(required)*
- `events` — array<[WebhookEventType](#schema-webhookeventtype)> *(required)*
- `description` — string,null *(required)*
- `status` — `"enabled"` · `"disabled"` *(required)*
- `disabled_reason` — `"auto_failure"` · `"user"` *(required)* — auto_failure = 72h of 100% delivery failure (§16.3). Re-enable via POST …/enable.
- `secret` — string — Signing secret — returned EXACTLY ONCE (create / rotate_secret), never readable again.
- `last_success_at` — string,null (date-time) *(required)*
- `created_at` — string (date-time) *(required)*

### WebhookDelivery

<a id="schema-webhookdelivery"></a>One event→endpoint delivery record (§16.3). Dead-lettered rows visible 30 days.

- `id` — string *(required)* — Webhook delivery id (prefixed ULID).
- `object` — `"webhook_delivery"` *(required)*
- `event_id` — string *(required)* — Stable across retries — consumers dedupe on this.
- `event_type` — [WebhookEventType](#schema-webhookeventtype) *(required)* — Closed v1 event vocabulary (§4.5). Payload = the full resource DTO.
- `status` — `"pending"` · `"delivering"` · `"delivered"` · `"failed"` · `"dead"` *(required)*
- `attempt_count` — integer *(required)*
- `last_response_status` — integer,null *(required)*
- `last_error` — string,null *(required)*
- `next_attempt_at` — string,null (date-time) *(required)* — Null once delivered or dead-lettered.
- `last_attempt_at` — string,null (date-time) *(required)*
- `delivered_at` — string,null (date-time) *(required)*
- `created_at` — string (date-time) *(required)*

### Meter

<a id="schema-meter"></a>One named usage meter (§4.6). The list is data-driven — pricing changes add/change rows, never the schema.

- `meter` — string *(required)*
- `unit` — string *(required)*
- `balance` — number — Balance-shaped meters (e.g. credit_seconds).
- `limit` — number,null — null = unlimited/PAYG.
- `used` — number — Period-shaped meters.
- `resets_at` — string (date-time) — Omitted for non-period meters.

### CreditPricing

<a id="schema-creditpricing"></a>Current pricing configuration. Data-driven — a price change is a config update, not a schema change. Batch discount included (PRI-508) — read it from here instead of hardcoding.

- `price_per_second_cents` — number *(required)* — Price per billed video-second, in cents. Cost of an analysis = usage.billed_seconds × this value.
- `signup_grant_seconds` — integer *(required)* — Free credit-seconds granted on signup / free-grant upgrade.
- `card_grant_seconds` — integer *(required)* — Bonus credit-seconds granted when a card is added.
- `auto_refill_threshold_seconds` — integer *(required)* — Balance below which auto-refill (if enabled) triggers.
- `allowed_purchase_cents` — array<integer> *(required)* — Preset purchase amounts, in cents.
- `custom_purchase_enabled` — boolean *(required)*
- `min_purchase_cents` — integer *(required)*
- `batch_discount_pct` — integer *(required)* — Batch analyses (POST /v1/analyses/batch): percent discount applied to each prompt after the first. The first prompt is always full price.
- `batch_min_prompts` — integer *(required)* — Minimum prompts accepted per batch request.
- `batch_max_prompts` — integer *(required)* — Maximum prompts accepted per batch request.

### ErrorEnvelope

<a id="schema-errorenvelope"></a>- `error` — object *(required)*

### CreateVideoRequest

<a id="schema-createvideorequest"></a>Exactly one mode: presigned ({filename?, content_type, size_bytes?}) or ingest ({url}). Unknown fields are rejected.

- `filename` — string
- `content_type` — `"video/mp4"` · `"video/quicktime"` — Required in presigned mode.
- `size_bytes` — integer
- `url` — string (uri) — URL-ingest mode: https-only, SSRF-guarded (§18.2). Mutually exclusive with filename/size_bytes.
- `metadata` — object — Free-form key/value metadata. ≤16 keys, key ≤40 chars [a-zA-Z0-9_.-], value ≤500 chars.

### VideoList

<a id="schema-videolist"></a>- `object` — `"list"` *(required)*
- `data` — array<[Video](#schema-video)> *(required)*
- `has_more` — boolean *(required)*

### AnalysisList

<a id="schema-analysislist"></a>- `object` — `"list"` *(required)*
- `data` — array<[Analysis](#schema-analysis)> *(required)*
- `has_more` — boolean *(required)*

### DeletedVideo

<a id="schema-deletedvideo"></a>- `id` — string *(required)* — Video resource id (prefixed ULID). Immutable.
- `object` — `"video"` *(required)*
- `deleted` — `true` *(required)*

### AnalysisPreview

<a id="schema-analysispreview"></a>- `object` — `"analysis_preview"` *(required)*
- `query` — object *(required)* — The compiled query the analysis would run.
- `parse_mode` — `"llm"` · `"heuristic"` · `"client"` *(required)* — How the prompt was compiled. "client" means you supplied query directly.
- `assessable` — boolean *(required)* — False for open_ended queries — the model cannot score them, so detections will be empty.
- `video_duration_s` — number,null *(required)* — Null when duration is not yet known (probe pending).
- `estimated_seconds` — integer,null *(required)* — Billable seconds this analysis would consume. Null when duration is unknown.
- `estimated_cost_usd` — number,null *(required)* — Estimated cost at current pricing. Null when duration is unknown.

### CreateAnalysisRequest

<a id="schema-createanalysisrequest"></a>- `video_id` — string *(required)* — Video resource id (prefixed ULID). Immutable.
- `prompt` — string — Free text — compiled server-side. Exactly one of prompt/query.
- `query` — object — Structured CompiledQuery JSON (schema at /v1/schemas/compiled-query.json).
- `model` — string — Defaults to latest; response always echoes the concrete version.
- `options` — object — narrative: true — generate a narrative (timestamped event sentences) for the analysis. Populates the narrative object on the completed analysis; generation runs asynchronously after completion (poll the GET — status generating → ready|failed). Included in the analysis price.
- `metadata` — object — Free-form key/value metadata. ≤16 keys, key ≤40 chars [a-zA-Z0-9_.-], value ≤500 chars.
- `webhook` — object — Per-request webhook override (delivery ships P7). Write-only — never echoed.
- `validate_only` — boolean — Dry run (PRI-480). Compiles the prompt, reports whether the query is assessable, and estimates duration/cost — WITHOUT creating an analysis, touching the GPU, or burning credits. Returns an AnalysisPreview (object: "analysis_preview") with 200, not an Analysis. Use it to check a prompt before paying for it.

### DemoVideoResult

<a id="schema-demovideoresult"></a>Precomputed result for one prompt. Documented subset — additional fields (result_json, clips) exist but are undocumented and may change.

- `answer` — string,null *(required)*
- `confidence` — number,null *(required)*
- `query_type` — string,null *(required)*
- `video_url` — string,null *(required)* — Playable annotated result video (CDN).
- `computed_at` — string *(required)*

### DemoVideo

<a id="schema-demovideo"></a>- `id` — string *(required)*
- `display_name` — string *(required)*
- `duration_seconds` — number,null *(required)*
- `thumbnail_url` — string,null *(required)*
- `results_by_prompt` — object *(required)* — Map of prompt text → precomputed result.

### DemoVideoList

<a id="schema-demovideolist"></a>- `videos` — array<[DemoVideo](#schema-demovideo)> *(required)*

### AnalysisBatchPreviewPrompt

<a id="schema-analysisbatchpreviewprompt"></a>- `index` — integer *(required)* — Zero-based prompt index.
- `query` — object *(required)* — Compiled query structure.
- `parse_mode` — `"heuristic"` · `"llm"` · `"client"` *(required)*
- `assessable` — boolean *(required)* — True when the query type can produce a yes/no/count answer.
- `estimated_seconds` — integer,null *(required)* — Estimated credit-seconds for this prompt. Null when video has no known duration.
- `estimated_cost_usd` — number,null *(required)* — Estimated cost in USD. Null when video has no known duration.
- `discount_pct` — `0` | `50` *(required)* — 0 for the first (full-price) prompt; the configured batch discount (GET /v1/credit-pricing batch_discount_pct) for additional prompts.

### AnalysisBatchPricing

<a id="schema-analysisbatchpricing"></a>- `full_price_prompts` — integer *(required)*
- `discounted_prompts` — integer *(required)*
- `discount_pct` — integer *(required)* — Config-driven batch discount percent — the same value GET /v1/credit-pricing serves as batch_discount_pct.
- `estimated_total_seconds` — integer,null *(required)* — Estimated total credit-seconds for all prompts combined (full + discounted). Null when the video has no known duration.
- `estimated_total_cost_usd` — number,null *(required)* — Estimated total cost in USD. Null when duration unknown.

### AnalysisBatchPreview

<a id="schema-analysisbatchpreview"></a>- `object` — `"analysis_batch_preview"` *(required)*
- `video_id` — string *(required)* — Video resource id (prefixed ULID). Immutable.
- `prompts` — array<[AnalysisBatchPreviewPrompt](#schema-analysisbatchpreviewprompt)> *(required)*
- `pricing` — [AnalysisBatchPricing](#schema-analysisbatchpricing) *(required)*

### AnalysisBatch

<a id="schema-analysisbatch"></a>- `object` — `"analysis_batch"` *(required)*
- `id` — string *(required)* — Stable batch identifier.
- `video_id` — string *(required)* — Video resource id (prefixed ULID). Immutable.
- `analyses` — array<[Analysis](#schema-analysis)> *(required)*
- `pricing` — [AnalysisBatchPricing](#schema-analysisbatchpricing) *(required)*

### CreateAnalysisBatchRequest

<a id="schema-createanalysisbatchrequest"></a>- `video_id` — string *(required)* — Video resource id (prefixed ULID). Immutable.
- `prompts` — array<string> *(required)* — 2–10 prompts to run against the video. First prompt is full price; each additional is billed at 50% (default — the live discount is served by GET /v1/credit-pricing as batch_discount_pct).
- `model` — string
- `options` — object
- `metadata` — object — Free-form key/value metadata. ≤16 keys, key ≤40 chars [a-zA-Z0-9_.-], value ≤500 chars.
- `validate_only` — boolean — Dry-run mode. When true, parses all prompts and returns an analysis_batch_preview (HTTP 200) with per-prompt cost estimates. No credits are reserved and no jobs are created.

### Model

<a id="schema-model"></a>- `id` — string *(required)*
- `object` — `"model"` *(required)*
- `status` — `"stable"` · `"preview"` · `"deprecated"` *(required)*
- `default` — boolean *(required)*
- `capabilities` — object *(required)*
- `sunset_at` — string,null *(required)*
- `created_at` — string *(required)*

### ParseResponse

<a id="schema-parseresponse"></a>- `compiled_query` — object *(required)* — CompiledQuery — canonical JSON Schema served at /v1/schemas/compiled-query.json. Same structure accepted by POST /v1/analyses `query`.
- `parse_mode` — `"haiku"` · `"heuristic"` *(required)* — How the prompt was compiled: "haiku" = LLM parser; "heuristic" = deterministic fallback (used when the LLM is unavailable).

### ParseRequest

<a id="schema-parserequest"></a>- `prompt` — string *(required)* — Free-text video question. Must be a non-empty string.

### ErrorRegistryEntry

<a id="schema-errorregistryentry"></a>- `code` — string *(required)*
- `kind` — `"http"` · `"resource"` *(required)*
- `status` — integer,null *(required)*
- `retryable` — boolean *(required)*
- `idem` — boolean *(required)*
- `message` — string *(required)*
- `docs_url` — string (uri) *(required)*

### ClientToken

<a id="schema-clienttoken"></a>- `object` — `"client_token"` *(required)*
- `id` — string *(required)*
- `token` — string *(required)* — 256-bit CSPRNG bearer, SHA-256 at rest — shown exactly once. Browser-safe: accepted on product-plane endpoints (CORS *).
- `scopes` — array<string> *(required)*
- `expires_at` — string *(required)*
- `video_id` — string — Video resource id (prefixed ULID). Immutable.
- `stream_id` — string

### CreateClientTokenRequest

<a id="schema-createclienttokenrequest"></a>- `scopes` — array<`"videos:write"` · `"analyses:read"` · `"analyses:write"` · `"streams:create"` · `"streams:signal"`> *(required)* — Non-empty subset of the §4.9 scope set. Must also be a subset of the minting key’s scopes.
- `ttl_s` — integer — Token lifetime in seconds (60–900, default 900). Tokens are never refreshable — mint a new one.
- `video_id` — string — Optional resource binding: narrows the token to this one video. Mutually exclusive with stream_id.
- `stream_id` — string — Optional resource binding to one stream (WS acceptance ships P6).

### CreateStreamRequest

<a id="schema-createstreamrequest"></a>Requires positive credit balance. One queued|ready|live stream per account by default (409 stream_already_active otherwise).

- `prompt` — string — Free text — compiled server-side. Exactly one of prompt/query.
- `query` — object — Structured CompiledQuery JSON.
- `model` — string
- `options` — object — narrative: true — opt into live narrative. Result-frame detections then carry a narrative_update object ({t_s, text}) whenever the engine produces a new sentence (event-driven, not per frame; absent on frames without one). Included in the per-second price.
- `recording` — boolean — PRI-496: true — opt into retrieval of the server-side session recording. The ended stream then carries recording: {status} and GET /v1/streams/{id}/recording returns a signed 1h playback URL (raw H.264 elementary stream; lossless remux: ffmpeg -i in.h264 -c copy out.mp4). Default false.
- `metadata` — object — Free-form key/value metadata. ≤16 keys, key ≤40 chars [a-zA-Z0-9_.-], value ≤500 chars.
- `webhook` — object — Per-request webhook override (delivery ships P7). Write-only.

### StreamList

<a id="schema-streamlist"></a>- `object` — `"list"` *(required)*
- `data` — array<[Stream](#schema-stream)> *(required)*
- `has_more` — boolean *(required)*

### StreamRecording

<a id="schema-streamrecording"></a>- `object` — `"stream_recording"` *(required)*
- `stream_id` — string *(required)* — Stream resource id (prefixed ULID). P6 §4.8.
- `url` — string (uri) *(required)* — Signed playback URL — fresh on every GET, 1h TTL. Re-fetch for a new one.
- `expires_at` — string *(required)* — RFC 3339 expiry of the signed URL.
- `content_type` — `"video/h264"` *(required)*
- `container` — `"h264-annex-b"` *(required)* — Raw H.264 Annex-B elementary stream. Lossless remux to MP4: ffmpeg -i in.h264 -c copy out.mp4.

### SandboxKey

<a id="schema-sandboxkey"></a>- `object` — `"sandbox_key"` *(required)*
- `api_key` — string *(required)* — Shown exactly once — store it now. Fixture-only test mode: no GPU, no credits.
- `mode` — `"test"` *(required)*
- `expires_at` — string *(required)* — Keys expire 7 days after issuance.
- `fixture_video_id` — string,null *(required)* — Pre-seeded ready video for immediate POST /v1/analyses.
- `fixture_prompt` — string *(required)*
- `expected_answer` — string *(required)* — Deterministic canned answer for the fixture prompt — assert this in CI.
- `docs_url` — string *(required)*
- `note` — string *(required)*

### UpgradeKeyResponse

<a id="schema-upgradekeyresponse"></a>- `object` — `"api_key"` *(required)*
- `livemode` — `true` *(required)*
- `api_key` — string *(required)* — The raw `pv_live_` key — store securely, shown only once.
- `tier` — `"free_grant"` *(required)*
- `expires_at` — string (date-time) *(required)*
- `grant_seconds` — integer *(required)*
- `note` — string
- `claim_flow` — object

### UpgradeKeyRequest

<a id="schema-upgradekeyrequest"></a>- `github_token` — string — PRI-479: GitHub OAuth access token from the device flow. Required when the GitHub gate is enabled (calling without one returns 403 github_verification_required with device-flow bootstrap details). Verified against GitHub and discarded — never stored. One free grant per GitHub account, ever.

### DeviceCodeRequest

<a id="schema-devicecoderequest"></a>- `object` — `"device_code_request"` *(required)*
- `device_code` — string *(required)* — Opaque code; use as path param when polling.
- `claim_url` — string (uri) *(required)* — Open in browser for the user to approve.
- `expires_at` — string (date-time) *(required)*
- `poll_interval` — integer *(required)* — Seconds between polls.
- `instructions` — string

### DeviceCodeStatus

<a id="schema-devicecodestatus"></a>- `object` — `"device_code_request"` *(required)*
- `status` — `"pending"` · `"consumed"` *(required)*

### CreateWebhookEndpointRequest

<a id="schema-createwebhookendpointrequest"></a>- `url` — string (uri) *(required)* — HTTPS delivery URL.
- `events` — array<[WebhookEventType](#schema-webhookeventtype)> *(required)* — Subscribed event types — a non-empty subset of the closed v1 vocabulary.
- `description` — string

### WebhookEndpointList

<a id="schema-webhookendpointlist"></a>- `object` — `"list"` *(required)*
- `data` — array<[WebhookEndpoint](#schema-webhookendpoint)> *(required)*
- `has_more` — boolean *(required)*

### WebhookDeliveryList

<a id="schema-webhookdeliverylist"></a>- `object` — `"list"` *(required)*
- `data` — array<[WebhookDelivery](#schema-webhookdelivery)> *(required)*
- `has_more` — boolean *(required)*

### UsageResponse

<a id="schema-usageresponse"></a>Usage meters + legacy dashboard fields (the flat fields are deprecated in favor of meters[]; they remain during the web-client migration).

- `meters` — array<[Meter](#schema-meter)> *(required)* — Data-driven meter list (§4.6 invariant #1). At launch: credit_seconds (balance) + seconds_processed.period (used/resets_at). Pricing changes add/change rows — never the schema. quota_exceeded errors carry details.meter naming which meter tripped.

### CreditTransaction

<a id="schema-credittransaction"></a>- `id` — string *(required)*
- `kind` — string *(required)* — e.g. signup_grant, analysis_debit, stream_debit, purchase.
- `seconds_delta` — number *(required)* — Signed: negative debits, positive credits.
- `balance_after_seconds` — number *(required)*
- `source_type` — string,null *(required)*
- `source_id` — string,null *(required)*
- `created_at` — string *(required)*

### CreditsResponse

<a id="schema-creditsresponse"></a>- `object` — `"credits"` *(required)*
- `balance_seconds` — number *(required)* — Seconds of analysis remaining. The authoritative balance.
- `grant_seconds` — number,null *(required)* — Original signup grant, when one exists.
- `used_seconds` — number,null *(required)* — Consumed against the signup grant (grant minus balance).
- `transactions` — array<[CreditTransaction](#schema-credittransaction)> *(required)*
- `has_more` — boolean *(required)*



---

<!-- source: https://primateintelligence.ai/blog/darwin-video-jepa-benchmarks.md -->

---
title: "Darwin: Video JEPA model that outperforms SOTA models while running on edge CPU"
slug: "darwin-video-jepa-benchmarks"
author: "Mehdi Nikkhah"
date: "2026-07-29"
readTime: "5 min read"
tags: ["Research","Benchmarks","JEPA"]
excerpt: "Language models had their GPT moment. Vision, especially video, hasn’t yet!"
canonical: https://primateintelligence.ai/blog/darwin-video-jepa-benchmarks
---

Language models had their GPT moment. Vision, especially video, hasn’t yet!

Despite remarkable progress over the past few years, we still don't have a foundation model for the physical world that has become the default starting point in the way GPT models transformed language. Video remains expensive to train on, difficult to represent efficiently, and surprisingly hard to generalize across tasks. This is the problem many researchers are trying to solve through **world models**.

Different groups have taken different approaches. Generative Video World Models learn by predicting future frames or reconstructing videos. Contrastive methods learn representations by distinguishing positive and negative examples. Both have produced impressive results, but each comes with fundamental compromises in scalability, efficiency, and the quality of the representations they learn.

More recently, **Joint Embedding Predictive Architectures (JEPA)** introduced a different idea. Instead of reconstructing pixels or comparing every possible pair of examples, JEPA predicts high-level representations of unseen regions from visible context. The objective is to model meaning rather than appearance—to learn the underlying structure of the world instead of memorizing visual details. We think this is the direction to bet on, based on our 10+ years of researching and building real-time video models for computer vision.

Over the past several months, we've been building **Darwin**, our implementation of a video JEPA model. At first glance, the obvious question is:

**Why build another JEPA?**

Because we believe world models need to become dramatically more accessible.

Today, training state-of-the-art video models often requires enormous datasets, massive GPU clusters, and budgets that place them beyond the reach of nearly every startup and research lab, let alone customer deployments on the edge hardware that needs to run them. That reality slows progress by limiting meaningful experimentation to only a handful of organizations.

We're a small team, who have previously built the world’s best edge computer vision models, at [6D.ai](http://6D.ai) (acquired by Niantic Spatial, now mapping the world from within Pokemon Go). Rather than assuming world models require virtually unlimited compute, we asked a different question:

**How far can careful engineering, better training methodology, and better data take us?**

The answer surprised even us. Using techniques we developed in the past, combined with the latest cutting edge research, we trained Darwin on a cluster of just **8 H100 GPUs over three weeks**, using roughly **100× less data** than comparable efforts such as VJEPA 2/VJEPA 2.1. For context, recent funding has gone into developing Generative World Models that require 100x more data and compute even compared to VJEPA2.1. Despite that dramatically smaller training budget, Darwin matches, or exceeds, existing VJEPA models of comparable size across standard downstream benchmarks.

**Video understanding (linear probe on frozen backbone):**

- Something-Something V2: **~73%**
- Kinetics-700: **~78%**

**Image understanding:**

- ImageNet-1K: **~88%**

Figure 1 compares Darwin against V-JEPA 2 across all three benchmarks.

![Bar chart comparing Darwin and VJEPA2 probe accuracy on Something-Something-v2, Kinetics-700, and ImageNet1k](/images/darwin-benchmarks/figure-1-probe-accuracy.png)

*Figure 1. Probe accuracy with a frozen backbone on benchmark datasets*

Benchmark accuracy by itself is important but ultimately an academic pursuit. Customers need to be able to run these models on their edge devices for any real business use case. If world models are going to power robots, autonomous systems, and edge devices, they can't require datacenter-scale hardware just to run inference. Efficiency matters just as much as capability. So we designed Darwin to be lightweight enough to run **in real time on a single commodity CPU**.

Figures 2 and 3 compare Darwin's throughput against V-JEPA 2 on both CPU and GPU, demonstrating that strong world representations don't have to come at the cost of enormous inference requirements.

![Bar chart comparing Darwin and VJEPA2 throughput in frames per second on an Intel Xeon CPU](/images/darwin-benchmarks/figure-2-throughput-intel-xeon.png)

*Figure 2. Throughput on an Intel Xeon Processor*

![Bar chart comparing Darwin and VJEPA2 throughput in frames per second on an Nvidia H200 GPU](/images/darwin-benchmarks/figure-3-throughput-nvidia-h200.png)

*Figure 3. Throughput on Nvidia H200 GPU*

We also wanted the learned representations themselves to be immediately useful. Most self-supervised vision models still require task-specific classification heads before they can solve downstream problems. Instead, we aligned Darwin's embedding space with natural language using text alignment. The result is a model that can be prompted directly with text to retrieve relevant visual concepts from its learned representations, without training separate classifiers for every task.

## Darwin: the first commercially available Video JEPA model

Customers can use Darwin today with open vocabulary prompts on a live video stream in real-time.

For us, this is just the beginning. We don't believe the future of physical-world intelligence should belong exclusively to organizations with tens of thousands of GPUs. Our experience in building models that run on tens for millions of phones to understand messy physical reality taught us that a different type of research approach is needed. Applying principles of customer focus, deep engineering, and innovative training methodologies, alongside pushing the limits of published academic research, is what gives the breakthroughs that enable models to be useful on billions of devices.

At **Primate Intelligence**, our mission is to make world models efficient enough to run anywhere, adaptable enough to solve real problems, and accessible enough that innovation isn't limited by compute budgets.


---

<!-- source: https://primateintelligence.ai/compare/aws-rekognition.md -->

---
title: "Primate Vision vs. AWS Rekognition + Bedrock Nova (2026): The Door AWS Closed"
slug: "aws-rekognition"
author: "Matt Miesnieks"
date: "2026-07-31"
readTime: "7 min read"
tags: ["Comparisons","Cloud Vision"]
excerpt: "AWS closed Rekognition streaming to new customers on April 30, 2026 — its own migration advice is to snapshot frames into the image API. Here's what remains, what Bedrock Nova actually offers, and the honest head-to-head against a real-time verdict API."
canonical: https://primateintelligence.ai/compare/aws-rekognition
---

The most important fact about AWS Rekognition in 2026 comes from AWS itself: **"Streaming Video and Bulk Image Analysis features will no longer be available to new customers, effective April 30, 2026."** ([availability changes](https://docs.aws.amazon.com/rekognition/latest/dg/rekognition-availability-changes.html))

Read that again. The largest cloud on earth decided real-time video analysis wasn't a business it wanted new customers in. A new customer cannot buy it from Rekognition at any price — AWS's own published migration advice is to snapshot frames and call the *image* API via Lambda. What remains is a solid fixed-taxonomy stored-video API (labels, faces, moderation, bounding boxes) and, on the generative side, Bedrock Nova — an explicitly **offline** video Q&A model.

Primate Vision is the managed real-time lane AWS exited: live WebRTC ingest at native frame rates, plain-English questions, and a **deterministic verdict** — yes / no / indeterminate — with calibrated confidence, timestamps, and an annotated evidence video.

*This is part of our [full 2026 video-AI landscape comparison](/compare/video-ai-landscape-2026).*

---

## What happened to Rekognition streaming

Rekognition's real-time story was Streaming Video Events: Kinesis Video Streams in, SNS alerts out. Even at its best it was narrow. The streaming detector covers "people, packages and pets" only, and processes a **maximum of 120 seconds of video per motion event** ([streaming docs](https://docs.aws.amazon.com/rekognition/latest/dg/streaming-video-detect-labels.html)). Doorbell alerts, not continuous monitoring. No actions, no activities, no questions.

As of April 30, 2026, that lane is closed to new customers. Grandfathered accounts (active in the last 12 months) keep access; everyone else is pointed at frame-snapshotting into the image API. AWS's designated forward path for open-vocabulary video understanding is Bedrock Nova — which we'll take seriously below, because on its own terms it's very cheap.

## The head-to-head

| Capability | Primate Vision | AWS Rekognition | Bedrock Nova |
|---|---|---|---|
| Live stream input | ✅ managed WebRTC, native frame rate | ⚠️ closed to new customers; grandfathered: people/pets/packages, ≤120s/event | ❌ file/S3 only |
| Open-vocab NL prompting | ✅ core product | ❌ fixed detector taxonomy, no prompts anywhere | ✅ free-form prompts |
| Verdict contract | ✅ deterministic yes/no/indeterminate + calibrated 0–1 confidence | ❌ exhaustive label lists; you post-process | ⚠️ generated text/JSON, uncalibrated, stochastic |
| Fidelity | ✅ native frame rate, full resolution | ✅ duration-billed, AWS samples internally | ⚠️ all video resized "with distortion" to 672×672; 1fps only ≤16 min (960-frame budget — a 1-hour video drops to 0.14fps) ([Nova docs](https://docs.aws.amazon.com/nova/latest/userguide/modalities-video.html)) |
| Evidence artifacts | ✅ annotated overlay evidence video + timestamped clips | ⚠️ one first-detection frame JPEG (streaming); JSON only (stored) | ❌ text only |
| Bounding-box JSON | ❌ overlay video only | ✅ frame-accurate, normalized, GA — genuinely excellent | ⚠️ limited spatial reasoning (self-declared) |
| Audio | ❌ visual only | ⚠️ OCR yes; transcription is a separate service | ❌ "Audio tracks in videos are not processed" |
| Ingest | ✅ direct upload or URL ingest, ≤2 GiB | ⚠️ S3 only, no upload/URL endpoint | ⚠️ base64 ≤25 MB or S3 ≤1 GB |
| Async model | ✅ webhooks (Standard Webhooks, retries, redelivery) + `Prefer: wait` | ✅ SNS/SQS/Lambda — the gold standard | ✅ Bedrock async |
| Onboarding | ✅ sandbox key in one POST, no email, no card | ⚠️ SigV4 signing; no sandbox, no curl-first path | ⚠️ Bedrock + IAM setup |
| Published latency | ✅ 45ms p50 / 316ms p95, 11.8 fps, 6.6s setup | ❌ none | ❌ none (minutes-scale per request) |

## Verdicts vs. label lists

Ask Rekognition about a warehouse camera and it returns everything it saw: labels, confidences, bounding boxes, timestamps. Thorough, machine-consumable, and entirely question-free. "Is the forklift reversing without a spotter?" is unanswerable — you build the reasoning layer yourself, on top of label streams, and you maintain it forever.

Nova can take the question. But it answers with a sampled LLM generation over 672×672 distorted frames at 1fps — no calibrated confidence, no indeterminate semantics, and re-asking means re-billing the full token count.

Primate Vision answers the question as a measurement: a deterministic verdict from a closed vocabulary, computed by similarity scoring against the video — not sampled from a language model — with a calibrated 0–1 confidence, a pinnable model version (`darwin-1.3`), timestamped evidence segments, and an annotated overlay video you can attach to an incident report.

## Pricing: three very different meters

**Rekognition (stored)** bills per minute *per API*: Label Detection $0.10/min, Content Moderation $0.10/min, Shot Detection $0.05/min — run three detectors on the same footage and pay three times. That's **$6.00/camera-hour for one detector**. Streaming (grandfathered only) is $0.00817/min ≈ **$0.49/camera-hour** — plus a separate Kinesis Video Streams bill. ([Rekognition pricing](https://aws.amazon.com/rekognition/pricing/))

**Nova** is the cheapest open-vocabulary option in the entire landscape for offline clips: roughly **$0.06/camera-hour** on Nova Lite (estimate; input tokens only, third-party-corroborated rates — AWS's own pricing page is JS-gated and its Price List API shows an ambiguous $0.03/M meter). The caveats the raw number hides: four separate requests per hour (16-minute cap at 1fps), minutes of latency per request, no audio, downscale distortion, one question per pass, no verdicts, no evidence.

**Primate Vision has two lanes:**

1. **Metered — $0.01 per second of source video** ($0.60/min), flat, fps-independent. Queued time free; failed jobs free. A 30-second clip → verdict + confidence + timestamps + evidence video = $0.30. Re-query the same upload for 30 days; batch 2–10 questions against one video at 50% off each after the first.
2. **[Enterprise — contact us](https://primateintelligence.ai/pricing#enterprise)** for 24/7 continuous monitoring and camera fleets: dedicated capacity or on-site deployment at a small fraction of the metered rate, under highly discounted enterprise plans.

Normalized honestly: our metered lane is $36/camera-hour† — versus $6/hr for one Rekognition detector, $0.49/hr for grandfathered streaming, ~$0.06/hr for offline Nova. Up to ~70× cheaper on raw numbers. If a fixed detector or an offline batch answer genuinely solves your problem, take the cheaper row. But none of those rows include the thing our meter prices: a live, deterministic, evidenced answer to *your* question. On the metered lane you buy verdicts, not camera-hours. When the workload really is camera-hours, that's the enterprise lane.

† *Metered rate normalized for comparison. Primate does not sell 24/7 continuous monitoring at the metered rate — continuous and fleet workloads use enterprise plans; [contact us](https://primateintelligence.ai/pricing#enterprise).*

## Choose AWS instead when…

There are real answers here, not straw men:

- **You need face identity at scale.** Face search against collections of up to 20M vectors, person pathing, celebrity recognition — Rekognition is purpose-built for it and we have nothing in the category.
- **You need bounding-box JSON for downstream geometry** — zones, dwell logic, heatmaps — on stored footage. Rekognition's spatial JSON is frame-accurate and GA; Primate Vision returns overlay video, not coordinates.
- **You need content moderation or media-catalog labeling** of large stored libraries at the lowest per-minute price, with the SNS/SQS/Lambda event architecture your team already runs.
- **You're grandfathered into Streaming Video Events** for doorbell-style person/pet/package alerts and it does the job — keep it running.
- **You can wait, chunk, and post-process**: Nova is the cheapest open-vocabulary clip Q&A anywhere. If offline is fine and evidence isn't required, it's hard to beat on cost.
- **AWS compliance surface is the requirement**: HIPAA eligibility, SOC scope, IAM/CloudTrail. We have published availability targets and Enterprise SLAs, but no SOC 2 certification yet.

## Choose Primate Vision when…

- **You're a new customer who needs real-time video analysis.** AWS does not sell it to you anymore. Frame-snapshotting into an image API is a workaround, not a product.
- **Your question isn't in the taxonomy.** Open-vocabulary, live or stored: "did anyone climb the fence," not "person detected."
- **The answer needs to be actionable and auditable**: deterministic verdict, calibrated confidence, timestamped clips, watchable evidence — one clean JSON contract instead of label streams plus a reasoning layer you build.
- **Fidelity matters**: native frame rate and full resolution, versus 1fps sampling and 672×672 distortion.
- **You want to evaluate in five minutes**: sandbox key in one POST, curl-first docs — versus SigV4 signing and IAM setup before your first call.

## Try it

**[Try it for free](https://primateintelligence.ai/signup)** — real processing, no card required to start.

**Your AI agent can do it for you — right from Claude.** Primate Vision ships an MCP server, llms.txt, markdown docs twins, and a sandbox key available in a single POST — your agent can get a key, upload a clip, and read back a verdict without a human touching a dashboard.

For 24/7 monitoring and camera fleets: [see our enterprise plans](https://primateintelligence.ai/pricing#enterprise).

---

*Method note: every AWS claim traces to AWS docs, the Rekognition/Bedrock Price List APIs, and pricing pages accessed 2026-07-31. Nova per-token rates are third-party-corroborated where AWS's page is JS-gated; estimates are labeled.*


---

<!-- source: https://primateintelligence.ai/compare/azure-video-indexer.md -->

---
title: "Primate Vision vs. Azure Video Indexer (2026): An Insight Catalog vs. a Live Answer"
slug: "azure-video-indexer"
author: "Matt Miesnieks"
date: "2026-07-31"
readTime: "7 min read"
tags: ["Comparisons","Cloud Vision"]
excerpt: "Azure Video Indexer is a genuinely good archive-enrichment product — thirty insight types, far cheaper per hour than us. But it's a catalog, not an answer: no live-stream API, no way to ask your own question of the pixels, no verdict. Here's the honest head-to-head."
canonical: https://primateintelligence.ai/compare/azure-video-indexer
---

Microsoft walked away from live video analysis. Twice. That history is the most useful thing I can tell you about Azure Video Indexer, so let's start there.

**Azure AI Vision Spatial Analysis** — people counting and zone events on live camera feeds — was retired March 30, 2025 ([Microsoft Q&A](https://learn.microsoft.com/en-us/answers/questions/2151867/)). **Live Video Analytics** was absorbed into Video Indexer's branding years ago; the pricing page still says VI extracts insights "whether stored or streaming," but there is no live-stream API anywhere in the 2026 product — the actual API is upload → index → poll (or webhook). Even the edge option, "Video Indexer enabled by Arc," indexes *files*, limited to Basic presets, and bills per minute at cloud rates despite running on your hardware ([FAQ](https://learn.microsoft.com/en-us/azure/azure-video-indexer/faq)).

What remains is a genuinely good product for a different job: upload a file, get ~30 insight types — transcript, translation, OCR, faces, celebrities, topics, scenes, moderation — on one shared timeline, inside the Azure compliance umbrella. For cataloging stored media at scale it's also far cheaper per hour than we are, and I'll print those numbers below. But it's a catalog, not an answer. Primate Vision watches video **live, at native frame rates**, takes a plain-English question, and returns a deterministic verdict — yes / no / indeterminate — with calibrated confidence and evidence you can watch.

*This is part of our [full 2026 video-AI landscape comparison](/compare/video-ai-landscape-2026).*

---

## A catalog vs. an answer

Azure Video Indexer answers: *"What's in this file?"* Upload (or point it at a URL), wait for indexing, and get back an exhaustive insights JSON: who spoke, what they said in which language, what text appeared on screen, which celebrities showed up, what topics and scenes the video contains. Everything lands on a shared timeline you can search, embed in a player widget, or export. For media asset management, accessibility captioning, and compliance review of stored libraries, that's a real product with a decade of Microsoft engineering behind it. ([VI pricing/product page](https://azure.microsoft.com/en-us/pricing/details/video-indexer/))

Primate Vision answers: *"Is X happening — and can you prove it?"* Point it at a live stream (managed WebRTC) or a file (upload or URL ingest, up to 2 GiB), ask in plain English, and get a deterministic verdict (`yes` / `no` / `indeterminate`), a 0–1 confidence score, timestamped evidence segments, and an annotated overlay evidence video — at 45ms p50 per-frame inference, published on a public [/performance](https://www.primateintelligence.ai/performance) page.

The difference isn't polish; it's the shape of the output. VI hands you thirty insight types and leaves the deciding to you. Primate Vision hands you the decision.

## The head-to-head

| Capability | Primate Vision | Azure Video Indexer |
|---|---|---|
| Live stream input | ✅ managed WebRTC, native frame rate | ❌ upload-and-index only; "stored or streaming" is legacy marketing ([pricing page](https://azure.microsoft.com/en-us/pricing/details/video-indexer/) + [FAQ](https://learn.microsoft.com/en-us/azure/azure-video-indexer/faq)) |
| Published latency | ✅ 45ms p50 / 316ms p95, 11.8 fps sustained | ❌ batch/async, no indexing-latency SLO published |
| Ask your own question | ✅ open-vocab plain-English prompts, live + stored | ⚠️ fixed insight models; LLM features run on extracted insight *text*, not pixels ([release notes](https://learn.microsoft.com/en-us/azure/azure-video-indexer/release-notes)) |
| Verdict contract | ✅ deterministic yes/no/indeterminate + calibrated 0–1 confidence | ❌ exhaustive insights JSON; no question→verdict model ([output schema](https://learn.microsoft.com/en-us/azure/azure-video-indexer/video-indexer-output-json-v2)) |
| Timestamped segments | ✅ | ✅ every insight carries time ranges on a shared timeline; strong |
| Evidence artifacts | ✅ annotated overlay evidence video | ⚠️ keyframes + player widgets + face redaction; no detection-overlay output |
| Transcript / OCR / audio | ❌ visual only | ✅ core strength: multi-language transcription, translation, speaker indexing, sentiment, OCR |
| Face identity / redaction | ❌ | ✅ celebrity recognition, face search, $0.01/min redaction |
| Webhooks | ✅ Standard Webhooks signing, retries, redelivery + `Prefer: wait` | ✅ `callbackUrl` on upload ([API portal](https://api-portal.videoindexer.ai)) |
| Edge / sovereignty | ❌ cloud only | ✅ Arc edge extension — Basic presets only, billed per minute at cloud rates ([FAQ](https://learn.microsoft.com/en-us/azure/azure-video-indexer/faq)) |
| Agent accessibility | ✅ llms.txt, markdown docs twins, MCP server, sandbox key in one POST | ⚠️ two-layer ARM-token auth; no llms.txt/MCP found |

## LLM features that never see the video

VI's 2024–2026 roadmap looks generative — GPT-4o, then GPT-5.x textual summarization, a "Prompt Content" API for RAG. Read the docs closely, though: these features operate on the *already-extracted insight text*, not on the pixels ([release notes](https://learn.microsoft.com/en-us/azure/azure-video-indexer/release-notes), Nov 2024 / Mar 2026 entries). The fixed insight models run first; the LLM summarizes what they caught.

The consequence is structural: **a question the fixed taxonomy didn't capture cannot be answered afterward.** If the object detector didn't log it, no amount of GPT-5.2 summarization will find it. And the generative features require you to bring your own Azure OpenAI resource, billed separately.

Primate Vision inverts this. Your question drives the vision model directly: "Is anyone climbing the fence?" isn't looked up in a pre-built catalog — it's asked of the frames, live or stored. You can change the question mid-stream with a single WebSocket message, or re-run new questions against an uploaded video for 30 days without re-uploading.

## Verdicts: a decision vs. a data dump

VI's output is honest about what it is: an insights JSON with per-instance confidences, designed for downstream code and human reviewers. To turn "did the forklift enter the loading zone?" into an alert, you write the logic yourself — pick the right insight types, define thresholds, handle the cases the taxonomy doesn't cover. That reasoning layer is yours to build and maintain forever.

Primate Vision's verdict is a different kind of object: a **deterministic answer** from a closed vocabulary — `yes` / `no` / `indeterminate` — never free-form generation, with a calibrated 0–1 confidence score, model version pinning (`darwin-1.3`), and a fully deterministic test mode for CI. And every yes ships with proof: timestamped clips plus an annotated overlay evidence video a human can watch and forward. VI gives you keyframes and a player widget; it does not render what it detected onto the video.

## Pricing: two very different meters

**Video Indexer bills per input minute, and the meters stack.** Audio analysis and video analysis are separate meters, each in three presets; index both and you pay both. Current-generation East US rates from the [Azure Retail Prices API](https://prices.azure.com/api/retail/prices?$filter=contains(productName,%20%27Video%20Indexer%27)) (the public pricing page renders "$-" until you interact with it): Basic Video $0.045/min, Standard Video $0.09/min, Advanced Video $0.15/min; audio adds $0.0126–$0.04/min; face redaction $0.01/min. Normalized: **$2.70/camera-hour (Basic Video) up to $11.40/camera-hour (Advanced Video + Audio)**. Gotchas: re-indexing re-bills, legacy pre-2023 accounts sit on pricier meters, and Arc edge indexing bills per minute even on your own hardware. There's a generous trial: 10 free hours on the website, 40 via the API.

**Primate Vision has two lanes**, stated plainly:

1. **Metered — $0.01 per second of source video** ($0.60/min), flat and fps-independent. Queued time free; failed jobs free. A 30-second clip → verdict + confidence + timestamps + evidence video = $0.30.
2. **[Enterprise — contact us](https://primateintelligence.ai/pricing#enterprise)** for 24/7 continuous monitoring and camera fleets: dedicated capacity or on-site deployment at a small fraction of the metered rate, under highly discounted enterprise plans.

The honest normalization: VI works out to $2.70–$11.40 per camera-hour versus $36/camera-hour† at Primate's metered rate — **3–13× cheaper per raw hour**, and if bulk archive enrichment is your workload, that gap is real and you should weigh it. But the meters buy different things. VI's dollar buys an insight catalog that still needs customer-built decision logic — and it answers no live question at any price, because there's no live path to buy. Primate's dollar buys an answered question with evidence. You don't buy camera-hours on our metered lane. You buy verdicts.

† *Metered rate normalized for comparison. Primate does not sell 24/7 continuous monitoring at the metered rate — continuous and fleet workloads use enterprise plans; [contact us](https://primateintelligence.ai/pricing#enterprise).*

## Choose Azure Video Indexer instead when…

We'd rather you pick the right tool than the wrong one of ours:

- **You're building a searchable media archive.** Broadcaster libraries, e-learning catalogs, corporate comms — transcript + faces + topics + scenes across thousands of hours, with an embeddable player and insight widgets. This is VI's home turf; Primate Vision has no stored-asset search at all.
- **You need speech, translation, or on-screen text.** Multi-language transcription, speaker indexing, sentiment, OCR — accessibility and captioning pipelines at $0.0126–$0.04/min audio rates. Primate Vision is visual-only: no transcription, no OCR, no audio path.
- **You need face identity or redaction.** Celebrity recognition, face search within your account, $0.01/min redaction before disclosure or publication — a whole category we don't touch.
- **You're Azure-committed with sovereignty or Gov-cloud requirements.** VI ships in US Gov regions and offers the Arc edge extension for restricted environments, all under Microsoft's compliance umbrella (SOC, ISO, HIPAA per the Trust Center). Primate Vision is single-region (us-west-2), no SOC 2 yet.
- **You're enriching huge stored libraries on a budget.** $2.70–$11.40/camera-hour for thirty insight types is strong value if cataloging — not answering — is the job.

## Choose Primate Vision when…

- **The camera is live.** Microsoft retired its live-camera analysis (Spatial Analysis, March 2025) and VI has no streaming API. Primate Vision is managed WebRTC at native frame rates, with mid-stream prompt changes over one WebSocket message.
- **Your question isn't in the taxonomy.** VI detects what its fixed insight models detect; a question they didn't capture can't be answered afterward, even by the GPT-5.x summarizers. Primate Vision takes any plain-English question and asks it of the pixels directly.
- **You need a decision, not a data dump.** A deterministic yes/no/indeterminate with calibrated confidence feeds an alert, a workflow, or a compliance record without customer-side interpretation logic.
- **You need proof.** An annotated overlay evidence video plus timestamped clips — versus keyframes and a player widget.
- **Latency is a requirement, not a hope.** We publish 45ms p50 / 316ms p95 / 11.8 fps / 6.6s session setup. VI publishes feature lists, not speed — no turnaround numbers anywhere.
- **You want to start in one POST, not one portal.** A sandbox key via a single API call, llms.txt, markdown docs twins, and an MCP server — versus VI's two-layer ARM-token auth and account provisioning before your first request.

## Try it

**[Try it for free](https://primateintelligence.ai/signup)** — real processing, no card required to start.

**Your AI agent can do it for you — right from Claude.** Primate Vision ships an MCP server, llms.txt, markdown docs twins, and a sandbox key available in a single POST — your agent can get a key, upload a clip, and read back a verdict without a human touching a dashboard.

For 24/7 monitoring and camera fleets: [see our enterprise plans](https://primateintelligence.ai/pricing#enterprise).

---

*Method note: every Azure Video Indexer claim traces to Microsoft's live docs, the Azure Retail Prices API, and release notes accessed 2026-07-31. Estimates are labeled.*


---

<!-- source: https://primateintelligence.ai/compare/best-video-understanding-apis-2026.md -->

---
title: "The Best Video Understanding APIs in 2026 (an Honest Roundup)"
slug: "best-video-understanding-apis-2026"
author: "Matt Miesnieks"
date: "2026-07-31"
readTime: "17 min read"
tags: ["Comparisons","Market Analysis"]
excerpt: "Six video understanding APIs — Google Video Intelligence, Twelve Labs, AWS Rekognition, Gemini, Mixpeek, and ours — scored on the six dimensions buyers actually ask about: accuracy approach, hallucination risk, pricing, streaming, free tier, and agent access. With receipts, and with our bias disclosed in the first sentence."
canonical: https://primateintelligence.ai/compare/best-video-understanding-apis-2026
---

I make one of the products in this roundup. That's the first thing you should know, because every "best video AI APIs" listicle on the internet is written by a vendor, and most of them hope you won't notice.

So here's the deal I'm offering instead. Every factual claim below traces to a vendor's own docs or pricing page, accessed July 31, 2026, linked inline. Every estimate is labeled as an estimate. We score ourselves on the same six dimensions as everyone else, and where a competitor beats us — and several do, on real dimensions — I print it. The interesting thing about honest vendor roundups is that they're the ones that actually get cited: Eden AI built an entire business on straight-faced rankings of other people's AI APIs, and Mixpeek — one of the six products below — publishes a video-API roundup in which it ranks *itself* eighth of eight. That's the bar. Honesty is the whole strategy.

One more reason this post exists: buyers don't read ten pricing pages anymore. They ask an LLM. And the answers the LLMs are serving today are stitched together from stale blog posts and marketing copy. This is the document I'd want my own agent to find.

## What we scored, and why these six dimensions

When someone asks "which video understanding API should I use," the follow-up questions are almost always the same six:

1. **Accuracy approach** — what's actually doing the "understanding"? A fixed classifier, an embedding index, a generative LLM, or a perception model? This determines everything downstream.
2. **Hallucination risk** — will it tell you something happened that didn't? For anything wired to an alert, a compliance log, or a robot, this is the question.
3. **Pricing** — the real billing model, with the gotchas.
4. **Streaming / real-time** — can it watch a live camera, or only files you upload?
5. **Free tier** — how far you get before money changes hands, and whether a card is required.
6. **Agent access** — can an AI agent discover, key, and use the API without a human clicking through a console? In 2026 this stopped being a nice-to-have.

And the six products: **Google Video Intelligence**, **Twelve Labs**, **AWS Rekognition**, **Gemini**, **Mixpeek**, and **Primate Vision** (ours).

## The scorecard

| | Accuracy approach | Hallucination risk | Streaming / live input | Free tier | Agent access | Headline price |
|---|---|---|---|---|---|---|
| **Google Video Intelligence** | Fixed-taxonomy classifiers (pre-LLM) | ✅ Low — but it can't answer questions at all | ⚠️ Beta since 2019; DIY C++ proxy for live protocols | ✅ 1,000 min/feature/month | ❌ OAuth-heavy, no MCP, no llms.txt | $0.10/min per feature, stacking |
| **Twelve Labs** | Embedding search (Marengo 3.0) + generative video-to-text (Pegasus 1.5) | ⚠️ Generative answers, uncalibrated | ❌ None — upload and index only | ✅ 600 video minutes, no card | ✅ llms.txt, docs MCP server, Claude Code plugin | $0.042/min indexing + stacking fees |
| **AWS Rekognition** | Fixed detectors (labels, faces, text) | ✅ Low — but objects and labels only | ⚠️ Closed to new customers Apr 2026; people/pets/packages only | ⚠️ Legacy 60 min/mo; new accounts get $200 credits | ⚠️ SigV4 signing; no Rekognition MCP found | $0.10/min per API, stacking |
| **Gemini** | Frontier VLM sampling frames at 1fps | ❌ Stochastic LLM; sampling params deprecated on 3.x | ⚠️ Live input capped at ≤1fps JPEG stills; 2-min A/V sessions | ✅ Generous free tier (trains on your data) | ✅ Instant key, .md docs, best hyperscaler onboarding | ~$0.005–$0.023/min at 1fps (tokens, input-only, estimate) |
| **Mixpeek** | Embedding retrieval over pre-extracted segments — no QA endpoint | — Doesn't answer questions; search results, not claims | ❌ None — batch pipeline over object storage | ⚠️ No free tier ($25/mo floor); free no-signup sample sandbox | ✅ Best-in-class: 4 hosted MCP servers + llms.txt | $0.05/min processed + $25/mo floor + storage rent |
| **Primate Vision** | JEPA perception model → closed-vocabulary verdict + calibrated 0–1 confidence | ✅ Designed out — yes/no/indeterminate, never free-form generation | ✅ Managed WebRTC, 45ms p50 per frame | ✅ 6,000s signup grant, no card; instant sandbox key for agents | ✅ MCP, llms.txt, `POST /v1/sandbox` | $0.01/sec metered; enterprise plans for 24/7 |

Six products, but really four different *kinds* of product wearing the same category label. Two are classifier services from the pre-LLM era (Google VI, Rekognition). One is an archive search engine (Twelve Labs). One is a general-purpose LLM with a video mouth (Gemini). One is retrieval infrastructure (Mixpeek). And one is a perception API that answers questions with verdicts (ours). The rest of this post takes them one at a time.

---

## Google Video Intelligence

**What it is.** Google's 2017-era video annotation API: you pick features from a fixed menu — labels, shots, objects, faces, text, explicit content — and get timestamped JSON annotations back. There is no prompt parameter anywhere in the API. You cannot ask it a question; you can only ask it to run its detectors.

**What it's built for.** Media-library metadata at GCP scale: catalog indexing, content moderation, broadcast compliance. It does that job with a formal [99.9% SLA](https://cloud.google.com/video-intelligence/sla) and mature bounding-box JSON — two things plenty of newer products (ours included) don't offer.

**The streaming story, told straight.** Google VI's streaming API deserves an honest paragraph, because it's the closest thing to a managed streaming action-recognition service any hyperscaler sells: gRPC in, annotations out at roughly one-second granularity ([streaming docs](https://docs.cloud.google.com/video-intelligence/docs/streaming/action-recognition)). We're not going to pretend Google can't stream. But the qualifiers are heavy: the streaming surface has been **Beta since 2019** and is excluded from the SLA; action recognition requires you to **train your own AutoML model first**; and live protocols (RTSP/HLS/RTMP) require compiling and operating Google's AIStreamer C++/GStreamer proxy yourself ([live-streaming docs](https://docs.cloud.google.com/video-intelligence/docs/streaming/live-streaming)). And the product around it is frozen: the [release notes](https://docs.cloud.google.com/video-intelligence/docs/release-notes) have no entries after November 1, 2021. The [pricing page](https://cloud.google.com/products/video-intelligence/pricing) still lists Celebrity Recognition — a feature Google [shut down in September 2025](https://docs.cloud.google.com/video-intelligence/docs/deprecations).

**Pricing.** Per feature, per minute, stacking: label detection is $0.10/min stored, $0.12/min streaming, and each additional feature re-meters the same footage. Two gotchas: partial minutes round *up* per request (1,000 five-second clips bill as 1,000 minutes, not 84), and the first 1,000 minutes per feature per month are free — genuinely generous for evaluation and hobby workloads.

**Performance.** No published latency numbers for stored or streaming analysis — an SLA but zero benchmarks.

**Choose Google VI when** you need commodity annotations — labels, OCR, explicit-content, shot detection — over stored media on GCP, with an SLA, at $0.10/min or free under 1,000 min/month. **Choose us when** the thing you need is an answer to a question, which is precisely what a fixed feature menu cannot give you.

---

## Twelve Labs

**What it is.** The best-funded pure-play in video understanding — a [$100M Series B closed July 1, 2026](https://www.globenewswire.com/news-release/2026/07/01/3320545/0/en/) (NEA and NAVER Ventures co-led, Amazon participating), built around two models: Marengo 3.0 for multimodal embeddings and Pegasus 1.5 for video-to-text. The architecture is index-then-search: upload videos, index them, then query the corpus in natural language.

**What it's built for.** Finding moments in large stored archives. Media libraries, sports footage, ad inventory. At that job it's the category leader, and I mean that without a wink: cross-modal search over visual, speech, and on-screen text, in 36 languages, with the kind of vector-DB partner ecosystem (Pinecone, Weaviate, Milvus, Databricks…) that signals a real platform.

**The line to be clear about.** There is no live input path. Not WebRTC, not RTSP, not gRPC, not WebSockets — nothing in the [upload methods docs](https://docs.twelvelabs.io/v1.3/docs/concepts/upload-methods.md) ingests a stream. "Real-time" in the Twelve Labs story means token-by-token text delivery on generated answers, plus a partner integration that does the streaming for them. It's not that Twelve Labs is a worse monitoring product than ours — it's that it isn't a monitoring product at all. Batch and indexed, by architecture.

On answers: Pegasus generates text, and a [JSON-schema mode](https://docs.twelvelabs.io/v1.3/docs/guides/analyze-videos/structured-responses.md) will happily emit `{"answer": "yes"}` — but it's a generative approximation with no calibration behind it, and the docs offer temperature tuning rather than a determinism guarantee. Ask the same question twice, you may get different answers.

**Pricing.** From [their pricing page](https://www.twelvelabs.io/pricing): indexing $0.042/min (one time), Pegasus analysis $0.0292/min plus output tokens, search $4 per 1,000 queries — and a recurring infrastructure fee of $0.0015 per indexed minute *per month*. That last one compounds quietly: a 10,000-hour archive costs $900/month just to stay searchable, before anyone runs a query. Their own FAQ notes a 60-minute video segmented with 4 segment definitions bills as 240 minutes. The free plan is genuinely useful — 600 video minutes, no credit card.

**Performance.** No published latency numbers anywhere in the public docs — no indexing throughput, no query p95, nothing.

**Agent access** is excellent: llms.txt, every docs page in markdown, a docs MCP server, and a Claude Code plugin. Credit where due — alongside Mixpeek, this is the best agent surface in the roundup.

**Choose Twelve Labs when** your videos are already recorded and your problem is *finding moments across thousands of hours*. We don't do search at all — no embeddings, no corpus, nothing. **Choose us when** your problem is *knowing whether something is happening* — live, or in a clip, with an answer you can act on and evidence you can watch.

---

## AWS Rekognition

**What it is.** AWS's managed CV service: pre-built detectors (labels, faces, celebrities, text, moderation, person pathing) over video sitting in S3, returning JSON with millisecond timestamps, confidences, and frame-accurate bounding boxes. No prompting anywhere; the taxonomy is the taxonomy.

**What it's built for.** AWS-native pipelines: face search against collections of up to 20 million vectors, content moderation on stored libraries, media labeling — wired into the most mature async job architecture in the market (SNS/SQS/Lambda, idempotency tokens).

**The streaming story, told straight.** Rekognition's streaming tier never did activity recognition — the docs are explicit that streaming label detection covers ["people, packages and pets"](https://docs.aws.amazon.com/rekognition/latest/dg/streaming-video-detect-labels.html), processing at most 120 seconds of video per motion event. Activity and action-style labels are stored-video only. And now even that narrow lane is closing: [Streaming Video Events closed to new customers effective April 30, 2026](https://docs.aws.amazon.com/rekognition/latest/dg/rekognition-availability-changes.html). A new customer cannot buy real-time video analysis from Rekognition at all; AWS's own published migration path is to snapshot frames and call the *image* API via Lambda. AWS's designated forward path for open-vocabulary video Q&A is Bedrock Nova — which resizes every video to 672×672 "with distortion," samples 1fps only up to 16 minutes, processes no audio, and takes [no live input](https://docs.aws.amazon.com/nova/latest/userguide/modalities-video.html).

**Pricing.** Stored video is billed per minute *per API* — label detection $0.10/min, and running moderation on the same footage doubles the meter. Streaming (grandfathered accounts only) is [$0.00817/min](https://aws.amazon.com/rekognition/pricing/), with Kinesis Video Streams billed separately. The legacy free tier was 60 minutes of video analysis per month for 12 months; accounts created after July 15, 2025 get up to $200 in credits instead.

**Performance.** No published latency figures for job turnaround or streaming alerts.

**Agent access** is the weak spot: SigV4 request signing makes curl-first onboarding painful, and we found no Rekognition-specific MCP server or llms.txt.

**Choose Rekognition when** you need face identity at scale, bounding-box JSON on stored footage, or content moderation inside an AWS-native stack — or you're already grandfathered into streaming person/pet/package alerts. **Choose us when** the question doesn't map onto a fixed detector, or the camera is live and you weren't an AWS streaming customer before April 2026 — because then the Rekognition door is closed regardless.

---

## Gemini

**What it is.** Google's frontier multimodal model family, where video is one input among many. For stored-file Q&A it's genuinely strong — arguably the strongest general-purpose video reasoner you can rent: it reads signage, understands context, answers arbitrary follow-ups with world knowledge no perception model can match.

**The mechanism to understand.** Gemini doesn't watch video; it samples it. Frames are extracted at [1fps by default](https://ai.google.dev/gemini-api/docs/video-understanding), tokenized at 258 tokens per frame (~300 tokens per second of video with audio), and fed to an LLM. Google's own docs warn that "fast action sequences might lose detail due to the 1 FPS sampling rate" and recommend *slowing down the clip* as the workaround. The live path is tighter still: the Live API's video input is spec-capped at ["images (JPEG <= 1FPS)"](https://ai.google.dev/gemini-api/docs/live-api), audio+video sessions are [limited to 2 minutes without context compression](https://ai.google.dev/gemini-api/docs/live-api/session-management), and the meter [re-bills the entire accumulated session context on every conversational turn](https://ai.google.dev/gemini-api/docs/live-api/best-practices) — Google's docs say it plainly: "As a session lengthens, the cost per turn increases."

**Hallucination risk** is the structural one: the output is a sample from a language model, and as of [July 21, 2026](https://ai.google.dev/gemini-api/docs/changelog) Google deprecated `temperature`/`top_p`/`top_k` on 3.x models — you can no longer even pin temperature to zero. There is no calibrated confidence and no indeterminate state. For a chat experience that's fine. For an alerting pipeline it means building your own verification layer.

**Pricing.** Pure token metering ([pricing page](https://ai.google.dev/gemini-api/docs/pricing)): Gemini 3.6 Flash at $1.50/M input tokens, 3.5 Flash-Lite at $0.30/M. At the 1fps default that works out to roughly $0.005–$0.023 per video-minute, input-only (estimate — output and thinking tokens extra). Cheap. But note what happens at fidelity parity: at 12fps, Gemini's input tokens alone reach $0.28–$0.37/min — approaching half of our all-in $0.60/min — with no verdict layer, no calibration, and no evidence artifacts on top. The free tier via AI Studio is the most generous in the roundup, with one catch stated in Google's own terms: free-tier content is used to improve their products.

**Performance.** No published latency numbers for video understanding or the Live API. **Agent access** is excellent — instant API key, every docs page available as markdown.

**Choose Gemini when** you want conversational understanding of stored video at very low cost, or a voice-and-vision assistant experience, and prose answers are the product. **Choose us when** the answer feeds a system rather than a person. It's not that Gemini is a worse model — it's that a stochastic paragraph and a calibrated verdict are different products, and wiring the first to an alarm is how you get 3 a.m. false pages.

---

## Mixpeek

**What it is — categorized honestly.** Mixpeek is not a video-understanding API, and including it here without saying so would be exactly the dishonesty this post exists to avoid. It's **multimodal indexing and retrieval infrastructure** — in their own words, a ["multimodal data warehouse"](https://mixpeek.com/about): point it at video, audio, or documents in your own object storage and it extracts scenes, faces, OCR, transcripts, and embeddings, then serves hybrid semantic search over the results. Video is one of five modalities. There is no question-answering endpoint, no verdict primitive, no live input of any kind — their own extractor docs list "real-time live streams" as a when-*not*-to-use.

They know this about themselves, and I respect it: their own ["Best Video Intelligence APIs" list](https://mixpeek.com/curated-lists/best-video-intelligence-apis) ranks Mixpeek eighth of eight. (That list is stamped "Last tested: February 1, 2026," and for the record doesn't mention us at all — which is roughly why this post exists.)

**What it's genuinely great at.** Two things. First, retrieval over your own data: hybrid dense+sparse+BM25 search on your own S3/GCS bucket, cross-modal joins ("find the moment our CEO said *guidance* while the slide read *Q4 outlook*"), time-travel queries, unsupervised taxonomy discovery. We have no equivalent to any of that — no embeddings, no corpus layer, nothing. Second, the agent surface, which is the best we catalogued anywhere: **four hosted MCP servers** (48/20/11/17 tools), a per-retriever typed MCP server, llms.txt, per-agent budget caps, standing queries, and a live sample namespace requiring no account, no API key, no setup. If you're building an agent that needs multimodal recall as a tool, this is the most agent-complete option on the market.

**Pricing.** From [their pricing page](https://mixpeek.com/pricing) (as of July 2026): video processing $0.05/minute, plans start at $25/month (Build) and $250/month (Scale) — the minimum doubles as a usage pool — plus vector storage at $0.33/GB/month, with query overages at $2/M beyond the plan pool. That storage line is the same "rent on your index" pattern as Twelve Labs' infra fee, priced by the gigabyte instead of the minute. No free tier; the floor is $25/month. For a rough batch-workaround comparison: recording camera footage to a bucket and letting Mixpeek process it works out to about $3.00/camera-hour at the base rate (estimate — assumes base extraction only, no feature stacking).

**Performance.** One published number: hybrid queries return "well under 100ms p95" — retrieval latency over already-indexed vectors, with no methodology attached. The processing pipeline itself (upload → searchable) has no published latency at all.

**Choose Mixpeek when** you have a media archive in your own object storage and want it searchable by what's shown or said — or your agent needs a multimodal memory with real database semantics. **Choose us when** you need an *answer* rather than a *result set*. Mixpeek has a corpus layer and no verdict layer; we have a verdict layer and no corpus layer. Those are complements more than competitors, honestly.

---

## Primate Vision (ours — bias fully engaged)

**What it is.** A perception API that answers questions about video. You point it at a live stream (managed WebRTC) or an uploaded file, ask in plain English — "is anyone climbing the fence?", "did the forklift enter the loading bay?" — and get a verdict: **yes / no / indeterminate**, with a calibrated 0–1 confidence score, timestamped evidence segments, and an annotated overlay evidence clip you can watch and forward. The verdict vocabulary is closed. It is never free-form generation, which is the mechanism behind the hallucination-risk cell in the scorecard: a model that isn't generating language can't confabulate a paragraph about a kitchen that isn't there.

**What it doesn't do — our own honesty list.** No bounding-box JSON (evidence is the overlay video, not coordinates — Google VI and Rekognition beat us there). No embeddings or semantic search (Twelve Labs and Mixpeek beat us there). No transcription, OCR, or audio — we're visual-only (Gemini and Twelve Labs beat us there). No negative prompts, and no multi-video batch. If those are your primary needs, the sections above tell you who to pick, by name.

**Performance.** We publish ours: 45ms p50 / 316ms p95 per-frame streaming inference, 11.8 fps sustained, 6.6s session setup, on a public [/performance](https://www.primateintelligence.ai/performance) page. I keep waiting for a competitor to publish theirs. Across every vendor in this roundup — Google VI, Twelve Labs, Rekognition, Gemini, Mixpeek — the number of published end-to-end video-analysis latency figures is zero.

**Pricing — two lanes, stated plainly.** Metered: **$0.01 per second of source video** ($0.60/min), flat and fps-independent; queued time free, failed jobs free. That's the on-demand lane — clips, incidents, agent-invoked checks, bursty workloads. A 30-second clip in, a verdict with evidence out: $0.30. For 24/7 continuous monitoring and camera fleets we deliberately do *not* sell the metered path — a nonstop 30-day stream would be $25,920/month at self-serve rates and nobody should pay that. Continuous and fleet workloads run on [highly discounted enterprise plans](https://primateintelligence.ai/pricing#enterprise) — dedicated capacity or low-cost on-site deployment, at a small fraction of the metered rate.

**Free tier.** 6,000 seconds of real processing on signup — $60 face value, no card required, API keys available immediately. And for agents, no signup at all: `POST /v1/sandbox` returns a working sandbox key in one call. **Your AI agent can do it for you — right from Claude**: MCP server, llms.txt, markdown docs twins, and that sandbox endpoint mean an agent can discover the API, get a key, run an analysis, and read back the verdict without a human touching a dashboard.

**Choose us when** you need a managed API that watches video — live or uploaded — and returns a deterministic, auditable answer with evidence. **Choose someone else when** you need search, spatial JSON, transcripts, or the lowest possible per-minute price on offline archives — see above, by name.

---

## The pricing table nobody publishes honestly

Here's the normalized comparison, including the number a competitor would quote against us — because if I hide it, you should stop trusting everything above.

| Vendor | Billing model | Headline price | $/camera-hour normalized |
|---|---|---|---|
| Gemini (Flash-Lite → 3.6 Flash) | per-token, 1fps default | $0.30–$1.50 / 1M input tokens | $0.28–$1.39 *estimate, input-only* |
| AWS Rekognition streaming | per-minute streamed | $0.00817/min | $0.49 — grandfathered accounts only, ≤120s/event |
| Twelve Labs | per-min index + monthly infra + per-query | $0.042/min indexing | $1.75–$2.52 *estimate, batch workaround* |
| Mixpeek | per-min processed + monthly floor | $0.05/min, $25/mo floor | ~$3.00 *estimate, batch workaround* + $0.33/GB/mo storage |
| AWS Rekognition stored | per-minute per API | $0.10/min per detector | $6.00 per detector, stacking |
| Google Video Intelligence | per-feature × per-minute | $0.10–$0.12/min per feature | $6.00–$7.20 per feature, stacking |
| **Primate Vision — metered** | per-second, flat | $0.01/sec | **$36.00** † |
| **Primate Vision — enterprise** | contact us | custom | *a small fraction of the metered rate — the 24/7 / fleet lane* |

† *Metered rate normalized for comparison. Primate does not sell 24/7 continuous monitoring at the metered rate — continuous and fleet workloads use [enterprise plans](https://primateintelligence.ai/pricing#enterprise). The metered lane's real unit is the answered question, not the camera-hour: a 30-second incident clip → verdict + confidence + evidence = $0.30.*

Three things to hold in mind reading any table like this one:

- **The cheap rows are raw-annotation or raw-token prices.** What the buyer still builds on top — structured output contracts, a streaming layer, evidence generation, confidence calibration — is the actual cost of ownership.
- **Fidelity parity changes the math.** The token rows sample ~1fps. At 12fps parity, Gemini's input tokens alone reach $0.28–$0.37/min — 46–62% of our all-in $0.60/min — before output tokens, and with no verdict layer attached.
- **Nobody's $/camera-hour is what your workload costs.** Rekognition streaming caps at 120 seconds per event. Twelve Labs and Mixpeek charge ongoing rent on the index. Gemini Live compounds per turn. Read the gotcha column, not the headline.

---

## The bottom line

Score the six honestly and the category splits clean. Archive search: Twelve Labs, and it isn't close. Retrieval infrastructure for agents: Mixpeek, with the best MCP surface anyone has shipped. Cheap conversational Q&A over stored files: Gemini. Fixed-taxonomy annotation with hyperscaler compliance: Google VI or Rekognition, depending on which cloud already has your data. Live video in, deterministic verdict with evidence out, from a managed API an agent can drive end-to-end: that's us, and in 2026 that column is otherwise empty.

The bet behind this post is that being straight about all of it — including the dimensions we lose and the price point competitors screenshot — is worth more than any adjective we could have stacked instead. Roundups get cited when they're honest. We'll find out.

**[Try it free](https://primateintelligence.ai/signup)** — 6,000 seconds of real processing, no card. Or don't even sign up: tell your agent to `POST /v1/sandbox` and ask its first question.

---

*Method note: every competitor claim above traces to vendor docs, pricing pages, or release notes accessed 2026-07-31, linked inline. Estimates are labeled. If you find something stale or wrong, tell us and we'll fix it — that's the deal that makes a vendor roundup worth reading.*


---

<!-- source: https://primateintelligence.ai/compare/gemini.md -->

---
title: "Primate Vision vs. Gemini (API + Live) (2026): 1fps Stills vs. Native Frame Rate"
slug: "gemini"
author: "Matt Miesnieks"
date: "2026-07-31"
readTime: "6 min read"
tags: ["Comparisons","LLMs"]
excerpt: "Gemini is the strongest general-purpose video model you can rent — and at 1fps it's dramatically cheaper than us. But its live video input is spec-capped at 1fps JPEG stills with 2-minute sessions and a compounding token meter. Here's the honest head-to-head."
canonical: https://primateintelligence.ai/compare/gemini
---

Let me start with the concession, because it's a big one: Gemini is the strongest general-purpose video-understanding model you can rent, and at its default 1fps sampling it's dramatically cheaper than us. If you want conversational answers about stored video — summaries, Q&A, world-knowledge reasoning — it's excellent.

But for *real-time video analysis*, the spec sheet decides, not the marketing. Gemini Live's video input is hard-capped at **1fps JPEG stills**. Audio+video sessions last **2 minutes** without context compression. And the meter **re-bills the entire session context on every turn**. Primate Vision watches live video at native frame rates over managed WebRTC (45ms p50 per-frame inference) and returns a **deterministic verdict** — yes / no / indeterminate — with calibrated confidence, timestamps, and an annotated evidence video, on a flat per-second meter.

*A note on keeping Google straight: this post is about **Gemini** — Google's modern multimodal surface. Google Video Intelligence, the 2017-era annotation API, is a different product with different criticisms; we cover it in the [full landscape comparison](/compare/video-ai-landscape-2026). Don't conflate them — we don't.*

---

## Two different jobs

Gemini is a universal brain. Video is one input modality among many, decomposed into sampled frames plus audio tokens, answered with generated prose or JSON. Google's own docs describe the capability well — "describe, segment, and extract information from videos, answer questions about video content" ([video understanding docs](https://ai.google.dev/gemini-api/docs/video-understanding)).

Primate Vision is a measurement instrument. It watches video — live (managed WebRTC) or uploaded (up to 2 GiB, direct or URL ingest) — and answers one framed question with a deterministic verdict (`yes` / `no` / `indeterminate`), a calibrated 0–1 confidence score, timestamped evidence segments, and an annotated overlay evidence video. Verdicts are computed by similarity scoring against the video, not sampled from a language model, and you can pin the model version (`darwin-1.3`) so behavior changes only when you choose.

## The head-to-head

| Capability | Primate Vision | Gemini (API + Live) |
|---|---|---|
| Live video input | ✅ managed WebRTC, native frame rate | ⚠️ Live API: "images (JPEG <= 1FPS)" — frame-push stills, WebRTC only via third parties ([Live docs](https://ai.google.dev/gemini-api/docs/live-api)) |
| Live session length | ✅ up to 1 hour per session on top plans; API publishes your cap and warns before cutoff | ⚠️ audio+video capped at 2 minutes without compression; ~10-min connection lifetime ([session docs](https://ai.google.dev/gemini-api/docs/live-api/session-management)) |
| Published latency | ✅ 45ms p50 / 316ms p95, 11.8 fps sustained, 6.6s setup | ❌ none — "low-latency" marketing language only |
| Verdict contract | ✅ deterministic yes/no/indeterminate + calibrated 0–1 confidence | ⚠️ JSON schema possible, but uncalibrated, no indeterminate semantics, nondeterministic LLM sample |
| Reproducibility | ✅ deterministic verdict path, model pinning, deterministic test mode | ❌ `temperature`/`top_p`/`top_k` deprecated on 3.x (2026-07-21) — you can't even pin temperature=0 ([changelog](https://ai.google.dev/gemini-api/docs/changelog)) |
| Evidence artifacts | ✅ annotated overlay evidence video + timestamped clips | ❌ text/JSON/audio out only |
| Fast-action fidelity | ✅ native frame rate | ⚠️ Google's own docs: 1fps "might lose detail… Consider slowing down such clips" ([docs](https://ai.google.dev/gemini-api/docs/video-understanding)) |
| Transcript / OCR / audio | ❌ visual only | ✅ native audio, transcription, OCR |
| Embeddings / search | ❌ none | ✅ multimodal embeddings GA |
| Webhooks | ✅ Standard Webhooks + `Prefer: wait` | ✅ launched 2026-05-04 for Batch + long-running ops |

That last row matters for honesty: earlier analyses called Google "polling-only." That's now wrong for Gemini — they shipped webhooks in May 2026. (It remains true for Google Video Intelligence, the other Google surface.)

## The 1fps wall

This is the crux, and it's Google's spec, not our characterization. The Live API's input modalities are listed as "Audio…, images (JPEG <= 1FPS), text."

Think about what one frame per second means physically. A person falls in 0.7 seconds. A package is snatched between frames. At one still per second, fast events cannot be seen — not "detected less accurately," *not seen* — which is why Google's own video docs recommend "slowing down such clips" as the workaround for fast action. That's a fine tip for stored files. It is not available advice for a live camera.

On the file path you can request a higher custom fps. Then the token meter answers. Which brings us to pricing.

## Pricing: flat meter vs. compounding token curve

**Gemini bills tokens.** Each frame is 258 tokens at default resolution; ~300 tokens/second of video all-in. At the 1fps default on Flash-Lite, that's roughly **$0.28 per camera-hour input** (estimate, input-only) — 100×+ cheaper than our metered rate. I'm not going to pretend otherwise. If 1fps is enough and prose is enough, Gemini is very cheap and you should use it.

But two curves bend the other way:

1. **Fidelity parity.** At 12fps — what Primate Vision actually sustains — Gemini's *input tokens alone* reach **$0.28–$0.37/min** on frontier models (3.6 Flash / 3.1 Pro): 46–62% of Primate's **all-in $0.60/min**, before output tokens, before thinking tokens, and with no verdict layer, no calibration, and no evidence artifacts on top.
2. **The Live re-billing trap.** Google's own best-practices page: "Past tokens are re-processed and accounted for in each new turn… As a session lengthens, the cost per turn increases because the conversational history is re-processed" ([best practices](https://ai.google.dev/gemini-api/docs/live-api/best-practices)). Interactive live-video sessions get superlinearly expensive and CFO-unpredictable.

**Primate Vision has two lanes**, stated plainly:

1. **Metered — $0.01 per second of source video** ($0.60/min), flat, fps-independent — 10 seconds at 60fps bills as 10 seconds. Queued time free; failed jobs free. A 30-second clip → verdict + confidence + timestamps + evidence video = $0.30.
2. **[Enterprise — contact us](https://primateintelligence.ai/pricing#enterprise)** for 24/7 continuous monitoring and camera fleets: dedicated capacity or on-site deployment at a small fraction of the metered rate, under highly discounted enterprise plans.

Normalized honestly: at 1fps Gemini is ~$0.28–$1.39/camera-hour (estimate, input-only) versus $36/camera-hour† at our metered rate. The metered lane isn't priced for passive camera-hours — it's priced per answered question with evidence. When the workload *is* camera-hours, that's the enterprise lane.

† *Metered rate normalized for comparison. Primate does not sell 24/7 continuous monitoring at the metered rate — continuous and fleet workloads use enterprise plans; [contact us](https://primateintelligence.ai/pricing#enterprise).*

One more production consideration: model churn. Gemini 2.0 was shut down June 2026. Preview models are routinely retired on ~30-day notice. The Live model on the developer API is still `-preview` (GA only via Vertex). Every forced migration means re-validating your video pipeline — anyone who's shipped on a fast-moving model API knows exactly what that costs. Primate Vision versions models as pinnable resources. Behavior changes when you choose.

## Choose Gemini instead when…

These are its lanes, not ours:

- **Conversational "what am I looking at?" experiences.** Voice assistant over a screen share or camera where prose/audio *is* the product — Shopify's Sidekick is built on Gemini Live via Vertex. We don't do conversation.
- **Low-fps, low-stakes monitoring at massive scale**, where ~$0.30–$1.40/camera-hour beats everything and an occasional miss is tolerable.
- **Rich Q&A and summarization over stored footage** where world knowledge matters more than temporal precision — lectures, meetings, content libraries. And it can hear: transcription and audio understanding we simply don't have.
- **Semantic search over video** via multimodal embeddings (GA) — we offer no embeddings.
- **One-vendor GCP shops** that want LLM + vision + audio + embeddings under Vertex SLAs.

## Choose Primate Vision when…

- **The event is faster than one second.** Native frame rate vs. 1fps stills isn't a tuning difference; it's whether the event is visible at all.
- **The output feeds an alert, a workflow, or a compliance record.** A deterministic verdict with calibrated confidence and a watchable evidence video — versus a nondeterministic prose sample you can no longer even pin to temperature=0.
- **You need sessions longer than 2 minutes** on live audio+video, without context-compression gymnastics and resumption tokens.
- **You need a predictable bill.** Seconds × $0.01, flat — versus a token meter that compounds per turn.
- **Latency is a requirement.** We publish 45ms p50 / 316ms p95 / 11.8 fps / 6.6s setup on a public /performance page. Google publishes no latency numbers for video on either surface.

## Try it

**[Try it for free](https://primateintelligence.ai/signup)** — real processing, no card required to start.

**Your AI agent can do it for you — right from Claude.** Primate Vision ships an MCP server, llms.txt, markdown docs twins, and a sandbox key available in a single POST — your agent can get a key, upload a clip, and read back a verdict without a human touching a dashboard.

For 24/7 monitoring and camera fleets: [see our enterprise plans](https://primateintelligence.ai/pricing#enterprise).

---

*Method note: every Gemini claim traces to ai.google.dev docs, pricing, and changelog pages accessed 2026-07-31. Estimates are labeled. Google's own per-minute price equivalences are internally inconsistent with its documented tokenization (~8× gap) — we quote the documented token math.*


---

<!-- source: https://primateintelligence.ai/compare/google-video-intelligence.md -->

---
title: "Primate Vision vs. Google Video Intelligence (2026): Any Question You Can Type vs. a Label Menu Frozen in 2021"
slug: "google-video-intelligence"
author: "Matt Miesnieks"
date: "2026-07-31"
readTime: "7 min read"
tags: ["Comparisons","Cloud Vision"]
excerpt: "Google Video Intelligence is the cheapest serious way to bulk-annotate stored video — with a real SLA and a generous free tier. It's also a pre-LLM product whose release notes end in 2021, with no prompt parameter anywhere in the API. Here's the honest head-to-head."
canonical: https://primateintelligence.ai/compare/google-video-intelligence
---

Google Cloud Video Intelligence has not shipped a feature since November 1, 2021. That's not my characterization — it's Google's own [release notes page](https://docs.cloud.google.com/video-intelligence/docs/release-notes), where the entries simply stop. The only product event since is a shutdown: Celebrity Recognition, deprecated 2024, switched off September 2025 — and the [pricing page](https://cloud.google.com/products/video-intelligence/pricing) still lists a price for it.

Here's the thing though: a stable legacy annotator kept running under a real SLA is a legitimate thing to be. GVI is still the cheapest serious way to bulk-annotate stored video with commodity metadata — labels, shots, OCR, transcripts, explicit-content flags — with a formal 99.9% SLA and a genuinely generous free tier. If your question maps onto Google's fixed feature list, it's ~5–6× cheaper per raw hour than us and you should use it.

But it is a pre-LLM product. There is no prompt parameter anywhere in the API. If your question is your own — *"did the forklift enter the loading bay?"* — GVI has no way to hear it. Primate Vision answers exactly that: plain English in, a deterministic verdict out (`yes` / `no` / `indeterminate`), with calibrated confidence, timestamped evidence clips, and an annotated evidence video, live or on demand.

*A note on keeping Google straight: this post is about Google Video Intelligence, the 2017-era annotation API. Gemini — Google's modern, promptable video surface — is a different product with different criticisms, covered in [its own comparison](/compare/gemini). Don't conflate them; we don't.*

---

## A menu of nine features vs. any question you can type

Video Intelligence answers: *"What does Google's taxonomy see in this video?"* You pick from a feature enum — `LABEL_DETECTION`, `OBJECT_TRACKING`, `SHOT_CHANGE_DETECTION`, text, logos, faces, people, explicit content, speech — and get back an annotation document: entities, timestamps, confidences, bounding boxes. For catalog indexing or content moderation, the fixed classifier is exactly the job. ([Google VI docs](https://cloud.google.com/video-intelligence/docs))

What it cannot do is take a question. If "forklift entering a loading bay" isn't in the taxonomy, your options are to reason over raw label JSON yourself, or to train your own AutoML model on Vertex first — a path that only exists for streaming, has been Beta since 2019, and now points into a Vertex platform Google is actively reshuffling into its Gemini agent stack ([action recognition docs](https://docs.cloud.google.com/video-intelligence/docs/streaming/action-recognition)).

Primate Vision answers: *"Is X happening — and can you prove it?"* Point it at a live stream (managed WebRTC) or a file (upload or URL ingest, up to 2 GiB), ask in plain English, and get a verdict from a closed vocabulary — never free-form generation — with a calibrated confidence score, timestamped segments, and an annotated overlay evidence video, at 45ms p50 per-frame inference published on a public [/performance](https://www.primateintelligence.ai/performance) page.

## The head-to-head

| Capability | Primate Vision | Google Video Intelligence |
|---|---|---|
| Open-vocabulary questions | ✅ core product — plain English in, verdict out | ❌ feature enums only; no prompt parameter anywhere ([docs](https://cloud.google.com/video-intelligence/docs)) |
| Verdict contract | ✅ yes/no/indeterminate + calibrated 0–1 confidence, model pinning (`darwin-1.3`) | ❌ annotation documents with per-entity confidence — never an answer to a question |
| Live stream input | ✅ managed WebRTC, native frame rate | ⚠️ Beta gRPC; live protocols require compiling Google's AIStreamer C++/GStreamer proxy yourself ([live streaming docs](https://docs.cloud.google.com/video-intelligence/docs/streaming/live-streaming)) |
| Published latency | ✅ 45ms p50 / 316ms p95, 11.8 fps sustained | ❌ none published; streaming granularity "about 1 second of video" (sample-code comment) |
| Custom action/event detection | ✅ open-vocab, live + stored | ⚠️ BYO AutoML model, streaming Beta only ([action recognition](https://docs.cloud.google.com/video-intelligence/docs/streaming/action-recognition)) |
| Timestamped segments | ✅ | ✅ segment/shot/frame offsets — solid |
| Evidence artifacts | ✅ annotated overlay evidence video + clips | ❌ JSON annotations only |
| Bounding boxes in JSON | ❌ overlay only, no coordinates | ✅ normalized boxes, GA — a real GVI win |
| Transcript / OCR | ❌ visual only | ✅ GA speech transcription with diarization + text detection |
| Webhooks | ✅ Standard Webhooks signing, retries, redelivery + `Prefer: wait` | ❌ long-running Operations, polling only |
| SLA | ⚠️ published targets (99.9% control plane), not contractual in v1; Enterprise SLA available | ✅ formal 99.9% with financial credits — but streaming/Beta excluded ([SLA](https://cloud.google.com/video-intelligence/sla)) |
| Product momentum | ✅ shipping (webhooks, URL ingest, SDKs, MCP — all 2026) | ❌ release notes end 2021-11-01; only recent change is a feature shutdown |

## Maintenance mode, by Google's own record

This isn't spin; it's Google's own documentation trail:

- **The release notes have no entries after November 1, 2021.** Four-plus years without a shipped feature.
- **The only recent product event is a shutdown.** Celebrity Recognition was switched off September 16, 2025 — the [pricing page](https://cloud.google.com/products/video-intelligence/pricing) still lists a price for it while the [quota page](https://docs.cloud.google.com/video-intelligence/quotas) shows its quota forced to zero ([deprecations](https://docs.cloud.google.com/video-intelligence/docs/deprecations)).
- **The customization escape hatch is wobbling.** GVI's action-recognition docs still point at Vertex AI for AutoML training, while Google reorganizes Vertex into its Gemini agent platform and has already deprecated the adjacent Vertex AI Vision (EOL September 30, 2026) ([Vision AI overview](https://docs.cloud.google.com/vision-ai/docs/overview)). Whether you can still train a *new* AutoML video action model today is unverified — which is itself the tell.

Google's video-understanding future is Gemini — promptable, multimodal, actively developed. GVI is the legacy annotator kept running under SLA. Nobody should start a new build in 2026 expecting this API to grow toward their use case. The roadmap visibly points elsewhere.

## Live video: Beta since 2019, bring your own C++ proxy

GVI does have a streaming surface — a gRPC bidirectional API — but the fine print matters. It has carried a Beta label since 2019, it's explicitly excluded from the SLA under Pre-GA terms, and it accepts file-format chunks, not camera protocols: to connect RTSP, HLS, or RTMP you compile and operate Google's AIStreamer proxy yourself — a C++/GStreamer binary you build with Bazel or Docker and wire up with named pipes ([live streaming docs](https://docs.cloud.google.com/video-intelligence/docs/streaming/live-streaming)). No published latency numbers exist for it, stored or streaming.

Primate Vision's live path is a managed WebRTC handshake at native frame rates, with mid-stream prompt changes over one WebSocket message and per-session limits the API surfaces up front.

## Annotations vs. answers

Here's the deepest difference. GVI's output is an *input to your system*: a JSON document of everything its models saw, which your code must then reason over to decide whether the thing you care about happened. That reasoning layer — mapping "labels: forklift (0.87), warehouse (0.91)" onto "did the forklift enter the loading bay between 2 and 3 AM?" — is left entirely to you.

Primate Vision's output is a *decision*: a verdict from a closed vocabulary with a calibrated 0–1 confidence, computed by similarity scoring against the video rather than sampled from a language model, with model version pinning and a fully deterministic test mode for CI. And every yes ships with proof a human can watch: timestamped clips plus an annotated overlay evidence video. GVI returns coordinates. We return a case file.

## Pricing: two very different meters

**GVI stacks features.** Billing is per-feature × per-minute: label detection $0.10/min stored ($0.12 streaming), object tracking $0.15/min, shot detection $0.05/min, and so on — each feature you request re-meters the same footage, so four features on one video bills four times the minutes. Two gotchas from Google's own pricing page: partial minutes round *up* per request (1,000 five-second clips bill as 1,000 minutes, not 84), and the first 1,000 minutes per feature per month are free — genuinely generous for evaluation and hobby workloads ([GVI pricing](https://cloud.google.com/products/video-intelligence/pricing)).

**Primate Vision has two lanes**, stated plainly:

1. **Metered — $0.01 per second of source video** ($0.60/min), flat and fps-independent. Queued time free; failed jobs free; `validate_only` dry-runs estimate cost without touching GPU or credits. A 30-second clip → verdict + confidence + timestamps + evidence video = $0.30.
2. **[Enterprise — contact us](https://primateintelligence.ai/pricing#enterprise)** for 24/7 continuous monitoring and camera fleets: dedicated capacity or on-site deployment at a small fraction of the metered rate, under highly discounted enterprise plans.

The honest normalization: single-feature label detection on GVI works out to **$6.00/camera-hour stored, $7.20 streaming** (post-free-tier), versus $36/camera-hour† at Primate's metered rate — roughly 5–6× cheaper per raw hour. If a fixed label taxonomy actually answers your question, take that gap. But the meters buy different things: GVI's dollar buys an annotation document you still have to reason over (and stacking the features you'd need narrows the gap fast — labels + object tracking is already $17.40/streaming-hour); ours buys the answered question, with evidence, live or on demand. You don't buy camera-hours on our metered lane. You buy verdicts.

† *Metered rate normalized for comparison. Primate does not sell 24/7 continuous monitoring at the metered rate — continuous and fleet workloads use enterprise plans; [contact us](https://primateintelligence.ai/pricing#enterprise).*

## Choose Google Video Intelligence instead when…

We'd rather you pick the right tool than the wrong one of ours:

- **You need commodity metadata over a large stored archive on GCP.** Labels, shots, OCR, transcripts at ~$0.05–$0.15/min per feature — with 1,000 free minutes per feature every month. For catalog indexing at scale, GVI's economics beat any promptable model, ours included.
- **Content moderation is the whole job.** The explicit-content classifier is exactly the fixed task GVI was built for, GA, under a real SLA.
- **You need bounding-box coordinates in JSON.** GVI's object tracking and person/face detection emit normalized boxes per frame for downstream analytics. Primate Vision renders overlays but does not emit box coordinates.
- **You need speech transcription or on-screen text.** Both GA, with speaker diarization. Primate Vision is visual-only: no transcription, no OCR, no audio path.
- **Your pipeline lives in GCP IAM/Audit-Log land** and a 99.9% SLA with financial credits (on the stored path) is a procurement requirement. We publish availability targets; we don't yet offer a self-serve contractual SLA.
- **Your files are huge.** GVI takes 50GB videos up to 3 hours; our cap is 2 GiB.

## Choose Primate Vision when…

- **Your question isn't on Google's menu.** Nine fixed features vs. anything you can type. No AutoML training project, no Vertex dependency, no reasoning layer to build — one API call.
- **You need an answer, not an annotation document.** A yes/no/indeterminate verdict with calibrated confidence feeds an alert, a workflow, or a compliance record directly. GVI's JSON still needs your own decision logic on top.
- **The camera is live.** Managed WebRTC at native frame rates vs. a 2019 Beta gRPC surface that requires self-compiling a C++/GStreamer proxy and sits outside the SLA.
- **You need proof.** Timestamped evidence clips plus an annotated overlay video a human can watch and forward — versus raw JSON.
- **You want to be told when it's done.** Webhooks with Standard-Webhooks signing, retries, and redelivery — or `Prefer: wait` — versus polling a long-running Operation.
- **You're betting on a roadmap.** We shipped URL ingest, webhooks, SDKs, and an MCP server this year. GVI's last release note is from 2021.

## Try it

**[Try it for free](https://primateintelligence.ai/signup)** — real processing, no card required to start.

**Your AI agent can do it for you — right from Claude.** Primate Vision ships an MCP server, llms.txt, markdown docs twins, and a sandbox key available in a single POST — your agent can get a key, upload a clip, and read back a verdict without a human touching a dashboard.

For 24/7 monitoring and camera fleets: [see our enterprise plans](https://primateintelligence.ai/pricing#enterprise).

---

*Method note: every Google Video Intelligence claim traces to Google's live docs, pricing page, quotas, deprecations, and SLA pages accessed 2026-07-31. Estimates are labeled.*


---

<!-- source: https://primateintelligence.ai/compare/nvidia-vss.md -->

---
title: "Primate Vision vs. NVIDIA VSS (2026): A Managed API vs. a Build-It-Yourself Blueprint"
slug: "nvidia-vss"
author: "Matt Miesnieks"
date: "2026-07-31"
readTime: "7 min read"
tags: ["Comparisons","Infrastructure"]
excerpt: "NVIDIA VSS is the strongest self-hosted real-time video-analytics stack — and by NVIDIA's own docs, it's a reference architecture you must build and operate, not a product you can buy. Here's the honest head-to-head against a managed API."
canonical: https://primateintelligence.ai/compare/nvidia-vss
---

NVIDIA VSS is the strongest self-hosted real-time video-analytics stack you can deploy — genuinely real-time, RTSP-first, GA since June 2026, with published benchmarks and the most forward-looking agent-operations story in the category. I have real respect for it.

It is also, by NVIDIA's own documentation, not a product. It's a reference architecture you must build and operate: you supply the GPUs, the authentication, the TLS, the Elasticsearch, the message brokers, and the ops team. A project, not a product.

Primate Vision is the managed inverse: one API call, a **deterministic verdict** (yes / no / indeterminate) with calibrated confidence, timestamped evidence, an annotated evidence video, and a published 45ms p50 latency — with nothing to deploy. Which one you want comes down to two questions: do you own GPUs and an infra team, and can your data leave the building?

*This is part of our [full 2026 video-AI landscape comparison](/compare/video-ai-landscape-2026).*

---

## What VSS actually is

In NVIDIA's own words, VSS is "a GPU-accelerated reference architecture for building video analytics agents with real-time verified alerts, visual Q&A, and automated reporting" — VLMs (Cosmos Reason, Qwen3-VL), LLMs (Nemotron), RAG, and NIM microservices, shipped as Docker Compose profiles and Helm charts you deploy on your own NVIDIA GPUs. ([VSS docs](https://docs.nvidia.com/vss/latest/))

The docs are refreshingly explicit about what that means: "Deploy VSS in a trusted, isolated network. Do not expose the VSS services directly to untrusted networks. **This release assumes authentication, TLS termination, rate limiting, and external access controls are provided by the infrastructure.**"

You are the infrastructure.

Primate Vision is a hosted API. You point it at a live stream (managed WebRTC) or a file (upload or URL ingest, up to 2 GiB), ask a question in plain English, and get a deterministic verdict — `yes` / `no` / `indeterminate` — with a 0–1 calibrated confidence score, timestamped evidence segments, and an annotated overlay evidence video. Time-to-first-answer on Primate Vision: minutes, with a curl call — a free sandbox key is one POST away. Time-to-first-answer on VSS: days to weeks of deployment work. That's not a criticism of NVIDIA; it's the definition of a reference architecture.

## The head-to-head

| Dimension | Primate Vision | NVIDIA VSS |
|---|---|---|
| Delivery model | ✅ managed API, nothing to deploy | ❌ self-hosted blueprint: Compose/Helm, NIMs, Elasticsearch, brokers, GPU tuning |
| Live stream input | ✅ managed WebRTC | ✅ RTSP first-class; up to 100-stream configs, 16×1080p concurrent tested |
| Real-time, GA | ✅ 45ms p50 / 316ms p95 published | ✅ GA 3.2 (June 2026), hardware-dependent benchmarks published per microservice |
| Verdict contract | ✅ deterministic yes/no/indeterminate + calibrated 0–1 confidence, one clean JSON | ⚠️ VLM-generated yes/no verdicts — stochastic, uncalibrated; results fragment across Elasticsearch, Kafka topics, and agent chat |
| Documented verdict failure modes | — | ⚠️ their own release notes: captions containing "yes"/"true" create spurious incident records; negative-intent search matches positive results ([release notes](https://docs.nvidia.com/vss/latest/release-notes.html)) |
| Bounding boxes / 3D tracking | ❌ overlay evidence video only, no bbox JSON | ✅ DeepStream detection/tracking JSON + synchronized multi-camera 3D tracking |
| Embeddings / search | ❌ none | ✅ Cosmos-Embed1 semantic search (note: default index lifecycle deletes search embeddings after 48h min age) |
| Edge / air-gap | ❌ cloud only | ✅ core story — DGX Spark to AGX Thor, zero data egress |
| Multi-camera scale | ❌ one active stream per account today | ✅ tested at 16 concurrent 1080p, 100-stream configs |
| Auth / TLS / SLA | ✅ managed; published availability targets | ❌ your burden; no SaaS SLA (nothing is hosted) |
| Webhooks | ✅ Standard Webhooks signing, retries, redelivery, `Prefer: wait` | ⚠️ Kafka/Redis/MQTT broker integration — not developer-facing webhooks |

## Verdicts you can put in an audit trail

VSS's alert-verification layer answers yes/no — but the answer is generated by a VLM, with no calibration and with failure modes NVIDIA itself documents: caption text containing "yes" or "true" spawning spurious incidents, prompt clobbering when concurrent agent sessions share a backend, NIM crash-recovery requiring a stack redeploy. None of that disqualifies a reference architecture. It's honest engineering documentation, and I respect NVIDIA for publishing it. But it is the difference between a research-grade stack and a verdict you'd attach to a compliance record.

Primate Vision's verdict is deterministic: a closed vocabulary computed by similarity scoring against the video — not sampled from a language model — with a calibrated confidence score, a pinnable model version (`darwin-1.3`), a deterministic test mode for CI, and one clean response shape: verdict, confidence, timestamped clips, evidence video URL. One contract, not three storage systems.

## Pricing: opposite cost structures

**VSS has no usage meter at all.** The software is downloadable; the costs are (1) your GPUs and (2) an NVIDIA AI Enterprise subscription at **$4,500 per GPU per year list** ($1,125 for Inception startups) for supported production, or $1/GPU-hour consumption in the cloud plus instance costs. ([NVAIE pricing](https://docs.nvidia.com/ai-enterprise/planning-resource/licensing-guide/latest/pricing.html))

The honest normalization (estimate — every number depends on GPU choice, utilization, and model layout): a minimal cloud deployment on 2× H100 runs roughly **$14–22/hour for the box**. Spread across the tested 16 concurrent streams at full utilization, that's ~$0.9–1.4 per camera-hour; self-managed at fleet scale, perhaps $0.3–0.6. At single-stream utilization, it's the full $14–22/hour. The structural truth is simple: **high fixed cost, near-zero marginal cost**. The exact inverse of a usage meter.

**Primate Vision has two lanes:**

1. **Metered — $0.01 per second of source video** ($0.60/min), flat, fps-independent. Queued time free; failed jobs free. A 30-second clip → verdict + evidence = $0.30, with zero fixed cost, zero GPUs, zero NVAIE contracts.
2. **[Enterprise — contact us](https://primateintelligence.ai/pricing#enterprise)** for 24/7 continuous monitoring and camera fleets: dedicated capacity or on-site deployment at a small fraction of the metered rate, under highly discounted enterprise plans — which is the correct comparison against a VSS fleet deployment, not the metered rate.

Normalized at the metered rate we're $36/camera-hour† — far above VSS at fleet utilization. We print that number ourselves because hiding it would be dishonest. The two cost curves cross exactly where you'd expect: occasional questions and bursty workloads favor a meter; owned 24/7 fleets favor owned GPUs — or our enterprise lane, which exists for precisely that workload and can include on-site deployment.

† *Metered rate normalized for comparison. Primate does not sell 24/7 continuous monitoring at the metered rate — continuous and fleet workloads use enterprise plans; [contact us](https://primateintelligence.ai/pricing#enterprise).*

One VSS budget line worth knowing: all 2.x/3.0/3.1 container images are deprecated and get **removed from NGC on September 30, 2026** — self-hosters are on a forced re-deploy treadmill.

## Choose NVIDIA VSS instead when…

For some buyers these are decisive, and they should be:

- **Your data cannot leave the building.** Sovereign, defense, critical-infrastructure, or privacy-bound deployments where *any* cloud API — including ours — is disqualified. VSS runs air-gapped, edge profiles down to DGX Spark and AGX Thor.
- **You run a 24/7 multi-camera estate with budgeted GPU capex and a platform team.** Dozens-to-hundreds of streams, near-zero marginal cost per added camera — structurally cheaper than any usage meter at that scale.
- **You need classic CV fused with VLM reasoning**: DeepStream tracking, tripwires, ROI rules, synchronized multi-camera 3D tracking — capabilities no cloud vision API (ours included) exposes.
- **You're an ISV building a video-analytics product** and want an NVIDIA-blessed reference architecture instead of assembling DeepStream + vLLM + RAG yourself. Their agent-ops story (Agent Skills, VA-MCP servers, validated agent harnesses) is genuinely the most forward-looking in the category.

## Choose Primate Vision when…

- **You want an answer this afternoon, not a deployment this quarter.** One curl call vs. Compose stacks, Elasticsearch, brokers, and GPU memory tuning.
- **You don't have (or want) GPUs and an infra team.** No $4,500/GPU/year licensing, no capex, no ops burden, no upgrade treadmill.
- **The verdict has to be trustworthy and auditable.** Deterministic closed-vocabulary answers with calibrated confidence and watchable evidence — versus stochastic VLM verdicts with documented spurious-incident bugs.
- **You need managed security and a clean contract**: auth, TLS, and rate limiting are our problem, and every answer arrives in one JSON shape with webhooks or `Prefer: wait`.
- **Your workload is bursty or on-demand.** Pay $0.30 for a 30-second question; pay nothing when idle.

## Try it

**[Try it for free](https://primateintelligence.ai/signup)** — real processing, no card required to start.

**Your AI agent can do it for you — right from Claude.** Primate Vision ships an MCP server, llms.txt, markdown docs twins, and a sandbox key available in a single POST — your agent can get a key, upload a clip, and read back a verdict without a human touching a dashboard.

For 24/7 monitoring and camera fleets: [see our enterprise plans](https://primateintelligence.ai/pricing#enterprise).

---

*Method note: every VSS claim traces to NVIDIA's docs, release notes, performance pages, and NVAIE licensing guide accessed 2026-07-31. All $/camera-hour figures for VSS are labeled estimates with assumptions stated; NVIDIA publishes no such number.*


---

<!-- source: https://primateintelligence.ai/compare/open-vlms.md -->

---
title: "Primate Vision vs. Self-Hosted Open VLMs (2026): Buying Answers vs. Building a Video Pipeline"
slug: "open-vlms"
author: "Matt Miesnieks"
date: "2026-07-31"
readTime: "7 min read"
tags: ["Comparisons","Open Source"]
excerpt: "Qwen3-VL, InternVL, LLaVA, Moondream — open VLMs are excellent, improving quarterly, and free to download. But the weights are not the product. No open model ships streaming ingest, calibrated verdicts, evidence artifacts, or a published latency number. Here's the honest math."
canonical: https://primateintelligence.ai/compare/open-vlms
---

Open vision-language models are the strongest honest case against any managed video API, ours included. Qwen3-VL, InternVL3.5, LLaVA-OneVision-2, Moondream — genuinely excellent, improving every quarter, free to download. If you have a large in-house ML team, a big 24/7 camera fleet, or video that legally cannot leave the building, self-hosting is the right call and I'll say so plainly below.

But the weights are not the product.

No open model ships a streaming ingestion layer, a calibrated verdict, evidence artifacts, webhooks, or a published latency number. Every one of those is a build project measured in engineer-months, then a permanent ops burden. Primate Vision is the finished version of that build: a managed API that watches video live over WebRTC at native frame rates and returns a deterministic verdict — `yes` / `no` / `indeterminate` — with a calibrated 0–1 confidence score and an annotated evidence video, at 45ms p50 per-frame inference, published on a public [/performance](https://www.primateintelligence.ai/performance) page.

The real comparison isn't model vs. model. It's a parts list vs. an answer.

*This is part of our [full 2026 video-AI landscape comparison](/compare/video-ai-landscape-2026).*

---

## Two different purchases

Downloading Qwen3-VL gets you a state-of-the-art model — Alibaba calls it "the most powerful vision-language model in the Qwen series to date," with native 256K context and timestamp-grounded video understanding ([Qwen3-VL GitHub](https://github.com/QwenLM/Qwen3-VL)). What it does *not* get you: frame capture from a camera, session management, batching, a verdict schema, confidence calibration, clip extraction, overlay rendering, job orchestration, webhooks, an eval harness, or anyone on call when it breaks at 3am. Every item on a managed product's feature list is a build project here — weeks to months of engineering, then permanent MLOps.

Primate Vision answers the question those teams are ultimately trying to answer: *"Is X happening — and can you prove it?"* Point it at a live stream (managed WebRTC) or a file (upload or URL ingest, up to 2 GiB), ask in plain English, and get a verdict from a closed vocabulary — never free-form generation — with a calibrated 0–1 confidence score, timestamped evidence segments, and an annotated overlay evidence video. Model version pinned (`darwin-1.3`), cost known before you run it (`validate_only` dry-run), zero GPUs owned.

## The head-to-head

| Capability | Primate Vision | Self-hosted open VLMs |
|---|---|---|
| Live stream input | ✅ managed WebRTC, native frame rate | ❌ no model ships a streaming layer — RTSP/WebRTC capture, batching, session state are all DIY |
| Published latency | ✅ 45ms p50 / 316ms p95, 11.8 fps sustained | ⚠️ per-frame real-time achievable with small models; no published end-to-end latency anywhere — entirely deployment-dependent |
| Verdict contract | ✅ closed-vocab yes/no/indeterminate + calibrated 0–1 confidence | ⚠️ constrained decoding can force a yes/no schema, but confidence is uncalibrated and hallucination risk is yours |
| Timestamped segments | ✅ included | ⚠️ Qwen3-VL claims "text–timestamp alignment… precise, timestamp-grounded event localization" — vendor claim, production accuracy unverified ([Qwen3-VL GitHub](https://github.com/QwenLM/Qwen3-VL)) |
| Evidence artifacts | ✅ annotated overlay evidence video + timestamped clips | ❌ nothing built-in; clip extraction and overlay rendering are yours to build |
| Bounding boxes / spatial JSON | ❌ overlay only, no JSON | ✅ Qwen3-VL ships 2D *and* 3D grounding cookbooks — a genuine open-VLM strength |
| Transcript / OCR / audio | ❌ visual only | ⚠️ Qwen3-VL OCR in 32 languages (strong); audio needs a separate ASR model |
| Batch/async + webhooks | ✅ webhooks (Standard Webhooks signing, retries, redelivery) + `Prefer: wait` | ❌ self-built — vLLM batch APIs help, but job orchestration and webhooks are yours |
| Edge / air-gap | ❌ cloud API (enterprise plans include on-site deployment options) | ✅ the category's ace: weights run fully offline, from Moondream on edge boxes to Qwen3-VL on your own racks |
| Docs / SLA / support | ✅ OpenAPI 3.1 on prod, npm + PyPI SDKs, MCP server, llms.txt; published availability targets | ⚠️ HF/vLLM community quality is good, but no product docs, no SLA, no support line |

## The streaming layer no model ships

This is the category's cleanest structural gap: **not one open VLM ships live-stream ingestion.** There's no RTSP listener, no WebRTC endpoint, no session manager in any repo — Qwen3-VL, InternVL3.5, LLaVA-OneVision-2, Moondream, none of them. "Video input" means frames or files you've already captured. To watch a camera you build the capture pipeline, the frame sampler, the batching logic, the reconnect handling, and the latency proof — and then you maintain it as the model landscape shifts under you every quarter.

On Primate Vision that layer is the product: managed WebRTC sessions at native frame rates, mid-stream prompt changes over one WebSocket message (`update_prompt` — no reconnect, no re-upload), session caps and warnings surfaced in the API, and per-second metering that stops when the session ends.

## Verdicts: a contract vs. constrained decoding

Self-hosted, you can JSON-mode a VLM into answering "yes" or "no." What you can't get from any open checkpoint is a *calibrated* answer. No vendor publishes verdict calibration, sampling defaults are stochastic (Qwen recommends temperature 0.7 for VL models), and generative decoders confidently assert events that never happened. Nobody publishes hallucination rates on surveillance-style footage — the category's biggest evidential gap. If your output feeds an alert, a workflow, or a compliance record, that gap is your problem to close, with your own eval harness and your own QA.

Primate Vision's verdict is a different kind of object: an answer drawn from a closed vocabulary, computed by similarity scoring against the video rather than sampled from a language model, with a calibrated 0–1 confidence score, model version pinning so behavior changes only when you choose, and a fully deterministic test mode for CI. And every yes ships with proof a human can watch and forward.

Worth stating honestly: open models are also still behind the closed frontier on hard video reasoning — on Video-MME-v2's non-linear split, Gemini-3-Pro scores 49.4 vs. 39.1 for the best open model ([arXiv 2604.05015](https://arxiv.org/abs/2604.05015)). The free lunch is real, but it isn't the best lunch.

## Pricing: two lanes vs. the GPU bill

**Self-hosting's cost floor is genuinely spectacular — let's do the math in the open.** Weights are free (check each checkpoint's license tag on Hugging Face — terms vary). An RTX 4090 rents for $0.69/hr on [RunPod](https://www.runpod.io/pricing), and spot marketplaces dip lower. Our engineering estimate: a batched Qwen3-VL-8B deployment on one 4090 sustains roughly 10–20 concurrent 1fps camera streams, which pencils out to **~$0.03–$0.15 per camera-hour** (estimate — throughput figures are unpublished; measure before relying on them). That is the cheapest raw compute in the entire category, by orders of magnitude.

But that headline **silently assumes 100% GPU utilization and free engineers.** Add what the invoice doesn't show: the streaming-infra build (weeks to months), roughly 0.5–1 FTE of MLOps (~$8–15k/month loaded), an eval and hallucination-QA harness, prompt regression testing, spot-interruption handling, and on-call. With those carried, our estimate is that self-hosting wins only above roughly **20–50 continuous cameras** or under hard data-privacy requirements. Below that line, you're paying senior-engineer salaries to reinvent a product.

**Primate Vision has two lanes**, stated plainly:

1. **Metered — $0.01 per second of source video** ($0.60/min), flat and fps-independent. Queued time free; failed jobs free. A 30-second clip → verdict + confidence + timestamps + evidence video = $0.30. Run up to 10 questions against one video in a single batch call, each after the first at 50% off. `validate_only` estimates cost before touching GPU or credits.
2. **[Enterprise — contact us](https://primateintelligence.ai/pricing#enterprise)** for 24/7 continuous monitoring and camera fleets: dedicated capacity or on-site deployment at a small fraction of the metered rate, under highly discounted enterprise plans.

The honest normalization: the metered lane works out to $36/camera-hour† against a self-hosted ~$0.03–$0.15 — raw compute is cheaper by two to three orders of magnitude, and if you're running a large fleet with an ML team, that gap is real and you should weigh it (then talk to our enterprise team, because the metered rate is deliberately not our fleet price). But the two numbers buy different things. The GPU rate buys *tokens out of a model* — no ingestion, no verdict contract, no calibration, no evidence clips, no webhooks, no SLA, no one to page. The Primate meter buys *answered questions with proof*, live or on demand, on infrastructure someone else keeps running. You don't buy camera-hours on our metered lane. You buy verdicts.

† *Metered rate normalized for comparison. Primate does not sell 24/7 continuous monitoring at the metered rate — continuous and fleet workloads use enterprise plans; [contact us](https://primateintelligence.ai/pricing#enterprise).*

(One managed exception inside the open ecosystem: [Moondream Cloud](https://moondream.ai/pricing) sells its small models token-priced — ~$0.36/camera-hour real-time by our token math, estimate. It's a real option for lightweight visual QA, in a much smaller model class, and still leaves the verdict/evidence/streaming layers to you.)

## Choose self-hosted open VLMs instead when…

This alternative is legitimately right for whole categories of buyers:

- **You run a large 24/7 camera fleet (roughly 20–50+ continuous streams) and have an in-house ML team.** At that scale the per-camera-hour cost floor dominates and the fixed engineering cost amortizes. This is the strongest honest case against any managed API, ours included.
- **Your video cannot leave the premises.** Classified, defense, hospital-internal, air-gapped environments: open weights running fully offline are the only answer. (If your constraint is deployment location rather than absolute isolation, our enterprise plans include on-site deployment options — [talk to us](https://primateintelligence.ai/pricing#enterprise).)
- **You need breadth beyond verdicts.** One Qwen3-VL deployment covers OCR in 32 languages, document parsing, 2D/3D grounding, GUI agents — many workloads sharing one set of GPUs. Primate Vision is deliberately narrow: visual verdicts with evidence.
- **You want zero marginal cost on re-querying.** Once the GPUs are sunk cost, asking the same footage a thousand questions is free. Our meter bills each analysis.
- **You're researching or prototyping.** A $0 license and a rented 4090 beats any procurement cycle, and the open frontier improves quarterly at no cost to you — LLaVA-OneVision-2's codec-aligned video processing (April 2026) is genuinely novel architecture, free ([LMMs-Lab](https://www.lmms-lab.com/posts/llava_onevision_2/)).
- **You need to fine-tune on your own domain.** Open weights (and Moondream's Lens at $0.60/1M training tokens) offer customization no managed verdict API matches today.

## Choose Primate Vision when…

- **You need the answer this quarter, not the infrastructure roadmap.** Streaming ingestion, verdict schema, calibration, evidence rendering, webhooks — with open VLMs each is a build project; with us it's one API call.
- **The output feeds a decision.** A closed-vocabulary verdict with calibrated confidence and a watchable evidence video beats an uncalibrated generative "yes" when an alert, a workflow, or a compliance record hangs on it — and nobody publishes hallucination rates for open VLMs on this class of footage.
- **Latency is a requirement, not a hope.** We publish 45ms p50 / 316ms p95 / 11.8 fps sustained / 6.6s session setup on a public page. No open-VLM stack publishes anything comparable — you'd have to achieve *and prove* it yourself.
- **Your true TCO includes engineers.** Below fleet scale, the managed meter is almost always cheaper than 0.5–1 FTE of MLOps plus GPU utilization risk — and it's zero ops from day one.
- **The camera count is one, or bursty, or agent-driven.** Clips, incidents, on-demand checks: $0.30 answers a question about a 30-second clip, with proof, no GPU rented.

## Try it

**[Try it for free](https://primateintelligence.ai/signup)** — real processing, no card required to start.

**Your AI agent can do it for you — right from Claude.** Primate Vision ships an MCP server, llms.txt, markdown docs twins, and a sandbox key available in a single POST — your agent can get a key, upload a clip, and read back a verdict without a human touching a dashboard.

For 24/7 monitoring, camera fleets, and on-site deployment: [see our enterprise plans](https://primateintelligence.ai/pricing#enterprise).

---

*Method note: every open-VLM claim traces to vendor repos, model cards, pricing pages, and papers accessed 2026-07-31. All $/camera-hour figures for self-hosted deployments are labeled estimates with assumptions stated.*


---

<!-- source: https://primateintelligence.ai/compare/openai.md -->

---
title: "Primate Vision vs. OpenAI (2026): A Video API vs. a Frame-Extraction Workaround"
slug: "openai"
author: "Matt Miesnieks"
date: "2026-07-31"
readTime: "7 min read"
tags: ["Comparisons","LLMs"]
excerpt: "OpenAI ships the best general-purpose visual reasoning on the planet — for still images. There is no video input path in any OpenAI API, and the only video product they had shuts down September 2026. Here's the honest head-to-head between a video-native API and a frame stack."
canonical: https://primateintelligence.ai/compare/openai
---

Here's a fact that surprises almost everyone I say it to: **there is no video API at OpenAI.** Not for files, not for URLs, not for streams. Every "GPT watches video" demo you've seen is a caller-built pipeline that rips frames out of a video and mails the model a stack of stills.

And the direction of travel is away from video, not toward it. The only video-related API OpenAI ever shipped — Sora 2, generation-only — was deprecated March 24, 2026 and shuts down September 24, 2026 ([deprecations](https://developers.openai.com/api/docs/deprecations)). The SDK feature request asking for the video-input parity Gemini already has is still open ([openai/openai-node#1778](https://github.com/openai/openai-node/issues/1778)).

None of this makes OpenAI bad. GPT-5.6's visual reasoning over stills — charts, receipts, screenshots, documents — is the best available, and I'll say so repeatedly below. But if your input is *video* — moving, time-ordered, possibly live — OpenAI doesn't sell the product. Primate Vision is exactly that product: live WebRTC streams and 2 GiB uploads at native frame rates, returning a deterministic verdict — yes / no / indeterminate — with calibrated confidence, timestamped evidence, and published latency.

*This is part of our [full 2026 video-AI landscape comparison](/compare/video-ai-landscape-2026).*

---

## Two different questions

OpenAI Vision answers: *"What do you see in this image — and what should we do about it?"* Vision is a modality of their flagship reasoning models, not a video product — "Vision is the ability for a model to 'see' and understand images," per their own docs. Feed it stills (up to 1,500 per request, 512 MB total) and get world-class open-ended understanding, OCR, comparison, extraction, with the whole LLM ecosystem wrapped around the answer. ([images & vision guide](https://developers.openai.com/api/docs/guides/images-vision))

Primate Vision answers: *"Is X happening in this video — and can you prove it?"* Point it at a live stream (managed WebRTC) or a file (upload or URL ingest, up to 2 GiB), ask in plain English, and get a verdict from a closed vocabulary (`yes` / `no` / `indeterminate`), a calibrated 0–1 confidence score, timestamped evidence segments, and an annotated overlay evidence video — at 45ms p50 per-frame inference, published on a public [/performance](https://www.primateintelligence.ai/performance) page.

One of these products accepts video. The other genuinely does not.

## The head-to-head

| Capability | Primate Vision | OpenAI |
|---|---|---|
| Video file upload | ✅ MP4/MOV up to 2 GiB, presigned or URL ingest | ❌ images only — ≤512 MB, ≤1,500 images/request ([docs](https://developers.openai.com/api/docs/guides/images-vision)) |
| Live stream input | ✅ managed WebRTC, native frame rate | ⚠️ Realtime API = audio + still images, no video track ([realtime guide](https://developers.openai.com/api/docs/guides/realtime)) |
| Published latency | ✅ 45ms p50 / 316ms p95, 11.8 fps sustained | ❌ none; "Fast mode" claims only "up to 2.5× faster… at twice the price" ([changelog](https://developers.openai.com/api/docs/changelog)) |
| Verdict contract | ✅ deterministic yes/no/indeterminate + calibrated 0–1 confidence | ⚠️ JSON schema forces the shape, but confidence is model-generated text and answers are stochastic |
| Timestamped segments | ✅ per-clip start/end + confidence | ❌ no video input → no native timestamps; caller injects frame labels as text |
| Evidence artifacts | ✅ annotated overlay evidence video | ❌ text only |
| General reasoning / OCR / documents | ❌ visual verdicts only — no OCR, no transcription | ✅ world-class; plus transcription APIs from $0.0045/min ([pricing](https://developers.openai.com/api/docs/pricing)) |
| Spatial localization | ❌ overlay only, no bbox JSON | ⚠️ model-emitted boxes, but their own docs warn the model "struggles with tasks requiring precise spatial localization" |
| Batch + webhooks | ✅ webhooks (signed, retries, redelivery) + `Prefer: wait`; 2–10 prompts/video, 50% off after the first | ✅ Batch API at 50% off; platform webhooks — genuinely strong |
| Agent accessibility | ✅ llms.txt, markdown docs twins, MCP server, sandbox key in one POST | ✅ best-in-class: llms.txt, .md docs, MCP in Responses API, 6 SDKs, Terraform |

## "Just extract the frames" is not a video pipeline

The standard workaround — ffmpeg the video into 1fps stills, batch them into requests — is what every OpenAI video tutorial actually teaches, including OpenAI's own cookbook. Here's what it costs you beyond dollars:

- **Motion is invisible.** Anything that happens *between* samples never reaches the model. A fall, a thrown punch, a package grab takes well under a second.
- **Long video doesn't fit.** The 1,500-images-per-request cap means a 1-hour video at 1fps must be chunked into 3+ requests, each blind to the others' temporal context — a ~25-minute ceiling per request. ([docs limits](https://developers.openai.com/api/docs/guides/images-vision))
- **Time is a string.** No temporal primitives — no segments, no event timeline. You label frames "t=00:14:32" in text and trust a stochastic model to echo them back accurately.
- **You own the pipeline.** Frame extraction, chunking, retries, timestamp bookkeeping, cross-chunk reconciliation — all yours to build and operate, forever.

That's not a video pipeline. That's a stack of photographs and a prayer.

On Primate Vision the equivalent is one API call: `POST /v1/analyses` with a video and a question; a webhook (or `Prefer: wait`) delivers verdict + confidence + timestamped clips + an annotated evidence video.

## Realtime is real-time for voice, not for video

OpenAI's Realtime API is genuinely GA (the beta interface was removed May 12, 2026) and genuinely impressive — WebRTC/WebSocket/SIP voice agents that now accept **still images** in-session, priced at $5.00/1M image input tokens. But there is no video track and no continuous video-stream input; a client pushing periodic snapshots is a workaround, not a product. ([realtime guide](https://developers.openai.com/api/docs/guides/realtime); [pricing](https://developers.openai.com/api/docs/pricing); corroborated on [Microsoft Learn](https://learn.microsoft.com/en-us/answers/questions/2147073/))

Primate Vision's live lane is managed WebRTC at native frame rates, with mid-stream prompt changes over one WebSocket message and session caps the API tells you about up front (up to 1 hour per session on top plans).

## Verdicts: a contract vs. a conversation

Ask GPT-5.6 "did anyone enter the loading zone?" over a frame stack and you get prose — or, with Structured Outputs, JSON shaped like an answer. For exploratory questions that flexibility is wonderful. But the confidence number inside that JSON is generated text, not a calibrated score; the answer is stochastic; and OpenAI's own limitations list says the model "may generate incorrect descriptions or captions." ([docs limitations](https://developers.openai.com/api/docs/guides/images-vision))

It's not that GPT-5.6 is a weak model — it's that a language model sampling tokens is the wrong instrument for a measurement. Primate Vision's verdict is a different kind of object: an answer drawn from a **closed vocabulary** — `yes` / `no` / `indeterminate` — never free-form generation, computed by similarity scoring against the video rather than sampled from an LLM, with a calibrated 0–1 confidence, model version pinning (`darwin-1.3`) so behavior changes only when you choose, and a fully deterministic test mode for CI. And every yes ships with proof: timestamped clips plus an annotated overlay evidence video a human can watch and forward.

## Pricing: token math vs. a flat meter

**OpenAI bills per token, and video has no price because video has no product.** Frames are tokenized — GPT-5.x counts 32×32-pixel patches, so a 720p frame at high detail is ~920 input tokens — billed at the model's rate (gpt-5.6-terra: $2.00/1M input; gpt-5.6-sol: $5.00/1M). Our normalized estimate for DIY frame-by-frame analysis at 1fps: roughly **$1.84/camera-hour** (terra, low detail) up to **$6.62** (terra, high) or **$16.56** (sol, high) — *estimates*; they exclude output tokens, prompt overhead, and the chunking penalty, so real cost runs higher. ([pricing](https://developers.openai.com/api/docs/pricing); tokenization per the [vision guide](https://developers.openai.com/api/docs/guides/images-vision))

**Primate Vision has two lanes**, stated plainly:

1. **Metered — $0.01 per second of source video** ($0.60/min), flat and fps-independent. Queued time free; failed jobs free. A 30-second clip → verdict + confidence + timestamps + evidence video = $0.30. A `validate_only` dry-run estimates cost before you spend a credit.
2. **[Enterprise — contact us](https://primateintelligence.ai/pricing#enterprise)** for 24/7 continuous monitoring and camera fleets: dedicated capacity or on-site deployment at a small fraction of the metered rate, under highly discounted enterprise plans.

Normalized honestly: at 1fps stills, OpenAI's DIY route works out ~5–20× cheaper per raw hour (estimate) than $36/camera-hour† at Primate's metered rate — *if* your use case tolerates 1fps sampling, no temporal continuity, stochastic answers, and building the entire frame pipeline yourself. The meters buy different things: their tokens buy raw model reads of stills; our seconds buy answered questions about video, with evidence, at native frame rate, live or on demand. You don't buy camera-hours on our metered lane. You buy verdicts.

† *Metered rate normalized for comparison. Primate does not sell 24/7 continuous monitoring at the metered rate — continuous and fleet workloads use enterprise plans; [contact us](https://primateintelligence.ai/pricing#enterprise).*

## Choose OpenAI instead when…

These are real lanes, and they're wide ones:

- **Your input is stills or documents, not video.** Screenshots, photos, charts, receipts, PDFs-as-images — GPT-5.6's open-ended visual reasoning plus OCR is the best available, and Primate Vision doesn't do any of it (no OCR, no transcription, no open-ended description).
- **You need reasoning around the answer.** Explain, compare, extract, then call a tool and act — one model, one vendor. Primate answers one question about one video; it doesn't write your incident report.
- **Sparse sampling is genuinely fine.** One screenshot a minute of a screen recording, thumbnail triage, still-image moderation at scale — the Batch API at 50% off makes this very cheap (~$1.84/camera-hour at 1fps low detail, estimate).
- **You're already on the OpenAI / Azure / Bedrock stack** and want one vendor for text + vision + audio + agents, with enterprise controls (ZDR, data residency, spend limits) already built.
- **You want images inside a voice agent.** Realtime image input is GA with published pricing — a conversational agent that glances at a photo mid-call is a real, shipping OpenAI product.

## Choose Primate Vision when…

- **The input is actually video.** Files up to 2 GiB, URL ingest, or live WebRTC — no frame ripping, no chunking, no 25-minute ceiling. OpenAI has no video input path in any API.
- **The camera is live.** Managed WebRTC at native frame rates with mid-stream prompt changes, versus pushing snapshots into a voice-agent API as a workaround.
- **Motion is the point.** Falls, intrusions, safety events — things that happen between 1fps samples. Native-frame-rate analysis sees them; a frame stack doesn't.
- **You need an answer, not an essay.** A closed-vocabulary yes/no/indeterminate with calibrated confidence beats generated prose when the output feeds an alert, a workflow, or a compliance record.
- **You need proof.** Timestamped clips plus an annotated overlay evidence video — versus text that says it saw something.
- **You need to know cost and latency before you run it.** One flat meter plus a `validate_only` dry-run, and published 45ms p50 / 316ms p95 / 11.8 fps / 6.6s session setup. OpenAI publishes no vision latency numbers at all.

## Try it

**[Try it for free](https://primateintelligence.ai/signup)** — real processing, no card required to start.

**Your AI agent can do it for you — right from Claude.** Primate Vision ships an MCP server, llms.txt, markdown docs twins, and a sandbox key available in a single POST — your agent can get a key, upload a clip, and read back a verdict without a human touching a dashboard. (Credit where due: OpenAI's developer platform sets the bar here, and we built to it.)

For 24/7 monitoring and camera fleets: [see our enterprise plans](https://primateintelligence.ai/pricing#enterprise).

---

*Method note: every OpenAI claim traces to their live developer docs, pricing page, changelog, and deprecations page accessed 2026-07-31. Estimates are labeled.*


---

<!-- source: https://primateintelligence.ai/compare/roboflow-ultralytics.md -->

---
title: "Primate Vision vs. Roboflow / Ultralytics (2026): Detections vs. Verdicts"
slug: "roboflow-ultralytics"
author: "Matt Miesnieks"
date: "2026-07-31"
readTime: "6 min read"
tags: ["Comparisons","Computer Vision"]
excerpt: "Roboflow + Ultralytics is the best per-frame object-detection stack in the world — 1.7ms YOLO inference, twenty edge formats, pennies per camera-hour at scale. But detections aren't answers. Here's the honest head-to-head between perception and judgment."
canonical: https://primateintelligence.ai/compare/roboflow-ultralytics
---

The Roboflow + Ultralytics ecosystem is the best per-frame object-detection stack in the world. YOLO26 runs at **1.7–11.8ms per frame**, deploys to twenty edge formats, and at fleet scale costs pennies per camera-hour. If your problem is "count the vehicles" or "is a hard hat present in this frame," they win on speed and unit cost, full stop.

But their output is **detections, not answers**. Boxes and class confidences per frame, with no temporal understanding, no event semantics, and no verdicts. Primate Vision answers the question the frames add up to — "did the forklift enter the loading zone?" — with a **deterministic verdict** (yes / no / indeterminate), a calibrated 0–1 confidence, timestamped evidence segments, and an annotated evidence video, live over managed WebRTC or against an uploaded file.

*This is part of our [full 2026 video-AI landscape comparison](/compare/video-ai-landscape-2026).*

---

## Different layers of the same problem

Roboflow is an end-to-end CV platform: label data, train custom detectors (YOLO26, RF-DETR), deploy to cloud or edge, orchestrate in Workflows. Ultralytics maintains the YOLO family and monetizes via AGPL-3.0 copyleft → negotiated Enterprise licenses. Together they're the dominant "train-your-own-detector" ecosystem, with genuinely best-in-class docs and real industrial logos (BNSF, Pella).

What they sell is **perception**: where objects are, per frame, fast. What Primate Vision sells is **judgment**: whether the thing you asked about happened, with proof. Different layers of the same stack — and plenty of serious deployments should run both.

## The head-to-head

| Capability | Primate Vision | Roboflow / Ultralytics |
|---|---|---|
| Per-frame latency | ✅ 45ms p50 / 316ms p95, published | ✅ 1.7–11.8ms/frame (YOLO26 on T4 TensorRT10) — best in class ([YOLO26](https://blog.roboflow.com/yolo26/)) |
| Live streams | ✅ managed WebRTC | ✅ Serverless Video Streams + self-hosted RTSP — objects only |
| Temporal / event understanding | ✅ "did X happen," timestamped segments | ❌ per-frame detections; aggregation logic is yours to build in Workflows |
| Open-vocab prompting | ✅ full questions, live or stored | ⚠️ object *nouns* only (YOLO-World, YOLOE-26, SAM 3) — behavioral/temporal queries still need training or a bolted-on third-party VLM |
| Verdict contract | ✅ deterministic yes/no/indeterminate + calibrated 0–1 confidence | ❌ boxes + class confidences; no question→answer abstraction |
| Evidence artifacts | ✅ annotated overlay evidence video + timestamped clips | ⚠️ DIY overlays/notifications via Workflow blocks |
| Bounding-box JSON | ❌ overlay video only | ✅ core competency — plus pose, masks, and metric depth now |
| Edge / air-gap | ❌ cloud only | ✅ their strongest card: 20 export formats, on-device, air-gapped Enterprise |
| Determinism | ✅ deterministic verdict path, model pinning, deterministic test mode | ✅ fixed trained weights — genuinely deterministic detections |
| Training required | ✅ none — ask in plain English | ⚠️ custom classes still mean dataset + labeling + training (open-vocab nouns excepted, at reduced accuracy) |
| Compliance | ⚠️ published targets; Enterprise SLA; no SOC 2 yet | ✅ SOC 2 on all Roboflow plans — ahead of us here |

One fairness note I insist on, because the stale version of this claim is everywhere: it is **no longer true** that new categories always require retraining in their stack. YOLO-World, YOLOE-26 (January 2026), and SAM 3 give text-prompted open-vocabulary detection at inference time. The accurate 2026 statement is narrower: open-vocab covers *object nouns only*, at reduced accuracy versus custom-trained models — and behavioral, temporal, or compositional questions ("employee skipped the handwash step," "person fell and didn't get up") still require training a model or bolting a per-frame VLM block onto a Workflow, which forfeits the latency and cost advantages that made you pick YOLO in the first place, and still returns no verdict.

## The gap between detections and answers

A detector tells you `person: 0.94, forklift: 0.91` sixty times a second. Whether the person was *in the forklift's path*, whether the event *happened during this shift*, whether the answer is *yes, no, or genuinely indeterminate* — all of that is aggregation logic you write, test, and maintain yourself. Every CV team I've talked to has a pile of this glue code, and nobody loves it. Roboflow's Workflows make that engineering nicer. They don't make it disappear.

Primate Vision's contract starts where detections end: one question in, one deterministic verdict out — a closed vocabulary computed by similarity scoring against the video (not sampled from a language model), a calibrated confidence, timestamped clips, and an annotated evidence video a human can review. Model versions are pinnable (`darwin-1.3`); test mode is fully deterministic for CI.

## Pricing: credits vs. one flat meter

**Roboflow bills subscriptions plus a universal credit meter.** Core is $79/mo (annual) with 50 credits; extra prepaid credits from $4, overage "Flex" credits at $6 — 50% more. Everything burns credits at published rates ([credits page](https://roboflow.com/credits)), including — notably — **inference you run on your own hardware**: "Running models that live on the Roboflow platform, whether run locally (self-hosted) or in the Roboflow Cloud, will consume credits." And on the Ultralytics side, the $29/mo Pro seat does **not** include a commercial YOLO license — commercial use of AGPL YOLO models requires a negotiated, unpublished annual Enterprise fee ([Ultralytics pricing](https://www.ultralytics.com/pricing)).

Normalized (estimates): managed Serverless Video Streams run **$3.00–$8.00/camera-hour** depending on GPU size; a dedicated GPU deployment amortized across many YOLO streams can reach **~$0.08–$0.40/camera-hour** at scale.

**Primate Vision has two lanes:**

1. **Metered — $0.01 per second of source video** ($0.60/min), flat, fps-independent, no subscriptions, no credit arithmetic, no license negotiations. Queued time free; failed jobs free. A 30-second clip → verdict + confidence + timestamps + evidence video = $0.30.
2. **[Enterprise — contact us](https://primateintelligence.ai/pricing#enterprise)** for 24/7 continuous monitoring and camera fleets: dedicated capacity or on-site deployment at a small fraction of the metered rate, under highly discounted enterprise plans — the correct comparison against a dedicated-GPU fleet deployment.

At the metered rate we normalize to $36/camera-hour† — 4.5–12× their managed streams and orders of magnitude above their amortized fleet numbers. But the rows price different goods. Theirs is raw per-frame detection, with the judgment layer coming out of your engineering budget. Ours is the answered question with evidence. Buy verdicts on the metered lane; buy fleets on the enterprise lane.

† *Metered rate normalized for comparison. Primate does not sell 24/7 continuous monitoring at the metered rate — continuous and fleet workloads use enterprise plans; [contact us](https://primateintelligence.ai/pricing#enterprise).*

## Choose Roboflow / Ultralytics instead when…

Often, honestly:

- **Fixed, known object taxonomy at high volume and low latency.** Manufacturing defects, PPE presence, vehicle counting at line speed — train once, run for ~$0.10–$4/camera-hour. At 1.7ms/frame, YOLO is the only viable class for line-speed industrial inspection; our 45ms p50 doesn't compete there.
- **Edge or air-gapped deployment.** On-device TFLite/CoreML/TensorRT, no cloud allowed — we're cloud-only.
- **You need pixel-accurate localization**: boxes, masks, pose, oriented boxes, now metric depth. We return overlay video, not spatial JSON.
- **You have labeled data and CV engineers**, and cost-per-frame is the dominant constraint.
- **SOC 2 on every tier matters today** — they have it; we don't yet.

## Choose Primate Vision when…

- **Your question is about events, not objects.** "Did anyone climb the fence" is not a class label; it's a temporal judgment with an evidence requirement.
- **You don't have a dataset, labelers, or a training pipeline** — and don't want to acquire them to answer one question. Plain English, first verdict in minutes via a sandbox key in one POST.
- **The output feeds an alert or audit trail**: deterministic verdict + calibrated confidence + watchable evidence, in one JSON contract with webhooks or `Prefer: wait` — versus detection streams plus aggregation code you own forever.
- **You want one legible meter.** $0.01/second, flat — no credit tiers, no Flex overage, no metering on your own hardware, no AGPL license negotiation.
- **The taxonomy changes weekly.** New question = new prompt (changeable mid-stream over one WebSocket message), not a new training run.

## Try it

**[Try it for free](https://primateintelligence.ai/signup)** — real processing, no card required to start.

**Your AI agent can do it for you — right from Claude.** Primate Vision ships an MCP server, llms.txt, markdown docs twins, and a sandbox key available in a single POST — your agent can get a key, upload a clip, and read back a verdict without a human touching a dashboard.

For 24/7 monitoring and camera fleets: [see our enterprise plans](https://primateintelligence.ai/pricing#enterprise).

---

*Method note: every Roboflow/Ultralytics claim traces to their pricing, credits, licensing, and blog pages accessed 2026-07-31. Estimates are labeled; community-reported unverified pricing anecdotes are deliberately excluded.*


---

<!-- source: https://primateintelligence.ai/compare/sieve.md -->

---
title: "Primate Vision vs. Sieve (2026): The Video API That Quietly Left the Market"
slug: "sieve"
author: "Matt Miesnieks"
date: "2026-07-31"
readTime: "6 min read"
tags: ["Comparisons","Market Analysis"]
excerpt: "If you're comparing us against Sieve's video-processing API in 2026, there's news you may have missed: that API no longer exists. Sieve pivoted to selling training data to frontier labs. Here's what happened, what they sell now, and what to use instead."
canonical: https://primateintelligence.ai/compare/sieve
---

If you arrived here comparing Primate Vision against Sieve's video-processing API, I have to start with news you may have missed: **that API no longer exists.**

Sometime between January and July 2026, Sieve (YC W22, formerly sievedata.com) pivoted out of the developer video-AI business entirely. Their site now redirects to [sieve.ai](https://www.sieve.ai/) — "The multimodal data lab" — a company that sells training datasets and environments to frontier AI labs, not APIs to developers. The docs site, the API endpoints, and the function-execution platform are gone from DNS. (Checks run 2026-07-31; the last archived capture of the docs host dates to 2022.)

What Sieve does now is genuinely impressive — embedding a billion videos to curate exabyte-scale training data is real engineering — but it's a different business, sold to a different buyer. If you're a model-training team shopping for licensed video data, Sieve is worth your call and I say so below. If you're a developer who needs to *analyze* video — ask a question of a stream or a file and get an answer you can act on — Sieve is no longer an option at any price. Primate Vision is the managed API built for exactly that.

*This is part of our [full 2026 video-AI landscape comparison](/compare/video-ai-landscape-2026).*

---

## A category exit, not a comparison

Most posts in this library weigh two live products against each other. This one documents a departure.

As of 2026-07-31, `sievedata.com` auto-redirects to **sieve.ai**, which describes the company as building "the data and environments frontier AI labs use to train the next generation of multimodal systems" — datasets and custom collection for teams working on "generative media, robotics, computer use, world models, and agentic systems." ([sieve.ai](https://www.sieve.ai/), [about](https://www.sieve.ai/about)) The buyer is a research or training team at an AI lab, engaged through purchase agreements — not a developer with an API key.

The developer platform that used to compete in our category — video ingestion APIs, a marketplace of GPU-backed processing functions, dubbing, transcription, key-moment extraction, a Python SDK — has been decommissioned from the public internet. `docs.sievedata.com`, `api.sievedata.com`, and `mango.sievedata.com` no longer resolve in DNS. We could find **no shutdown announcement, no deprecation notice, and no public migration path** for existing API customers — labeled explicitly as unverified-by-absence, since a private grandfathered arrangement could exist. The pivot was visible on their homepage as early as January 2026, when a Wayback snapshot shows "Video datasets for frontier AI… 500K hours of high quality, diverse video clips" with a request-samples → purchase-agreement → bucket-delivery sales motion. ([Wayback, 2026-01-12](http://web.archive.org/web/20260112044936/https://www.sievedata.com/))

So this post answers the two questions people actually arrive with: *what does Sieve sell now, and who should buy it* — and *what should you use instead if you needed the thing they stopped selling.*

## The head-to-head

Scored against Sieve's **live** product — a dataset business. That's the only fair way to score it; legacy API capabilities from our earlier audit could not be re-verified and are treated as gone.

| Capability | Primate Vision | Sieve (sieve.ai, live product) |
|---|---|---|
| Video file upload API | ✅ MP4/MOV up to 2 GiB, presigned or URL ingest | ❌ no public upload API; legacy ingestion endpoints gone from DNS |
| Live stream input | ✅ managed WebRTC, native frame rate | ❌ never offered pre-pivot; nothing now |
| Real-time output | ✅ 45ms p50 / 316ms p95, 11.8 fps sustained, published | ❌ N/A — data delivered "within 1-2 days via storage bucket access" ([Wayback, Jan 2026](http://web.archive.org/web/20260112044936/https://www.sievedata.com/)) |
| Open-vocab NL prompting | ✅ core product | ❌ no public prompting surface |
| Verdict contract | ✅ deterministic yes/no/indeterminate + calibrated 0–1 confidence | ❌ N/A — no inference product |
| Timestamped segments | ✅ included with every analysis | ⚠️ only as annotations *inside purchased datasets* ("action metadata," time-synced shapes) |
| Evidence artifacts | ✅ annotated overlay evidence video + timestamped clips | ❌ N/A |
| Embeddings / semantic search | ❌ none | ⚠️ internal capability at extraordinary scale (1B videos, 41B vectors) — used for their own curation, **not sold as an API** ([Sieve blog, 2026-07-24](https://www.sieve.ai/blog)) |
| Webhooks / async jobs | ✅ Standard Webhooks signing, retries, redelivery + `Prefer: wait` | ❌ legacy job/webhook model unverifiable; API hosts dead |
| SDKs / docs | ✅ OpenAPI 3.1 on prod, npm + PyPI SDKs, public examples | ❌ no public docs site remains |
| Training-data supply at exabyte scale | ❌ not our business | ✅ **their entire business** — and they appear very good at it |

## What happened to the developer platform

Our April 2026 audit described a two-generation Sieve API: a V1 video-database API and a V2 function-execution platform with a marketplace of GPU-backed functions (dubbing, transcription, editing building blocks) and developer-friendly ergonomics. Honesty requires two admissions. First, **none of that is re-verifiable today** — every claim about it is stale, because the docs and API hosts are simply gone. Second, our April audit was likely already partially stale when written: the January 2026 Wayback capture shows the dataset pivot was already the homepage story months earlier.

Here's the part that matters if you're evaluating vendors in this category: we found no public deprecation notice or migration path for the platform's customers. Whether existing API users got a private wind-down is unverified. But if you build on video-AI infrastructure, "the vendor's docs site can vanish from DNS without a public announcement" is exactly the platform risk you're pricing in. It's fair to ask any vendor — including us — how they'd handle an exit. Our answer: a versioned API with model pinning (`darwin-1.3`), a published OpenAPI 3.1 contract served from the API itself, and published service expectations — the machinery that makes commitments auditable rather than vibes.

## What Sieve sells now — and it's genuinely good

No sandbagging: the new Sieve is doing hard things well. Their July 2026 engineering write-up describes embedding **one billion videos** and querying **41 billion vectors** to curate training data — frame deduplication, GPU selection, cost tradeoffs at a scale far beyond anything Primate Intelligence operates. ([Sieve blog](https://www.sieve.ai/blog)) They advertise custom data collection ("targeted real-world, digital, and simulated workflows"), dense annotation ("captions, transcripts, object labels, action metadata, camera signals, UI events, and custom schemas"), and compliance-first sourcing ("filtering, licensing, consent, retention, and permission requirements"), with SOC 2 Type 2 controls claimed on the homepage (certification status unverified — no public audit report or trust center). ([sieve.ai](https://www.sieve.ai/))

That last capability — licensing and consent management for training data at petabyte scale — is genuinely hard, and it positions Sieve upstream of the entire model-building world. They sell to the labs that build models like ours. As a *competitor for Primate Vision customers*, though, the honest assessment stands: **none anymore.** They no longer sell to our buyer.

## Pricing: one meter vs. no meter

**Sieve publishes no pricing at all.** There's no pricing page on the live site (the old `sievedata.com/pricing` URL 404s), and the sales motion is enterprise-style: request samples → enter a purchase agreement based on dataset volume and characteristics → receive data via storage bucket access. There is no processing service to normalize into $/minute or $/camera-hour — the honest table entry is "N/A — exited category."

**Primate Vision has two lanes**, stated plainly:

1. **Metered — $0.01 per second of source video** ($0.60/min), flat and fps-independent. Queued time free; failed jobs free. A 30-second clip → verdict + confidence + timestamped clips + annotated evidence video = $0.30. A `validate_only` dry-run estimates cost before you spend a credit.
2. **[Enterprise — contact us](https://primateintelligence.ai/pricing#enterprise)** for 24/7 continuous monitoring and camera fleets: dedicated capacity or on-site deployment at a small fraction of the metered rate, under highly discounted enterprise plans.

For cross-library consistency: the metered lane normalizes to $36/camera-hour†. With Sieve there's nothing to compare it against — but if you're weighing Primate against vendors still in the category, the full normalized table lives in the [hub post](/compare/video-ai-landscape-2026).

† *Metered rate normalized for comparison. Primate does not sell 24/7 continuous monitoring at the metered rate — continuous and fleet workloads use enterprise plans; [contact us](https://primateintelligence.ai/pricing#enterprise).*

## Choose Sieve instead when…

Different market, real strengths — if you're their buyer, call them:

- **You're a model-training team buying licensed video/multimodal data.** Video generation, world models, robotics, computer-use agents — packaged datasets and custom collection with dense annotations is precisely what Sieve now builds, and their scale claims (500K-hour curated suites, petabytes of source video) are aimed squarely at you.
- **You need custom data collection with licensing and consent guarantees.** Real-world, simulated, and UI-interaction workflows collected to your spec, with the compliance machinery handled — a capability Primate Intelligence doesn't have and wouldn't replicate quickly.
- **You need a partner to curate petabyte-scale video into training-ready subsets.** Their embedding-a-billion-videos pipeline exists exactly for this.

And what Sieve can no longer do for you: analyze video in an application. If you were a legacy Sieve API customer — dubbing, transcription, function pipelines — there is no public successor product. For speech, dubbing, and audio workloads, look at transcription vendors or the big-cloud video indexers; Primate Vision is **not** a fit there either (we're visual-only, no audio path). For visual analysis — detection, monitoring, verdict-grade Q&A — read on.

## Choose Primate Vision when…

- **You need to ask a question of video and get an answer, managed, today.** One API call — upload up to 2 GiB or point us at a public https URL — returns a verdict from a closed vocabulary (`yes` / `no` / `indeterminate`), a calibrated 0–1 confidence score, and timestamped evidence. Not pipeline building blocks you assemble; an answer.
- **The camera is live.** Managed WebRTC ingest at native frame rates, with mid-stream prompt changes over one WebSocket message. Sieve never offered live input even pre-pivot; now there's no input at all.
- **You need proof, not just output.** Every yes ships with timestamped clips plus an annotated overlay evidence video a human can watch and forward.
- **Latency is a requirement, not a hope.** 45ms p50 / 316ms p95 per-frame inference, 11.8 fps sustained — published on a public [/performance](https://www.primateintelligence.ai/performance) page. Sieve's live product delivers data in 1–2 days by design; it's a different physics.
- **You want a vendor whose commitments are machine-auditable.** OpenAPI 3.1 served from the API, pinned model versions, webhooks with Standard-Webhooks signing and redelivery, a deterministic test mode for CI, and pricing you can compute before you run (`validate_only`).

## Try it

**[Try it for free](https://primateintelligence.ai/signup)** — real processing, no card required to start.

**Your AI agent can do it for you — right from Claude.** Primate Vision ships an MCP server, llms.txt, markdown docs twins, and a sandbox key available in a single POST — your agent can get a key, upload a clip, and read back a verdict without a human touching a dashboard.

For 24/7 monitoring and camera fleets: [see our enterprise plans](https://primateintelligence.ai/pricing#enterprise).

---

*Method note: every Sieve claim traces to their live site (sieve.ai), the January 2026 Wayback snapshot of sievedata.com, their 2026-07-24 blog post, and DNS checks — all accessed 2026-07-31. Items we could not verify — including any private continuation of the legacy API and the shutdown timeline — are labeled as unverified. Estimates are labeled.*


---

<!-- source: https://primateintelligence.ai/compare/twelve-labs.md -->

---
title: "Primate Vision vs. Twelve Labs (2026): Real-Time Verdicts vs. Archive Search"
slug: "twelve-labs"
author: "Matt Miesnieks"
date: "2026-07-31"
readTime: "6 min read"
tags: ["Comparisons","Video Search"]
excerpt: "Twelve Labs leads semantic search over stored video archives. Primate Vision watches video live and returns deterministic verdicts with evidence. These are genuinely different products — here's the honest head-to-head, with receipts."
canonical: https://primateintelligence.ai/compare/twelve-labs
---

People compare us to Twelve Labs constantly, and the comparison is mostly a category error. Twelve Labs is the best-funded pure-play in video understanding — a $100M Series B closed July 2026 — and the category leader in **semantic search over stored video archives**. Primate Vision is the only managed API that watches video **live, at native frame rates**, and returns a **deterministic verdict** — yes / no / indeterminate — with a calibrated confidence score and evidence you can watch.

Different products, answering different questions. If your videos are already recorded and your problem is *finding moments*, choose Twelve Labs. If your problem is *knowing whether something is happening* — on a camera, right now, with an answer you can act on — Twelve Labs cannot do it at all. Primate Vision was built for exactly that.

*This is part of our [full 2026 video-AI landscape comparison](/compare/video-ai-landscape-2026).*

---

## Two different questions

Every product in this category is secretly an answer to one question. Twelve Labs answers: *"Where in my 10,000-hour archive does X appear?"* You upload videos into indexes (Marengo 3.0 embeddings), then search, summarize, or extract structured metadata (Pegasus 1.5) across the corpus — in natural language, across 36 languages, spanning visuals, speech, on-screen text, and audio. That's a real category, and they lead it. ([TwelveLabs docs](https://docs.twelvelabs.io/v1.3/docs/get-started/introduction))

Primate Vision answers: *"Is X happening — and can you prove it?"* Point it at a live stream (managed WebRTC) or a file (upload or URL ingest, up to 2 GiB), ask in plain English, and get a deterministic verdict (`yes` / `no` / `indeterminate`), a 0–1 confidence score, timestamped evidence segments, and an annotated overlay evidence video — at 45ms p50 per-frame inference, published on a public [/performance](https://www.primateintelligence.ai/performance) page.

## The head-to-head

| Capability | Primate Vision | Twelve Labs |
|---|---|---|
| Live stream input | ✅ managed WebRTC, native frame rate | ❌ none — no WebRTC/RTSP/gRPC/WebSocket anywhere in docs ([upload methods](https://docs.twelvelabs.io/v1.3/docs/concepts/upload-methods.md)) |
| Published latency | ✅ 45ms p50 / 316ms p95, 11.8 fps sustained | ❌ no latency numbers published anywhere |
| Verdict contract | ✅ deterministic yes/no/indeterminate + calibrated 0–1 confidence | ⚠️ generative text or JSON-schema output — temperature-tunable, no calibration, no reproducibility guarantee |
| Timestamped segments | ✅ | ✅ start/end per segment; strong |
| Evidence artifacts | ✅ annotated overlay evidence video | ⚠️ thumbnails only; no rendered clips or overlays |
| Semantic search / embeddings | ❌ none | ✅ flagship — 512-dim Marengo embeddings, vector-DB ecosystem |
| Transcript / OCR / audio | ❌ visual only | ✅ core strength: speech search, OCR, logos, music |
| Multi-video batch | ❌ (batch = up to 10 prompts on one video, 50% off after the first) | ✅ up to 1,000 videos per Analyze call |
| Webhooks | ✅ Standard Webhooks signing, retries, redelivery + `Prefer: wait` | ✅ indexing notifications, dashboard-managed |
| Agent accessibility | ✅ llms.txt, markdown docs twins, MCP server, sandbox key in one POST | ✅ best-in-class: llms.txt, .md docs, docs MCP server, Claude Code plugin |

## The real-time gap is total, not partial

There is no live ingestion of any kind in Twelve Labs' documentation. No WebRTC, no RTSP, no streaming input path. Their "real-time" story is two things: token-by-token *text delivery* on sync Analyze (the video must be fully uploaded first), and a partner integration where VideoDB does the streaming and Pegasus does the analysis afterward. That's not real-time video analysis. That's fast typing.

And even for stored video, there's index-first friction: since July 2026 every upload is async — you poll each asset to `ready` before you can ask anything. Ask-one-question-of-one-clip is a multi-step pipeline. On Primate Vision it's one API call with `Prefer: wait`, or a webhook when it's done.

## Verdicts: deterministic vs. generative

When a Twelve Labs answer matters — "did the forklift enter the loading zone?" — you get generated text, or generated JSON shaped by a schema you define. The JSON-schema path (GA on Pegasus) genuinely gets close to a verdict shape. But it's still a generative approximation: temperature-tunable, uncalibrated, with no reproducibility guarantee in their docs. It's not that their model is worse — it's that a language model is the wrong instrument for a measurement.

Primate Vision's verdict is a different kind of object: a **deterministic answer** from a closed vocabulary — `yes` / `no` / `indeterminate` — never free-form generation, with a calibrated 0–1 confidence score, model version pinning (`darwin-1.3`), and a fully deterministic test mode for CI. And every yes ships with proof: timestamped clips plus an annotated overlay evidence video a human can watch and forward.

## Pricing: two very different meters

**Twelve Labs stacks meters.** One-time indexing at $0.042/min, **plus a perpetual infrastructure fee of $0.0015 per indexed minute per month** — a 10,000-hour archive costs $900/month just to stay searchable, before a single query — plus $4 per 1,000 searches, plus Pegasus analysis at $0.0292/min input and $0.0075/1k output tokens. And Segment multiplies: their own FAQ example shows a 60-minute video with 4 segment definitions billing **240 minutes**. ([Twelve Labs pricing](https://www.twelvelabs.io/pricing))

**Primate Vision has two lanes**, stated plainly:

1. **Metered — $0.01 per second of source video** ($0.60/min), flat and fps-independent. Queued time free; failed jobs free. A 30-second clip → verdict + confidence + timestamps + evidence video = $0.30.
2. **[Enterprise — contact us](https://primateintelligence.ai/pricing#enterprise)** for 24/7 continuous monitoring and camera fleets: dedicated capacity or on-site deployment at a small fraction of the metered rate, under highly discounted enterprise plans.

The honest normalization: on recorded footage, Twelve Labs works out to roughly $1.75–$2.52 per camera-hour (estimate, batch-upload workaround — they have no live path at any price), versus $36/camera-hour† at Primate's metered rate. That's ~14–20× cheaper per raw hour of archive. If archive search is your workload, that gap is real and you should weigh it. But the meters buy different things: theirs buys searchability; ours buys answered questions with evidence, live or on demand. You don't buy camera-hours on our metered lane. You buy verdicts.

† *Metered rate normalized for comparison. Primate does not sell 24/7 continuous monitoring at the metered rate — continuous and fleet workloads use enterprise plans; [contact us](https://primateintelligence.ai/pricing#enterprise).*

## Choose Twelve Labs instead when…

We'd rather you pick the right tool than the wrong one of ours:

- **You have a large stored archive and need to find moments in it.** Media libraries, sports footage, ad inventory, recorded bodycam/CCTV. Primate Vision has no search, no embeddings, no corpus features — this is Twelve Labs' home turf and they're the leader.
- **You need speech, on-screen text, or audio understanding.** Interviews, broadcasts, lyrics, logos. Primate Vision is visual-only: no transcription, no OCR, no audio path.
- **You're building a RAG/vector pipeline over video.** Marengo embeddings plus a dozen vector-DB partner integrations vs. nothing on our side.
- **You need bulk structured extraction over recorded content** — chapters, scene metadata, custom JSON schemas — at low per-minute cost, up to 1,000 videos per batch call. Primate Vision doesn't batch across videos at all.

## Choose Primate Vision when…

- **The camera is live.** Twelve Labs cannot ingest a stream at any price. Primate Vision is managed WebRTC at native frame rates, with mid-stream prompt changes over one WebSocket message.
- **You need an answer, not a reading list.** A deterministic yes/no/indeterminate with calibrated confidence beats generated prose when the output feeds an alert, a workflow, or a compliance record.
- **You need proof.** An annotated overlay evidence video plus timestamped clips — versus thumbnails.
- **You need to know what it costs before you run it.** One flat meter (and a `validate_only` dry-run that estimates cost without touching GPU or credits) versus five stacking meters and a segment multiplier.
- **Latency is a requirement, not a hope.** We publish 45ms p50 / 316ms p95 / 11.8 fps / 6.6s session setup. They publish nothing.

## Try it

**[Try it for free](https://primateintelligence.ai/signup)** — real processing, no card required to start.

**Your AI agent can do it for you — right from Claude.** Primate Vision ships an MCP server, llms.txt, markdown docs twins, and a sandbox key available in a single POST — your agent can get a key, upload a clip, and read back a verdict without a human touching a dashboard.

For 24/7 monitoring and camera fleets: [see our enterprise plans](https://primateintelligence.ai/pricing#enterprise).

---

*Method note: every Twelve Labs claim traces to their live docs, pricing page, and release notes accessed 2026-07-31. Estimates are labeled.*


---

<!-- source: https://primateintelligence.ai/compare/video-ai-landscape-2026.md -->

---
title: "Primate Vision vs. the Video-AI Landscape (2026)"
slug: "video-ai-landscape-2026"
author: "Matt Miesnieks"
date: "2026-07-31"
readTime: "14 min read"
tags: ["Comparisons","Market Analysis"]
excerpt: "In 2026 the managed real-time video-analysis lane emptied out. Here's the full map of what's left — batch indexers, 1fps token meters, and self-hosted stacks — with receipts, honest pricing math, and who should choose whom."
canonical: https://primateintelligence.ai/compare/video-ai-landscape-2026
---

Something strange happened to video AI in 2026, and as far as I can tell nobody has written it down: the managed real-time lane emptied out. AWS closed Rekognition streaming to new customers. Microsoft retired its live-camera analysis product. Google's streaming video API has been in Beta since 2019, with release notes frozen since 2021. Sieve left the category entirely. What's left is batch indexers, token-metered LLMs sampling one frame per second, and self-hosted build-it-yourself stacks.

Which leaves a claim I'll spend this post earning: **Primate Vision is the only managed, open-vocabulary, real-time video-analysis API operating at native frame rates.** You ask a question in plain English. You get back a deterministic verdict — yes / no / indeterminate — with a 0–1 confidence score, timestamped evidence segments, and an annotated overlay clip, at 45ms p50 per-frame inference. Here's the full map, with receipts.

---

## The market-structure story: the real-time lane emptied

If you tried to buy managed real-time video analysis in 2024, you had options. In 2026, one by one, the vendors walked away:

- **AWS Rekognition Streaming Video Events closed to new customers effective April 30, 2026.** A new customer cannot buy real-time video analysis from Rekognition at all. AWS's own published migration path: snapshot frames and call the *image* API via Lambda. ([AWS availability changes](https://docs.aws.amazon.com/rekognition/latest/dg/rekognition-availability-changes.html)) Even grandfathered accounts get only "people, packages and pets," max 120 seconds per motion event — not continuous monitoring. ([AWS docs](https://docs.aws.amazon.com/rekognition/latest/dg/streaming-video-detect-labels.html))
- **Microsoft retired Azure AI Vision Spatial Analysis** (live-camera people counting and zone events) **on March 30, 2025**. ([Microsoft Q&A](https://learn.microsoft.com/en-us/answers/questions/2151867/)) What remains — Azure Video Indexer — is an upload-and-index product; there is no live-stream API in 2026, despite legacy "stored or streaming" marketing on the pricing page. ([Azure VI pricing](https://azure.microsoft.com/en-us/pricing/details/video-indexer/), [FAQ](https://learn.microsoft.com/en-us/azure/azure-video-indexer/faq))
- **Google Video Intelligence is frozen in maintenance mode** — its release notes have no entries after November 1, 2021. ([release notes](https://docs.cloud.google.com/video-intelligence/docs/release-notes)) Its streaming mode has been Beta since 2019, is excluded from the SLA, and live protocols require compiling and operating Google's AIStreamer C++/GStreamer proxy yourself. ([live-streaming docs](https://docs.cloud.google.com/video-intelligence/docs/streaming/live-streaming))
- **[Sieve](/compare/sieve) exited the developer video-AI API business entirely.** sievedata.com now redirects to sieve.ai — "The multimodal data lab" — selling training datasets to frontier labs. The docs, api, and dashboard subdomains no longer resolve in DNS. Sieve is excluded from the comparison below because there is no processing product left to compare. (DNS + site checks, 2026-07-31.)
- **[OpenAI](/compare/openai) is exiting video APIs, not entering.** No OpenAI API accepts video input — not Responses, not Chat Completions, not Realtime ([vision docs](https://developers.openai.com/api/docs/guides/images-vision); [open SDK request](https://github.com/openai/openai-node/issues/1778)) — and the Sora 2 Videos API was deprecated March 24, 2026 with shutdown September 24, 2026. ([deprecations](https://developers.openai.com/api/docs/deprecations))

Every qualifier in our claim is load-bearing, and I want to show the work: **managed** (NVIDIA VSS is genuinely real-time, but it's a self-hosted blueprint — NVIDIA's docs instruct you to provide your own authentication, TLS termination, and network isolation ([NVIDIA VSS docs](https://docs.nvidia.com/vss/latest/))); **open-vocabulary** (Roboflow sells managed real-time streams, but for fixed object nouns per frame — no temporal events, no verdicts); **native frame rates** (Gemini Live is managed and low-latency, but its video input is spec-capped at "images (JPEG <= 1FPS)" ([Gemini Live docs](https://ai.google.dev/gemini-api/docs/live-api)) — Google's own docs admit 1fps sampling misses fast action and recommend "slowing down such clips" as the workaround ([video understanding docs](https://ai.google.dev/gemini-api/docs/video-understanding))).

One more thing nobody else does: **publish latency.** This one genuinely surprised me. None of Twelve Labs, Google Video Intelligence, Gemini, AWS Rekognition, Azure Video Indexer, or OpenAI publishes end-to-end latency numbers for video analysis. Primate publishes 45ms p50 / 316ms p95 per-frame inference, 11.8 fps sustained, and 6.6s session setup on a public [/performance](https://www.primateintelligence.ai/performance) page. (All vendor sources accessed 2026-07-31.)

---

## What Primate Vision actually is

Primate Vision answers questions about video. That's the whole product. You point it at a stream (managed WebRTC) or a file (upload or URL ingest, up to 2 GiB), ask a question in plain English — "Is anyone climbing the fence?" "Did the forklift enter the loading zone?" — and get:

- A **deterministic verdict**: `yes` / `no` / `indeterminate` — a closed vocabulary, never free-form generation
- A **calibrated confidence score** (0–1)
- **Timestamped evidence segments** showing exactly when
- An **annotated overlay evidence video** you can watch and share

Pricing is flat and legible: **$0.01 per second of source video** ($0.60/min), independent of frame rate — 10 seconds at 60fps bills as 10 seconds. Queued time is free. Failed jobs are free.

**And what it isn't.** A comparison post you can't audit is worthless, so here's ours: Primate Vision does **not** return bounding-box JSON (evidence is the overlay video, not coordinates), does **not** offer embeddings or semantic search over archives, does **not** do transcription, OCR, or audio understanding (it's visual-only), and does **not** batch-process multiple videos in one request (batch means up to 10 questions against one video, at 50% off each question after the first). If those are your primary needs, some of the products below are genuinely better choices — we say which ones.

---

## The landscape in three tiers

Analyst charts classify by marketing category. I'd rather classify by what you can actually buy and run today:

### Tier 1 — Managed, real-time, open-vocabulary, native frame rate

| Who | Why |
|---|---|
| **Primate Vision** | Managed WebRTC ingest, 45ms p50 per-frame inference, open-vocab question → verdict + timestamped evidence. The only entrant meeting all the criteria. |

### Tier 2 — Near-real-time, constrained, or build-it-yourself

| Who | The constraint |
|---|---|
| **[Gemini Live API](/compare/gemini)** | Managed and conversational, but video input is hard-capped at ≤1fps JPEG stills and audio+video sessions are limited to 2 minutes without context compression ([session docs](https://ai.google.dev/gemini-api/docs/live-api/session-management)). Real-time *conversation*, not real-time *video analysis*. And the meter compounds: Gemini Live re-bills the entire accumulated session context on every turn — "As a session lengthens, the cost per turn increases" ([Google's best-practices docs](https://ai.google.dev/gemini-api/docs/live-api/best-practices)). |
| **[NVIDIA VSS blueprint](/compare/nvidia-vss)** | Genuinely real-time (RTSP first-class, GA), but 100% self-hosted: you supply GPUs, auth, TLS, Elasticsearch, and ops. A project, not a product. Its own release notes document that captions containing "yes"/"true" create spurious incident records ([release notes](https://docs.nvidia.com/vss/latest/release-notes.html)). |
| **[Roboflow / Ultralytics](/compare/roboflow-ultralytics)** | Managed streams with millisecond YOLO inference — best-in-class per-frame *object detection*. But open-vocab covers object nouns only; no temporal events, no verdicts, no "did X happen." |
| **[Google VI streaming](/compare/google-video-intelligence)** | Beta since 2019, SLA-excluded, DIY C++ proxy for live protocols. Effectively abandoned. |
| **[AWS Rekognition streaming](/compare/aws-rekognition)** | Closed to new customers; grandfathered accounts get people/pets/packages, ≤120s per event. |
| **[Self-hosted open VLMs](/compare/open-vlms)** (Qwen3-VL, InternVL, Moondream…) | Per-frame real-time is achievable and raw compute is very cheap — but no model ships a streaming layer. WebRTC/RTSP ingest, session state, structured output, and latency proof are all yours to build and operate. |
| **[OpenAI Realtime](/compare/openai)** | GA and genuinely real-time — for audio and still images. No video track exists. |

### Tier 3 — Batch / async only

| Who | The model |
|---|---|
| **[Twelve Labs](/compare/twelve-labs)** | Upload → index → query. The best-funded pure-play in video understanding ($100M Series B, July 2026 — NEA + NAVER co-led, Amazon participating ([announcement](https://www.globenewswire.com/news-release/2026/07/01/3320545/0/en/))), with excellent semantic search and the best agent-facing docs in the category. But there is no live ingestion of any kind in its docs — no WebRTC, RTSP, gRPC, or WebSocket input ([upload methods](https://docs.twelvelabs.io/v1.3/docs/concepts/upload-methods.md)). "Real-time" in their story means token-by-token text streaming and a partner integration. |
| **[Azure Video Indexer](/compare/azure-video-indexer)** | Upload-and-index, rich audio+visual insights (transcription, OCR, faces). The strongest choice if your workload is *media archive enrichment* rather than *visual question answering*. |
| **[AWS Bedrock Nova](/compare/aws-rekognition)** | AWS's designated forward path for video Q&A is explicitly offline: file/S3 only, all video resized to 672×672 "with distortion," 1fps sampling only up to 16 minutes (a fixed 960-frame budget — a 1-hour video drops to 0.14fps), no audio ([Nova docs](https://docs.aws.amazon.com/nova/latest/userguide/modalities-video.html)). |

**A note on keeping Google straight:** Google sells two entirely different video surfaces and they must not be conflated. *Google Video Intelligence* is the 2017-era per-feature annotation API — fixed label taxonomies, no prompting anywhere, frozen release notes, and a pricing page that still lists Celebrity Recognition, a feature Google shut down in September 2025 ([pricing](https://cloud.google.com/products/video-intelligence/pricing), [deprecations](https://docs.cloud.google.com/video-intelligence/docs/deprecations)). *Gemini* is the modern surface — genuinely strong open-vocabulary video understanding for stored files, with the live-input constraints described above. When we say "Google's video API is abandoned," we mean GVI; when we say "1fps stills," we mean Gemini Live. Different products, different criticisms.

---

## Capability snapshot

Across ten providers, here's where the capabilities actually sit. Including the three where competitors beat us outright — I'd rather print those myself than have you find them later:

- **Live stream input.** Primate Vision is the only managed, native-frame-rate path (WebRTC). NVIDIA VSS does it well — self-hosted. Roboflow streams live, for object detection only. Gemini Live accepts ≤1fps JPEG stills. Google VI streaming is Beta with a DIY C++ proxy. AWS streaming is closed to new customers. Twelve Labs, Azure Video Indexer, and OpenAI have no live video input at all.
- **Published real-time latency.** Primate publishes 45ms p50 / 316ms p95. Roboflow publishes per-frame YOLO numbers; NVIDIA publishes hardware-dependent benchmarks. Nobody else in the landscape publishes any latency number for video analysis.
- **Open-vocabulary natural-language prompting.** Primate Vision, Twelve Labs, Gemini, NVIDIA VSS, and open VLMs all take plain-English prompts (OpenAI on still images only). Roboflow's open-vocab covers object nouns only. Google VI and AWS Rekognition have no prompting anywhere.
- **Deterministic verdict + calibrated confidence.** Primate Vision alone returns a closed-vocabulary yes / no / indeterminate with a calibrated 0–1 score. Twelve Labs' JSON-schema output approximates the shape but is generative and uncalibrated; every LLM-based alternative is stochastic.
- **Timestamped segments.** Widely available — Primate, Twelve Labs, Gemini, Google VI, AWS, Azure, and NVIDIA all deliver them. OpenAI and Roboflow don't.
- **Evidence clips / overlay video.** Primate ships an annotated overlay evidence video you can watch and forward. AWS streaming returns a single first-detection frame JPEG; most others return nothing watchable.
- **Bounding-box JSON.** We don't do it — and Google VI, AWS Rekognition, Roboflow, and NVIDIA VSS all deliver frame-accurate spatial JSON. If you need coordinates for downstream geometry, they beat us outright.
- **Embeddings / semantic search.** We have none. Twelve Labs is the category flagship; Gemini, NVIDIA VSS, and Roboflow ship it too.
- **Transcription / OCR / audio.** We're visual-only. Azure Video Indexer's core strength; Twelve Labs and Gemini are also strong here.
- **Webhooks.** Primate (Standard Webhooks signing, retries, redelivery, plus `Prefer: wait`), Twelve Labs, Gemini, AWS, Azure, and Roboflow: yes. Google VI: no. NVIDIA VSS: message-broker integration, not developer-facing webhooks.

Read the bounding-box, embeddings, and transcription bullets honestly: those are the three capabilities where several competitors beat us outright. If you need frame-accurate spatial JSON, Google VI, AWS Rekognition, Roboflow, and NVIDIA VSS all deliver it and we don't. If you need semantic search over a large archive, Twelve Labs is the category leader. If you need transcription and OCR, Azure Video Indexer and Gemini are built for it.

---

## Pricing: two lanes, and the honest math

Primate Vision has **two pricing lanes**, and every comparison in this library states both:

1. **Metered — $0.01/second of source video.** For on-demand analysis: clips, incidents, agent-invoked checks, dev/test, bursty workloads. Flat and fps-independent. Queued time free; failed jobs free. A 30-second clip answered with a verdict, confidence, timestamps, and an evidence video costs $0.30.
2. **[Enterprise — contact us](https://primateintelligence.ai/pricing#enterprise).** For 24/7 continuous monitoring and camera fleets: dedicated capacity or on-site deployment, priced at a small fraction of the metered rate under highly discounted enterprise plans. The metered price is deliberately not the continuous-monitoring price.

Here's the normalized table, cheapest to most expensive — including the number our competitors would quote against us, because hiding it would be dishonest:

| Provider | Billing model | $/camera-hour @1fps (normalized) | Key gotcha |
|---|---|---|---|
| Self-hosted open VLMs | GPU rental + free weights | $0.03–$0.15 *estimate* | Assumes 100% utilization + free engineers; realistically ~0.5–1 FTE of MLOps is hidden in this number |
| AWS Bedrock Nova (Lite) | per-token, offline only | ~$0.06 *estimate* | 1fps only ≤16 min; 672×672 distortion; no audio; no live input |
| Gemini (Flash-Lite → Flash) | per-token | $0.28–$1.39 *estimate, input-only* | Live re-bills entire session context every turn — compounding cost curve |
| NVIDIA VSS | infra + per-GPU license | ~$0.3–$1.4 *estimate at fleet utilization* | High fixed cost; ~$14–22/hr box cost at single-stream utilization |
| AWS Rekognition streaming | per-minute streamed | $0.49 | Grandfathered accounts only; people/pets/packages; ≤120s/event |
| Twelve Labs | per-min index + monthly infra + per-query | $1.75–$2.52 *estimate* | Perpetual $0.0015/min-month fee: a 10,000-hour archive costs $900/month just to stay searchable ([pricing](https://www.twelvelabs.io/pricing)) |
| OpenAI (frame workaround) | per-token | $1.84–$6.62 *estimate* | No video API — DIY frame pipeline; 1,500-image cap ≈ 25 min per request |
| Azure Video Indexer | per input-minute, meters stack | $2.70–$11.40 | Audio + video meters stack; re-index re-bills |
| Roboflow | subscription + credits | $3.00–$8.00 managed | Credits burn even on self-hosted inference |
| Google Video Intelligence | per-feature × per-minute | $6.00–$7.20 per feature | Partial minutes round UP: 1,000 five-second clips bill as 1,000 minutes ([pricing](https://cloud.google.com/products/video-intelligence/pricing)) |
| **Primate Vision — metered** | per-second, flat | **$36.00** † | See footnote — this is the on-demand lane, not the fleet price |
| **Primate Vision — enterprise** | contact us | *a small fraction of the metered rate* | 24/7 / fleet lane; dedicated capacity or on-site deployment |

† *Metered rate normalized for comparison. Primate does not sell 24/7 continuous monitoring at the metered rate — continuous and fleet workloads use enterprise plans; [contact us](https://primateintelligence.ai/pricing#enterprise). Our own pricing page states the arithmetic plainly: a nonstop 30-day stream would be $25,920/month at self-serve rates, which is exactly why that workload belongs on an enterprise agreement.*

Three things to hold in mind when reading any $/camera-hour table in this category:

- **Per-verdict is the metered lane's real unit.** You don't buy camera-hours on the metered lane; you buy answered questions. A 30-second incident clip → verdict + confidence + timestamped evidence = $0.30. Compare that to what a false alarm costs your ops team.
- **Like-for-like fidelity changes the math.** The cheap rows sample ~1fps. At 12fps parity, Gemini's input tokens alone reach $0.28–$0.37/min — 46–62% of Primate's all-in $0.60/min — with no verdict layer, no calibration, and no evidence artifacts on top.
- **Raw price ≠ delivered capability.** Every row cheaper than us is a raw-annotation or raw-token price. What the buyer still has to build on top — structured output contracts, a streaming layer, evidence generation, confidence calibration — is the actual cost of ownership.

---

## Who should choose whom

These are genuine recommendations, not straw men. Half of them point away from us.

- **Choose Twelve Labs** if your core need is semantic search over a large stored-video archive. It's the category leader there, well-funded, with excellent agent-facing docs. It cannot watch anything live. ([Full comparison →](/compare/twelve-labs))
- **Choose Gemini** for conversational understanding of stored video files at low cost — genuinely strong. Know the live path is ≤1fps stills with 2-minute A/V sessions and a compounding per-turn meter. ([Full comparison →](/compare/gemini))
- **Choose Azure Video Indexer** for media-archive enrichment: transcription, OCR, faces, translation. It's an indexer, not an analyst. ([Full comparison →](/compare/azure-video-indexer))
- **Choose Google VI** only if you need its specific GA annotations (label detection, explicit-content, OCR) with an SLA and can live with a product frozen since 2021. ([Full comparison →](/compare/google-video-intelligence))
- **Choose AWS Rekognition** if you're already grandfathered into streaming events for people/pets/packages, or need its GA bounding-box JSON on stored video. New real-time customers: AWS itself has closed the door. ([Full comparison →](/compare/aws-rekognition))
- **Choose NVIDIA VSS** if you have GPUs, an infra team, and an air-gap requirement — it's the strongest self-hosted real-time stack, at the cost of owning all of it. ([Full comparison →](/compare/nvidia-vss))
- **Choose Roboflow/Ultralytics** for millisecond per-frame object detection, custom-trained models, and edge deployment. It does not answer "did X happen." ([Full comparison →](/compare/roboflow-ultralytics))
- **Choose self-hosted open VLMs** if raw unit cost dominates everything, you have MLOps capacity, and you're prepared to build ingest, streaming, structured output, and evaluation yourself. ([Full comparison →](/compare/open-vlms))
- **Choose OpenAI** for still-image and document reasoning — the best there is. It sells no video input path in any API. ([Full comparison →](/compare/openai))
- **Choose Primate Vision** when you need a *managed* API that watches video — live or uploaded — and returns a *deterministic, auditable answer*: yes / no / indeterminate, a confidence score, timestamps, and evidence you can watch. That lane, in 2026, is ours alone.

---

## The deep dives

Each head-to-head gets its own full post, with per-claim citations to the vendor's live documentation:

- [Primate Vision vs. Twelve Labs](/compare/twelve-labs) — real-time verdicts vs. archive search
- [Primate Vision vs. Gemini](/compare/gemini) — 1fps stills vs. native frame rate
- [Primate Vision vs. NVIDIA VSS](/compare/nvidia-vss) — a managed API vs. a build-it-yourself blueprint
- [Primate Vision vs. AWS Rekognition + Bedrock Nova](/compare/aws-rekognition) — the door AWS closed
- [Primate Vision vs. Roboflow / Ultralytics](/compare/roboflow-ultralytics) — detections vs. verdicts
- [Primate Vision vs. OpenAI](/compare/openai) — a video API vs. a frame-extraction workaround
- [Primate Vision vs. Azure Video Indexer](/compare/azure-video-indexer) — an insight catalog vs. a live answer
- [Primate Vision vs. Google Video Intelligence](/compare/google-video-intelligence) — any question you can type vs. a label menu frozen in 2021
- [Primate Vision vs. Sieve](/compare/sieve) — the video API that quietly left the market
- [Primate Vision vs. self-hosted open VLMs](/compare/open-vlms) — buying answers vs. building a video pipeline

---

## Try it

**[Try it for free](https://primateintelligence.ai/signup)** — real processing, no card required to start.

**Your AI agent can do it for you — right from Claude.** Primate Vision ships an MCP server, llms.txt, markdown docs twins, and a sandbox key available in a single POST — your agent can get a key, upload a clip, and read back a verdict without a human touching a dashboard.

For 24/7 monitoring and camera fleets: [see our enterprise plans](https://primateintelligence.ai/pricing#enterprise).

---

*Method note: every competitor claim in this post traces to a primary source — vendor docs, pricing pages, release notes — accessed 2026-07-31. Estimates are labeled. An earlier April 2026 version of this analysis was re-verified from scratch; anything that couldn't be re-verified was dropped.*