Import limits and throughput
Import request limits and configurable worker capacity.
Split imports into bounded batches. Acceptance means the import is queued for asynchronous processing. Use the Sync status and Visibility watermark to determine when Documents are searchable.
| Input | Enforced limit |
|---|---|
| NDJSON batch request | 100 MiB |
| Operations per batch | 10,000 lines |
| JSON Document | 5 MiB |
| Uploaded file | 200 MiB |
One MiB is 1,048,576 bytes. These are request limits; the Collection's Schema and file safety checks also apply.
Worker capacity
Pipeline workers default to one replica. Operators may set
pipelineWorkers.replicas from 1 to 64 when the Cell has capacity.
ingestApi.databasePoolMaxConns configures the shared ingest-api database pool,
from 4 to 256 connections (default 32). Publication locks use a separate pool,
sized to feed workers plus the ACL consumer (up to 17 more connections). Ordinary feed uses at most 16
workers per ingest-api process, reduced for smaller shared pools.
Parse, enrich, batch-embed and feed admit at most 64 live Leases per Organization, 32 per Project and 16 per Source. Other Workload classes retain ceilings of 4, 2 and 1. These are concurrency ceilings, not guaranteed service rates. Failing Sources advance their scheduling turn on claim. ACL/tombstone work has an independent consumer and class admission lock. Lock waits return retryable errors rather than holding database capacity indefinitely.
New tombstones use reserved propagation capacity. Admission pauses when there
are 128 unconfirmed ACL/tombstone obligations for the Source or 512 across the
Cell. Public Document DELETE returns 429 and Retry-After: 1 when that
capacity is full. Split bulk API-push delete submissions into batches of at
most 128 and retry as confirmed deletions drain. A refused submission creates
no new accepted writes. ACL narrowing retains its reserved path, and accepted
deletions keep their original five-minute propagation deadline.
Throughput
A local development test on 2026-11-02 used real Postgres 16 and Vespa 8.728.1 with a batching encoder stub that waits 20 ms per request. Python workers ran on the Mac host; Postgres and Vespa ran on an eight-CPU Docker VM. Each scenario contained 128 Documents, eight shared database connections and six feed workers. Public intake, cached status and background Sync reconciliation were active. Stages were measured separately; status reads and an additional accept were probed while normalization was running. These are stage measurements, not an end-to-end import rate or a production capacity estimate. The encoder stub does not measure model inference or relevance.
The one-Chunk column is the median of five repeated short-Document samples after the completion-notification lock fix. The five-Chunk column retains the earlier single sample after the chunking optimizations.
| Stage | One Chunk per Document | Five Chunks per Document |
|---|---|---|
| Accept | 688.0 Documents/s | 511.2 Documents/s |
| Normalize | 53.7 Documents/s | 54.7 Documents/s |
| Chunk | 64.5 Documents/s | 32.4 Documents/s |
| Embed | 45.2 Documents/s | 51.0 Documents/s |
| Feed, existing bounded worker pool | 90.3 Documents/s | 66.9 Documents/s |
In the same public-route harness before the chunking optimizations, five-Chunk composition reached 8.1 Documents/s; after exact-count reuse and four CPU composition threads, it reached 32.4 Documents/s. Chunk boundaries, IDs and embedding context match the original pinned manifests. These runs used the same indexed Sync reconciliation query.
The earlier short-Document feed sample fell to 4.2 Documents/s during Postgres deadlocks and lease-expiry recovery. With completion notifications isolated by producing transaction, five repetitions measured 30.3, 59.7, 90.3, 93.3 and 96.0 Documents/s, with zero reported deadlocks and exactly one attempt per feed Job. All 640 Documents were confirmed in real Vespa. During those imports, public accepts took at most 28.8 ms and status reads at most 5.8 ms (160 reads). Other stage rates also varied across repetitions; these samples do not establish an end-to-end capacity improvement. The public-route 100,000-Document measurement remains pending.
The internal ACL/tombstone propagation target is p95 at most 60 seconds during a 10× ingestion burst; authorization freshness fails closed when its bound cannot be protected.
Homelab Cell import
On the homelab Cell, a public-route import was stopped at 3,251 Documents.
The first 250-Document batch on a fresh Collection was accepted in 3.6 seconds.
Later 250-Document batches took 35–57 seconds, and after the job table had
grown, the status read for a 250-Document batch did not return within 180
seconds. The Documents were still committed. That status read scans job once
per line, which is what stretches a batch from about 2 seconds to beyond 80
seconds.
Once Documents are queued, embed and feed drain together. For 1,000 Documents on eight workers, embed completed at 12.6 Documents/s and feed at 14.0 Documents/s. Feed's queue stayed under 20 while embed held the backlog. Workers peaked at 475 millicores of 8 cores, the relay at 17 millicores, Vespa at 374 millicores, and the Mac embedder at about 20% of one core. The Mac itself does about 56 items/s for a single item and about 800 items/s at 64. From inside the Cell, one item through the relay and tunnel is 16.4/s and eight items are 110/s. Workers request one Document at a time, so the searchable rate sits on that round trip. A run that showed about 4 Documents/s had no embed backlog: the status query was starving the queue. Vespa was not the limit. No 100 searchable Documents/s rate was reached.
A deterministic-double run used four workers, scaled down first so the
rollout fit the namespace quota. It embedded 400 Documents in about 14
seconds (about 28/s) with the Mac idle. The embed backlog stayed at 16. All
400 feed jobs then dead-lettered as poison (feed_rejected), so the double
did not raise the searchable rate.
Eight ACL narrows sent while the embed backlog was 17 left public search in 31.7–32.9 seconds (p95 32.9 seconds). The backlog was still 33 when they disappeared. A 10-Document second Source accepted in 24 seconds at backlog 22. Its embed jobs finished in about a second while the burst still had embed work left, and its feed completed about 118 seconds after the request. It was searchable after that, and it was not stuck behind the thousand-Document queue.