CLI (semvec)

The semvec command ships with the [api] extra and wraps uvicorn to run the REST API. Install it via:

pip install "semvec[api]"

Commands

semvec serve

Start the Semvec REST API server.

semvec serve [--host HOST] [--port PORT] [--workers N] \
             [--embedder URL] [--embedder-mode inline|sidecar] \
             [--reload] [--log-level LEVEL]

Flag	Type	Default	Description
--host	str	0.0.0.0	Bind address. Use 127.0.0.1 to restrict to localhost.
--port	int	8080	TCP port.
--workers	int	1	Number of uvicorn worker processes. With >1 the model would normally load N times — combine with --embedder or --embedder-mode sidecar to share a single embedder across workers. Incompatible with --reload.
--embedder	str	unset	Sidecar URL (unix:///abs/path.sock or tcp://host:port). Workers inject a SidecarEmbedderClient instead of loading the model in-process. The daemon must already be running (python -m semvec.embedder --listen ...).
--embedder-mode	inline / sidecar	inline	inline: each worker loads its own model. sidecar: semvec serve spawns one embedder daemon, waits for READY, then starts the API workers and points them at it via UDS. Best for multi-worker deployments.
--reload	bool flag	off	Enable uvicorn auto-reload on source change. Development only.
--log-level	critical / error / warning / info / debug	info	Log level for both uvicorn and the semvec application.

The server loads semvec.api:create_app via uvicorn's --factory mode, so every process creates its own SessionManager, ClusterManager, etc. (state is in-memory and therefore per-worker by default — for cross-worker persistence select a durable backend via SEMVEC_SESSION_BACKEND, or SEMVEC_STATE_PERSIST=1 on the memory backend; see State persistence & durability).

python -m semvec.embedder (sidecar daemon)

Stand-alone embedder daemon. The API workers connect to it over UDS or TCP. Use this when you want to scale API workers independently of the embedder, or run the embedder on a different host / GPU.

python -m semvec.embedder --listen unix:///run/semvec/embedder.sock \
                          --model all-MiniLM-L6-v2

Flag	Type	Default	Description
--listen	str	required	unix:///abs/path.sock (Linux/macOS) or tcp://host:port (Windows-friendly).
--model	str	all-MiniLM-L6-v2	Any sentence-transformers model name.
--dimension	int	384	Output dimension — must match the model.
--batch-max	int	32	Max texts coalesced per encode call.
--batch-wait-ms	float	5.0	Max wait (ms) for a batch to fill. Lower = lower latency, higher = better GPU utilisation.
--ready-fd	int	unset	Inheritable fd to write READY\n on once the listener is accepting. Used by semvec serve --embedder-mode sidecar for the parent/child handshake.
--log-level	critical / error / warning / info / debug	info	Daemon log level.

The daemon installs SIGTERM / SIGINT handlers that drain in-flight batches before exit. Clients that lose the connection during drain receive a clean error and can reconnect once the daemon restarts.

Environment variables read at start-up

All variables below are read once per worker at process start. Change a value, then restart semvec serve for it to take effect. Grouped by concern.

Server & auth

Variable	Default	Purpose
DATABASE_URL	(none — required when persisting)	SQLAlchemy URL for the session / cluster / audit metadata store. No default: only read when persistence is on, and refused rather than guessed if missing.
CORS_ORIGINS	empty (no cross-origin access)	Comma-separated list of allowed origins, e.g. https://app.example.com,http://localhost:5173. When unset, the CORS middleware is skipped entirely for a small per-request win.
SEMVEC_LICENSE_KEY	—	Ed25519-signed license JWT (Pro / Enterprise features).
METRICS_USER / METRICS_PASSWORD	—	Basic Auth for the /metrics endpoint. Must both be set to enable the endpoint.

Session lifecycle

Variable	Default	Purpose
SEMVEC_MAX_SESSIONS	10000	Hard cap on concurrent sessions per worker. Oldest-touched sessions are evicted on overflow.
SEMVEC_SESSION_IDLE_TTL_S	1800 (30 min)	Sessions untouched for this long are evicted by the background sweeper. Set to 0 to disable.
SEMVEC_SESSION_SWEEP_S	60	How often the background task scans for idle sessions. Set to 0 to disable the sweeper entirely (useful in tests).

State persistence (0.7.1+)

Variable	Default	Purpose
SEMVEC_STATE_PERSIST	0 (off)	memory backend only — redis and mongo are durable on their own and do not read it. Set 1 to make per-session state durable across worker restarts: state is flushed periodically (and on SIGTERM) to the DATABASE_URL store and lazily reloaded on first access. Requires Postgres for production use. Assumes one owner worker per session — use sticky routing. See State persistence & durability.
SEMVEC_STATE_DB_SHARDS	unset	Advanced scaling. Comma-separated Postgres DSNs. When set (with SEMVEC_STATE_PERSIST=1), the state-blob table is sharded by session_id via rendezvous (HRW) hashing — adding a shard moves only ~1/N keys. Metadata tables stay on the primary DATABASE_URL. A write-throughput lever for very large deployments.

Session backends (redis / mongo)

Variable	Default	Purpose
SEMVEC_REDIS_SKIP_DURABILITY_CHECK	unset	Escape hatch, and the one to think twice about. On startup the redis backend refuses a server configured to discard keys (no AOF/RDB, or a maxmemory-policy that evicts), because a discarded key is a lost session rather than a cold read. Set 1 to start anyway — appropriate only when you know the durability comes from somewhere else, e.g. a managed Redis whose config the client cannot see.
SEMVEC_MONGO_INDEX_WAIT_S	30.0	How long the first server-side query waits for the Atlas Vector Search and lexical indexes to finish building. The builds are asynchronous, so a fresh collection is not immediately queryable; on timeout the turn falls back rather than failing. Raise it on a large collection, lower it if you would rather degrade fast.

OpenAI-compatible proxy

Variable	Default	Purpose
SEMVEC_OPENAI_UPSTREAM_BASE_URL	unset	The upstream chat endpoint /openai/v1/* forwards to. Empty leaves the proxy unmounted, which is the default.
SEMVEC_OPENAI_UPSTREAM_TIMEOUT_S	120	Request timeout against that upstream, in seconds.

Embedder

Variable	Default	Purpose
SEMVEC_EMBEDDER_URL	unset	Same effect as --embedder. When set, the lifespan injects a SidecarEmbedderClient instead of loading the model in-process. Read by every worker.
SEMVEC_EMBEDDER_MODEL	all-MiniLM-L6-v2	Default sentence-transformers model name the sidecar daemon loads when --model is not provided. Override per-deployment.
SEMVEC_EMBEDDER_DIM	384	Output dimension expected from the sidecar; must match the model the daemon was launched with.
SEMVEC_EMBEDDER_CACHE_SIZE	0 (disabled)	When >0, wraps the injected embedder in a CachedEmbedder with this LRU capacity. Cache hits skip the model; concurrent submits for the same text dedup onto one underlying encode. Cheapest path to ×2–×3 RPS on chat traffic.
SEMVEC_USE_RUST_EMBEDDER / SEMVEC_EMBEDDER_BIN	unset	Opt-in switches that make --embedder-mode sidecar spawn the Rust semvec-embedder binary instead of the Python daemon. See Embedders guide.

Retrieval (/v1/run)

Variable	Default	Purpose
SEMVEC_RUN_TOP_K	5	Number of memories surfaced per /v1/run (used by the context block, short-circuit, and drift scoring). Raising it catches lexically-distant facts; lowering it keeps prompts tight.
SEMVEC_MMR_FETCH_K	0 (disabled)	When > SEMVEC_RUN_TOP_K, fetch this many candidates and Maximal-Marginal-Relevance rerank down to SEMVEC_RUN_TOP_K. Demotes near-duplicate memories so diverse facts survive into the final set. 50–200 is a good starting range.
SEMVEC_MMR_LAMBDA	0.5	MMR relevance/diversity mix. 1.0 = pure cosine retrieval (no diversity), 0.0 = pure diversity (no relevance).
SEMVEC_CONTEXT_BUDGET_CHARS	4000	Total character budget for the context string returned by /v1/run, packed sum-as-you-go across retrieved memories. Replaces the legacy per-memory 150-char cap. Long memories use what they need; short ones don't waste budget.

BM25-hybrid retrieval (opt-in, needs semvec[hybrid])

Variable	Default	Purpose
SEMVEC_HYBRID_BM25	0 (off)	Master switch. When 1, every session also maintains a per-session BM25 index and /v1/run fuses dense + lexical candidates via Reciprocal Rank Fusion.
SEMVEC_BM25_FETCH_K	50	BM25 top-K fed into the fusion.
SEMVEC_BM25_REBUILD_EVERY	64	Ingests between snapshot rebuilds of the per-session BM25 index. Lower = fresher BM25 at higher rebuild cost.
SEMVEC_RRF_K	60	RRF smoothing constant. The standard value from the RRF paper; rarely worth changing.
SEMVEC_RRF_WEIGHTS	unset (uniform)	Comma-separated per-list weights, e.g. "1.0,0.4" to halve the BM25 contribution. Useful when BM25 hurts single-fact precision.

Cross-encoder rerank (opt-in)

Variable	Default	Purpose
SEMVEC_RERANK_MODEL	unset (off)	HuggingFace model ID, e.g. cross-encoder/ms-marco-MiniLM-L-6-v2. When set, /v1/run reranks the BM25 / dense fusion output through this cross-encoder before returning the final top-K.
SEMVEC_RERANK_FETCH_K	50	Candidate pool fed into the cross-encoder.
SEMVEC_RERANK_BATCH	64	Cross-encoder batch size. Tune against the GPU/CPU running the worker.
SEMVEC_RERANK_FP16	0	Set 1 for FP16 inference on GPU — typically 1.5–2× faster with no observable quality loss.
SEMVEC_RERANK_THREADS	os.cpu_count()	Torch intra-op thread cap for CPU inference. Set lower if you co-locate the API with other CPU-heavy tasks.

Anchors & extraction

Variable	Default	Purpose
SEMVEC_TOPIC_SWITCH	1	Master switch for the topic-switch detector. 0 disables — useful for parity tests that must hold the state still.
PSS_TOPIC_SWITCH	—	Deprecated legacy alias for SEMVEC_TOPIC_SWITCH read as a fallback by the session manager; scheduled for removal in 1.0. Prefer the SEMVEC_*-prefixed variable.
SEMVEC_AUTO_ANCHOR_ON_TOPIC_SWITCH	0	Set 1 to snapshot semantic_state as a fresh anchor every time the detector fires. Capped by SEMVEC_MAX_AUTO_ANCHORS.
SEMVEC_AUTO_ANCHOR_FROM_EXTRACT	0	Set 1 to also create anchors from extracted-entity embeddings (when auto-extract is on).
SEMVEC_MAX_AUTO_ANCHORS	32	Cap on the number of anchors created via either auto-anchor path. (32 is the server default when SEMVEC_AUTO_ANCHOR_ON_TOPIC_SWITCH=1 and the variable is unset; the in-process SemvecConfig.max_auto_anchors default is 8.)
SEMVEC_AUTO_EXTRACT	0	Set 1 to enable best-effort numeric / entity extraction from ingested text.
SEMVEC_AUTO_EXTRACT_BROAD	0	Broader extractor profile (more recall, more noise). Implies SEMVEC_AUTO_EXTRACT=1.
SEMVEC_ENABLE_NUMERIC_EXTRACTOR	1	Set 0 to disable the numeric extractor (IBAN, amounts, IDs) — useful when downstream code does its own extraction.

Compliance & event store (semvec[compliance])

Variable	Default	Purpose
SEMVEC_COMPLIANCE	unset (off)	The enable flag for the pack on semvec serve: set 1, true, or on to mount the compliance routes. The audit trail lives in whichever store SEMVEC_SESSION_BACKEND selects; a backend with no durable event store (plain memory) is refused at startup. See the Compliance Pack guide.
SEMVEC_ENABLE_EVENT_STORE	0	Set 1 to write every state mutation into the append-only event store. Required for deterministic replay and signed deletion certificates.
SEMVEC_ENABLE_HMAC_SIGNING	0	Set 1 to sign every event-store entry with HMAC for tamper-evidence. Requires a key configured in the compliance config.
SEMVEC_ENABLE_RS256_JWT	0	Set 1 to issue RS256-signed user JWTs from the compliance routes (vs HS256). Requires a private key.
SEMVEC_ENABLE_RETENTION_SWEEPER	0	Set 1 to run the background retention sweeper that deletes events older than the configured retention horizon.
SEMVEC_RETENTION_DAYS_AUDIT	2557 (≈ 7 years)	Retention horizon for audit events.
SEMVEC_RETENTION_DAYS_CHAT	365	Retention horizon for chat events.
SEMVEC_COMPLIANCE_PUBKEY_FILE	unset	Path to the compliance verifier public key (PEM). Used to verify signed deletion certificates and RS256 JWTs.
SEMVEC_COMPLIANCE_PUBKEY_PEM	unset	Inline PEM alternative to …_FILE.
SEMVEC_COMPLIANCE_PRIVKEY_FILE	unset	Path to the compliance signing private key. Issuer side only — never set on verifying instances.
SEMVEC_COMPLIANCE_PRIVKEY_PEM	unset	Inline PEM alternative to …_FILE.

Licensing internals

Variable	Default	Purpose
SEMVEC_LICENSE_KEY	—	Ed25519-signed license JWT. Required for Pro / Enterprise features and quotas.
SEMVEC_LICENSE_LRU_SIZE	256	LRU cache size for verified JWTs. Higher = more memory, fewer signature verifies per second.

API process

Variable	Default	Purpose
SEMVEC_API_THREADPOOL	200	Size of the asyncio default executor thread pool. Cap that bounds in-flight blocking work.
SEMVEC_STATE_DIR	.semvec	Default directory CodingEngine and adjacent components use for persistent state.

Coding / MCP server (python -m semvec.coding.mcp_server + hooks)

Variable	Default	Purpose
SEMVEC_WORKSPACE	—	Workspace root the coding engine indexes.
SEMVEC_EMBED_MODEL	all-MiniLM-L6-v2	SentenceTransformer model the MCP server / hooks construct their embedder from.
SEMVEC_EMBED_DEVICE	cpu	Torch device for that embedder (cpu, cuda, …).

Each of these (and SEMVEC_STATE_DIR) also accepts the legacy PSS_* spelling (PSS_WORKSPACE, PSS_EMBED_MODEL, PSS_EMBED_DEVICE, PSS_STATE_DIR) as a deprecated fallback that emits a DeprecationWarning naming the SEMVEC_* replacement.

semvec-admin (client side, not read by the server)

Variable	Default	Purpose
SEMVEC_API_URL	http://127.0.0.1:8080	Base URL the admin CLI talks to; same as --url.
SEMVEC_METRICS_USER	unset	Basic-auth user the CLI sends to /metrics; same as --metrics-user.
SEMVEC_METRICS_PASSWORD	unset	Its password; same as --metrics-password.

Note the pairing: the server checks credentials from METRICS_USER / METRICS_PASSWORD (no SEMVEC_ prefix), while the CLI sends the SEMVEC_-prefixed pair. Two names, two sides of the same credential — setting only one side leaves /metrics either unprotected or unreachable.

The API contract version matches semvec.__version__ from the installed wheel (informational; not a runtime knob).

Build-time-only environment variables (wheel builders only)

Not consumed at runtime

The variables below are read only when building semvec from source (wheel / sdist construction). Setting them on a running semvec serve process has no effect — the installed wheel already has the relevant values baked in.

Variable	Default	Purpose
SEMVEC_BASE_URL	unset	Public base URL baked into the built artefact for absolute-link generation.
SEMVEC_EMBEDDED_PUBKEY_PATH	build-time baked	Override the embedded verifier public-key path picked up by the build.
SEMVEC_BUILD_ALLOW_DEV_KEY	0	Set 1 to allow the dev verifier in a release build (refused by default).
SEMVEC_COMPLIANCE_PUBKEY_TARGET	unset	Path the build rewrites with the latest pubkey when rotating from a hot key registry.

Examples

Local development

export SEMVEC_LICENSE_KEY="eyJ..."   # evaluation keys: vertrieb@versino.de
# No DATABASE_URL needed: the default memory backend opens no database.
# (Set one, e.g. DATABASE_URL="sqlite:///dev.db", only together with
# SEMVEC_STATE_PERSIST=1.)
semvec serve --host 127.0.0.1 --port 8080 --reload --log-level debug

Production behind a reverse proxy

export DATABASE_URL="postgresql://semvec:pass@db/semvec"
export CORS_ORIGINS="https://app.example.com"
export METRICS_USER="prom"
export METRICS_PASSWORD="$(cat /run/secrets/metrics_password)"
semvec serve --host 0.0.0.0 --port 8080 --log-level info

Behind nginx / an ALB, the server trusts the X-Forwarded-For and X-Real-IP headers for client-IP resolution (used by the rate limiter and the audit log).

Programmatic start (without the CLI)

python -m uvicorn semvec.api:create_app --factory --host 0.0.0.0 --port 8080

Same effect, handy when you want to wire the factory into a larger ASGI app (e.g. mounted under a prefix).

See also

• Quickstart — 5-minute REST + library walk-through
• Embedders — user-guide page that contextualises this API
• Architecture — abstract component model