Session backends¶

semvec ships three session backends — memory, redis and mongo — selected with SEMVEC_SESSION_BACKEND. An installed package can add a fourth. This page is the contract it has to satisfy.

If you only want to choose a backend, you do not need this page: see Production hardening. This is for authors of a backend package.

The seam: one entry point¶

A package contributes a backend by advertising it in the semvec.backends entry-point group:

# pyproject.toml, in the package that provides the backend
[project.entry-points."semvec.backends"]
pgvector = "my_pkg.adapter:PgVectorAdapter"

The key is the name an operator then selects:

export SEMVEC_SESSION_BACKEND=pgvector

Installing the package makes the name selectable; uninstalling it removes the name. There is no configuration file to keep in sync, which is why an entry point was chosen over an import-path environment variable.

Two behaviours worth knowing before you pick a name:

A name semvec ships always wins. The built-in registry is consulted first, so an entry point claiming memory, redis or mongo is never reached. Discovery widens what can be attached; it does not let a dependency redirect sessions in a process that never asked for it.
Names are folded before comparison. You write the key in pyproject.toml, an operator types it into an environment variable, and importlib.metadata preserves the case it finds. PgVector, pg-vector and pg_vector all resolve to the same backend, so a spelling mismatch between the two sides is not a silent failure.

Discovery reads metadata only. Your module is not imported until an operator actually selects your name, so a package that fails at import time breaks only itself.

What happens when the name is wrong¶

The two failure modes are deliberately different, and the difference matters when you are debugging a deployment:

Situation	Result
A name that is neither shipped nor installed	Degrades to `memory`. A typo is a user error, and a crash on start would be worse than a safe default.
A name your installed package advertises	Honoured. Since the entry-point seam landed, an advertised name is a stated intent, not a slip.
A shipped name whose driver is missing	Raises, naming the extra to install (`semvec[mongo]`). You asked for a durable store; handing you a process-local one instead would lose data silently.
An installed adapter that does not satisfy the contract	`TypeError` at load time, naming the member that is missing or not callable.
A name two installed packages advertise	`RuntimeError` when that name is selected, naming the packages to uninstall. Entry points are discovered in filesystem-traversal order, so picking one would send this process's sessions to whichever came last.

The contested case is worth a moment when you choose your entry-point key: the ambiguity is kept as a value rather than raised at discovery, so an unrelated collision never breaks resolving the backend an operator did select — but it does make your name unusable while the other package is installed. Pick something distinctive.

The contract¶

Your object must satisfy semvec.api.backend_adapter.BackendAdapter. It is a runtime_checkable Protocol, so conformance is checked when your adapter is attached — not only by a type checker you may never run.

Attributes¶

name: str — the name an operator selects. Must match the entry-point key it is found under.

is_persistent: bool — whether state written here outlives the process. This is not a gate. memory is a legitimate adapter and the default, so refusing a non-persistent one would refuse the default. The flag exists so the process can announce the truth at startup: a session that will not survive a restart should be stated, not discovered.

Methods¶

state_store() -> StateStoreProtocol — the session-state half. Must return the same instance on every call within one adapter; a fresh store per call would give every caller its own state.

event_store() -> Any | None — the compliance half, or None when this backend keeps no event log. None is a real answer, not an omission: a non-persistent backend cannot hold an audit trail, and pretending otherwise would hand an operator a scratch buffer they believe is a log. When the Compliance Pack is enabled, this is where its store comes from.

retrieval_backend() -> RetrievalBackend | None — the server-side retriever this backend supplies, or None to retrieve in-process. The same optional-capability shape as event_store(), for the retrieve stage instead of the audit trail. A backend that can rank server-side (Atlas $vectorSearch, pgvector, …) returns its own RetrievalBackend; None — the answer every shipped adapter gives today — means the engine uses the built-in InProcessRetrievalBackend (the cosine/BM25/MMR/cross-encoder pipeline). Declaring it now is what lets a future capable backend add server-side retrieval by implementing this one member, with no edit to the engine's retrieve path.

commit_turn(session_id, blob, *, expected_version, owner_subject=None, pending_message=None, events=None) -> int — write the state and its events as one unit, and return the new version.

This is the member the contract exists for. Compare-and-set semantics are unchanged from state_store().save: expected_version=None inserts and must never overwrite an existing row; an int updates only on an exact match and raises StateVersionConflict otherwise. The addition is events, and with it the obligation: either both halves land or neither does. An adapter that cannot promise that must not claim this interface.

It is one call rather than "save, then append" because two calls cannot be atomic across arbitrary backends without exposing a transaction object — and a transaction object would leak the storage model into the interface, since SQLAlchemy sessions, Mongo client sessions and Redis MULTI/Lua share nothing that can be typed. One call keeps the boundary inside your adapter, where your storage model already lives.

check_health() -> bool — can this backend be reached right now?

A live probe, not a declaration. /v1/readyz calls it, so it must be cheap: a PING, a SELECT 1, whatever your store answers without touching session data. An implementation that walks the keyspace turns a health check into an outage of its own.

Return False rather than raising. The caller reports several dependencies together and must not be derailed by the first unhappy one; you know which exceptions your client raises and the probe does not, so swallow them and answer the question.

This is a method and not an attribute for a reason worth repeating: is_persistent describes the backend and cannot change, while health changes under a running process. An attribute would be a promise that cannot fail.

A minimal adapter¶

from semvec.api.state_store import StateStoreProtocol


class PgVectorAdapter:
    name = "pgvector"
    is_persistent = True

    def __init__(self) -> None:
        self._store = PgVectorStateStore()      # your StateStoreProtocol implementation
        self._events = PgVectorEventStore()     # or None, if you keep no log

    def state_store(self) -> StateStoreProtocol:
        return self._store                      # the same instance every time

    def event_store(self):
        return self._events

    def retrieval_backend(self):
        return None                             # or your server-side RetrievalBackend

    def commit_turn(
        self,
        session_id,
        blob,
        *,
        expected_version,
        owner_subject=None,
        pending_message=None,
        events=None,
    ) -> int:
        # `transaction()` is YOUR method, not part of any semvec protocol — the contract
        # deliberately says nothing about how you achieve the atomicity, only that you do.
        with self._store.transaction() as tx:   # both halves, one unit
            version = tx.save(
                session_id,
                blob,
                expected_version=expected_version,
                owner_subject=owner_subject,
                pending_message=pending_message,
            )
            for event in events or ():
                tx.append(event)
            return version

    def check_health(self) -> bool:
        try:
            self._store.ping()
            return True
        except Exception:
            return False

Two shipped implementations are worth reading, and they answer different questions:

semvec.api.backend_adapter.MemoryBackendAdapter — the smallest thing that satisfies the contract. Read it for the shape. Note that its commit_turn raises when handed events rather than dropping them: it has no event store, and silently discarding an audit row is worse than refusing the turn.
semvec.api.redis_backend_adapter.RedisBackendAdapter — the reference implementation, and the one to read for the part that is actually hard. It shows what commit_turn's atomicity costs on a store with no transactions: the compare-and-set and the event writes are one Lua script, because MULTI queues commands but cannot branch on what it read. Its companion semvec.compliance.redis_event_store.RedisEventStore shows the other half — the secondary indexes that stand in for WHERE user_id = ? and created_at < cutoff.

Before you ship¶

Run the conformance suite against your store. tests/test_state_store_conformance.py pins the behaviours a StateStoreProtocol must have — that an insert onto an existing row is refused, that an omitted owner clears rather than keeps, that a version advances by exactly one. Structural conformance is checked at load time; these are the semantics that isinstance cannot see.
Decide is_persistent honestly. It drives what operators are told at startup.
Test the conflict path. A commit_turn that returns instead of raising on a version mismatch turns a lost update into silent data loss, and the caller's retry logic depends on the exception.

Server-side retrieval (mongo `$rankFusion` hybrid)¶

The mongo backend can rank retrieval server-side instead of in-process, the first concrete answer to retrieval_backend(). It is opt-in — set SEMVEC_MONGO_ITEM_SYNC=1 — and needs a deployment with the Atlas Vector Search service (the bundled mongodb/mongodb-atlas-local image, or MongoDB Atlas). With it off, the mongo backend keeps the in-process pipeline like every other backend, so existing deployments are unchanged.

Two moving parts:

Item-sync (semvec.api.mongo_item_sync.MongoItemSync). The engine exposes no item-mutation feed, so as a session's memory tiers change each memory item is dual-written as a queryable document into the shared semvec_memory_items collection. The mutations are derived by diffing the public tier enumeration (memory.short_term / medium_term / long_term, keyed on the stable item_id) against the persisted per-session view — add / update / soft-evict. This runs on the persist path, best-effort, so a sync failure never fails a durable turn. The same pass provisions both search indexes — the Atlas Vector Search index (dimensions = the embedder's dimension, cosine) and the Atlas lexical $search index over the item text; the builds are asynchronous, so wait_for_vector_index() polls each until it is queryable.
MongoRetrievalBackend (semvec.api.mongo_retrieval_backend). A real RetrievalBackend whose search(ctx) fuses up to three signals over one session's live items — semantic ($vectorSearch on embedding), lexical ($search on text, which rescues the rare high-signal tokens a dense turn-vector averages away) and recency ($sort on updated_at) — and returns the top-k as MemoryUnit objects, the same shape InProcessRetrievalBackend returns, so the engine's retrieve stage is untouched. It uses the query embedding and query text already on the RetrievalContext (the query is never re-embedded, so query and item embeddings always share the session's model and dimension) and scopes every sub-pipeline to session_id, excluding evicted items. When the deployment offers $rankFusion (MongoDB 8.1+) the fusion runs server-side with per-signal weights; otherwise — or with force_client_rrf=True — the sub-pipelines run separately and the wheel's rrf_fuse (semvec.api.hybrid_retrieval) performs the reciprocal-rank fusion client-side with the same weights (detect_rank_fusion() probes support once). A query with no text drops the lexical signal and fuses semantic + recency.

SessionManager injects the MongoRetrievalBackend per session automatically when the mongo backend is active with item-sync on.

What changes when retrieval moves server-side¶

Server-side retrieval is not the in-process pipeline executed elsewhere. The two rank on different signals and narrow differently, so the same query against the same session can come back with a different set of memories. Switching SEMVEC_MONGO_ITEM_SYNC on is a retrieval decision, not only a latency one.

Where they differ:

	in-process	mongo server-side
semantic	cosine over the live memory tiers	`$vectorSearch` over the mirrored item docs
lexical	BM25, only when enabled	`$search`, only when enabled
recency	—	`$sort` on `updated_at`; a fifth of the weight with lexical on, more without
diversity	MMR when `mmr_fetch_k` exceeds `top_k`	same rule, applied client-side
precision rerank	cross-encoder when one is configured	same rule, applied client-side
fusion tuning	`rrf_k` / `rrf_weights` from the caller	fixed `DEFAULT_WEIGHTS` (0.5 / 0.3 / 0.2)
candidate width	`fetch_width(ctx)`	`fetch_width(ctx)`, widened for the live-set filter

What both backends agree on. Two questions are answered once, in semvec.retrieval_backend, and every backend reads the answer rather than deciding for itself: fetch_width(ctx) — how many candidates to pull before narrowing — and narrowing_stage(ctx) — which of cross-encoder rerank, MMR diversity, or a plain slice applies. So a configured reranker runs on both, MMR diversity runs on both, and BM25 turned off is off on both. A backend that reimplements either rule instead of calling it is how these two came apart in the first place.

Recency has no equivalent in-process. The divergence runs both ways: mongo ranks partly on how recently an item was touched, and the in-process pipeline has no such term at all. Server-side retrieval is therefore not a subset of in-process retrieval — it is a different ranking.

Knobs that are inert on this backend. These RetrievalContext fields are declared, passed, and ignored server-side. Each is a divergence that cannot close rather than one nobody got to:

rrf_k, rrf_weights, fuse_with_bm25 — the fusion mechanics. rrf_weights weighs two signals (dense + BM25); the server-side fusion combines three named ones, and there is no meaning-preserving map between them, so the caller's weights are replaced by DEFAULT_WEIGHTS.
memory — mongo reads the item mirror, not the live tier state. The mirror is written by _maybe_sync_mongo_items on the persist path, which runs after the turn's state write, so server-side retrieval on a turn cannot see that turn's own memory mutations. In-process retrieval reads the tiers directly and can. Reading the live tiers is the thing server-side retrieval exists to avoid, so this one is a property of the design, not a gap in it.
stage — no retrieve / fuse / rerank trace events are emitted. The live debugger shows a gap where the retrieve stage would be on a mongo-backed turn. Closable, and not yet closed.

Measured and pinned by tests/test_retrieval_semantics_are_documented.py, which derives the inert set from the code so this list cannot quietly go stale; the internal record is docs-internal/known-limits.md §11.