Changelog¶
All notable changes to semvec are documented in this file.
The format is based on Keep a Changelog, and this project aims to follow Semantic Versioning. Versions on PyPI use PEP 440 pre-release notation (0.1.0a2); git tags and headings here mirror that.
Unreleased¶
Documentation¶
docs/guides/compliance.mdand the README link now spell out two Compliance-Pack details that surfaced during end-to-end QA:- The HMAC middleware verifies against
request.url.pathonly; the query string is not part of the canonical request. Sign the path without the query, treat query parameters as read-only-shape filters, and don't put tamper-relevant input (e.g.?action=delete) there. POST /v1/compliance/users/{uid}/forgetoverrides the request-bodyreasonfield with the fixed value"user_request"before the certificate is signed — the cert is an operator-issued attestation, not user-supplied content. Callers that need a different reason useforget_user()from Python directly.
[0.4.1] — 2026-05-01¶
Fixed — feedback/REPORT_v0.4.0_compliance.md follow-ups¶
SqliteEventStore(path=":memory:")now works end-to-end. Pre-fix every store operation opened a freshsqlite3.connect(":memory:"), soinit_schema()andappend()landed in disjoint ephemeral DBs and the very first append failed withno such table: memory_events. The:memory:branch now keeps a single connection alive for the store's lifetime, guarded by athreading.Lockso concurrent FastAPI workers cannot corrupt the DB. File-backed stores keep the per-operation connection pattern./v1/compliance/users/{uid}/forgetreturns a typed 503 when the operator has not configured a compliance signing key. Pre-fix the endpoint deleted the user's events and then failed with a generic 500 RuntimeError whensign_certificatecould not find the private key — operator-side mis-configuration silently ate user data.forget_user()now resolves the private key before the delete; the endpoint catches the resolution error and returnsHTTPException(503, detail="compliance_keypair_unconfigured").
Documentation¶
- README +
docs/guides/compliance.mdclarify that[compliance]is the pure-Python extra (cryptography>=42) and that mounting the FastAPI compliance router needs[api]on top (pip install "semvec[api,compliance]"). certificates.pydocstring spells out thatsign_certificateis RSA-PSS-SHA256 only — Ed25519 keys raise on signing; full Ed25519 support is roadmap.key_registry.pydocstring states thatregister / rotate / revoketake keyword-only arguments andUserKeyis read-side only (callers do not construct it).
[0.4.0] — 2026-05-01¶
Added — Compliance Pack (semvec.compliance)¶
A new sub-package next to cortex and coding, adding the
data-protection and cryptographic-verification layers that
regulated tenants need on top of the base SemvecState. Every
feature is gated behind a SEMVEC_ENABLE_* env var, all defaulting
to off.
Foundations
- EntityKind gains three new variants — numeric, date,
identifier — both Rust-side (src/literal_cache.rs) and on the
Python shim. Existing kinds untouched.
- ComplianceConfig.from_env() reads five feature flags and two
retention day counters.
Event store + replay
- MemoryEvent schema (UUID, tz-aware UTC, embedding, JSON-safe
meta, optional source-event back-reference).
- EventStore ABC + SqliteEventStore (file-backed; embeddings as
JSON arrays). Cosine top-N via NumPy scan.
- EventReplayService rebuilds SemvecState deterministically;
uses a new _internal_record_replay_step() PyO3 method that skips
the per-state community-tier rate limiter so replay does not lock
itself out re-folding its own log.
- ComplianceState wrapper composes SemvecState and mirrors every
successful update() into the store. Failures (dim mismatch,
isolation reject) propagate without writing.
Retention + GDPR
- RetentionSweeper.sweep(retention_days=30) — idempotent purge
with audit-log entries.
- forget_user() — synchronous Art. 17 wipe + signed
DeletionCertificate (RSA-PSS-SHA256). Always returns a
certificate, even on an empty store ("we looked, found nothing,
confirmed").
- _embedded_pubkey.py is rewritten by
scripts/embed_compliance_pubkey.py at wheel-build time so
customers can verify_certificate(cert) without configuring
anything. The CI step picks the value up from the
SEMVEC_COMPLIANCE_PUBKEY_PEM repo secret. Fork PRs and local
builds get a None stub.
Verbatim-precise facts
- NumericFact / DateFact / IdFact dataclasses with Decimal,
tz-aware datetime, and ISO-13616 IBAN mod-97 validation
respectively. Pure regex — no LLM in the hot path.
HMAC request signing + RS256 user JWT
- New src/compliance.rs Rust module: AWS-SigV4-style canonical
request, HMAC-SHA256, constant-time tag compare via subtle.
- _internal_hmac_canonical / _internal_hmac_sign /
_internal_hmac_verify PyO3 bindings. Python facade in
semvec.compliance.hmac_signing.
- _internal_verify_user_rs256_jwt for per-user RS256 JWTs (private
key never leaves the client device).
- KeyRegistry Protocol + InMemoryKeyRegistry with register /
rotate (24h grace) / revoke / lookup.
- ComplianceHmacMiddleware enforces the full flow on every
/v1/compliance/* request: mandatory headers, ±60 s timestamp
window, path-user-id ↔ signed-user-id check, signature verify,
nonce-replay check. Six typed failure codes.
REST API
- New router under /v1/compliance/users/{uid}/...:
- GET memory / DELETE memory[/event_id] / POST forget /
GET facts?type=numeric|date|identifier.
- Forget endpoint serialises the signed DeletionCertificate
so callers can verify offline.
Async worker
- InMemoryRebuildWorker decouples the post-DELETE rebuild from
the request path. Single daemon thread, flush() test seam,
shutdown() graceful-exit hook.
Demo script
- scripts/demo_compliance_pack.py — end-to-end walk through every
feature, runs against a temp directory in <2 s.
Tests¶
130+ new tests under tests/compliance/ across nine sub-files
(test_p1_foundation.py … test_p7_rebuild_worker.py). Full suite
at 777 passed.
Dependencies¶
- New runtime:
cryptography>=42(already in[api]extras for RSA-PSS-SHA256);hmac=0.12andsubtle=2Rust crates added toCargo.toml.
[0.3.8] — 2026-04-30¶
Fixed — feedback/REPORT_qa_v0.3.7.md QA findings¶
LiteralCache.clear()now wipes every field, not justentities. Pre-fix,clear()leftdecisions,invariants,error_patterns,test_historyandcode_structuresbehind, so a follow-uprecord_*call appended to old data. The new semantics match the method name: a cleared cache is empty.- Bad input to
SemvecState.update()now raises a typedValueErrorinstead of SIGABRT-ing the host process. Two cases covered: - dimension mismatch between
input_embeddingand the configured state dimension (message tells you both numbers and how to fix); - empty
input_embedding.
SemvecConfig(dimension=0) was already handled (raises
ConfigurationError); a regression test pins it.
Documentation¶
- README now states the
SemvecChatProxybreak-even point (~10 turns) explicitly so users do not flip the proxy on for very short conversations and conclude the library makes things worse.
Internal¶
chat_proxy.pyprefersmodel.get_embedding_dimension()when the bundledsentence-transformersexposes it (>= 3.x), falling back to the legacyget_sentence_embedding_dimension()— silences the FutureWarning in newersentence-transformersbuilds without breaking older ones.
[0.3.7] — 2026-04-30¶
Changed¶
- Pro and Enterprise license tiers now bypass the per-state rate
limits on
update()andcalculate_*. Paying customers are no longer subject to throttles meant to discourage anonymous probing. Tier is read fromSEMVEC_LICENSE_KEYonce atSemvecStateconstruction time and cached. Community / anonymous (no license) keep the existing limits (100/s update, 30/s calculate_*). RateLimitErrormessages no longer reference internal audit reports. They now state what hit, what to try (slow down, batch viaupdate_batch(), shard across separateSemvecStateinstances, or set a Pro/Enterprise license token), and where to upgrade. Threat-model context lives in this CHANGELOG, not in user-facing exceptions.
Fixed¶
- Privacy toggle now also covers the
LiteralCache. Theinclude_memory_text=Falseargument added in 0.3.6 only redacted the three memory tiers;state.to_dict()was still emitting every literal-cacheentities[].value/context/key, decisions, invariants, error patterns and code structures in clear. New keyword-only argumentinclude_literal_cache_text=Trueonto_dict()andto_bytes()(default backwards compatible). Calling both flags at once is the right move for full text redaction:
Embeddings, kind enum, timestamps, importances, and access
counts always ride along — the redacted snapshot is still
functionally restorable via SemvecState.from_dict() and
retrieval against it works.
[0.3.6] — 2026-04-30¶
Fixed — feedback/REPORT_v0.3.4_re_round2.md audit follow-up¶
Round-2 audit ran a wider Black-Box probe and found two open finding paths that the targeted Round-1 review had missed.
state.update()is now rate-limited (RE2-1). The Round-2 auditor measured 2538 update/s on a fresh state — unlimited enough to behaviorally map the update equation and to bypass the 0.3.4calculate_fsm30/s limit via thefsmfield returned in the update result-dict. 0.3.6 adds a per-state sliding-window limit at 100 calls / s forupdate()(and each item ofupdate_batch()). Production callers (1-10/s conversational, ~50/s batch ingest) stay well below; a system-identification probe at 1000+/s does not.RateLimitErroris raised with a threat-model-aware message.- Opt-in memory-text redaction in
to_dict()/to_bytes()(RE2-3). The default snapshot exports every memory entry's raw user text alongside the embedding. For GDPR/HIPAA workloads the blob becomes personal data; for any user a backup or shared snapshot leaks the entire conversation in clear. New keyword-only argumentinclude_memory_text=True. Set it toFalseto wipe thetextfield on every memory while keeping embedding, importance, timestamp, semantic hash, and protection score — retrieval against the redacted snapshot is unaffected. DefaultTruekeeps the existing snapshot contract.
Notes — items considered and deferred¶
update()result-dict reduction (Round-2 Pflicht 2). The audit's concern was that thefsmfield in the update result bypassed the calculate_fsm rate limit. With RE2-1 in place, the bypass rate is now bounded to the same 100/s ceiling. Removing the field would break every production caller that reads it (token-reduction serializer, coding engine, the docs example) — the rate-limit is the cleaner answer.- Configurable
norm_target(Round-2 nice-to-have 4). The hidden 1.2 phase-target is patent-claim adjacent but exposing it as a config field does not reduce claim coverage; it just makes the constant explicit. Skipped. - Library license-gate (Round-2 nice-to-have 6). Tier-based
per-second quota on
update()/calculate_*is on the roadmap with the online activation server already noted in the 0.3.3 CHANGELOG; not in scope for a point release.
[0.3.5] — 2026-04-30¶
Documentation¶
- README: removed patent-fix narrative (salt-defence walk-through, audit-N attributions, version-history breadcrumbs, telemetry patent-enforcement essay). The README is now user-facing only; engineering rationale stays in CHANGELOG and ARCHITECTURE.md.
- CHANGELOG: added an RE-B verification note under [0.3.4] — the follow-up audit's "NICHT GEFIXT" classification was a single-call probe; the rate-limiter is verified live in the published wheel.
No code changes vs. 0.3.4 — this is a docs-only release so the cleaned README ships in the PyPI wheel metadata.
[0.3.4] — 2026-04-30¶
Fixed — feedback/REPORT_v0.3.3_re_verification.md follow-up findings¶
The 0.3.3 RE audit confirmed the three patent-discoverability fixes from
0.3.3 but flagged three further leak paths that the original
REPORT_reverse_engineering.md had also called out. 0.3.4 closes them.
state.adaptive_paramsclear-text property (RE-A). ASemvecStateexposedstate.adaptive_params.beta_max,.decay_rate,.reinforcement_threshold,.diversity_penalty,.learning_rateas direct dot-access attributes — the same five tuning constants that_tuning_defaults()used to leak. Renamed tostate._internal_adaptive_params, and the per-field getters/setters onAdaptiveParametersare gone. The values now live behind a single_internal_dict()method (still quantize-4 view, audit-4 invariant). Persistence viastate.to_dict()is unchanged — this is the same trade-off the 0.3.3 CHANGELOG documents under "snapshot contract".state.calculate_fsm(arbitrary_history)synthetic probing (RE-B). An attacker could fire unlimitedcalculate_fsm/calculate_metrics/calculate_advanced_metricscalls with caller-supplied histories to reverse-engineer the FSM formula via system identification — and without ever touching theupdate()path, the diversity HLL sketch never saw the probes. 0.3.4 adds:- Per-state sliding-window rate limiter — 30 calls per rolling
second per
SemvecState. Production callers (update()invokescalculate_*once per turn) do not approach the limit.RateLimitErroris raised with a threat-model-aware message. - Diversity-sketch instrumentation — every accepted probe feeds
semvec._diversity.add(values), so a determined attacker that paces below the rate limit still flips the cardinality estimator that telemetry watches.
- Per-state sliding-window rate limiter — 30 calls per rolling
second per
- Cortex coherence/influence formula leak (RE-C). The aggregation
formula (
recent_sim*0.5 + stability*0.3 + min(norm,1)*0.2, plus thecoherence * (0.6 + 0.4 * experience)influence multiplier) lived inpython/semvec/cortex/service.pyas pure-python — the constants were readable verbatim from the installed wheel. Moved to the Rust core as_internal_cortex_score(interaction_count, norm, recent_similarities) -> (coherence, influence). The Python wrapper inservice.pyis now a thin forwarder; output is float-equal.
Verification — RE-B was addressed (note added 2026-04-30)¶
The follow-up audit (feedback/REPORT_v0.3.4_re_verification.md)
classified RE-B as "NICHT GEFIXT". Re-running the published 0.3.4
wheel against the audit's own option (b) recommendation
("Rate-Limit im Rust-Core, N Aufrufe pro Sekunde, dann
RateLimitError") shows the limiter triggers at the 31st call:
PyPI wheel version: 0.3.4
B fix — rate-limit triggered at call #31
RateLimitError: calculate_* probe rate limit exceeded
(30 calls / second per SemvecState — sliding window).
The audit's reproduction script fires a single call, which legitimately succeeds — that is the design (option b leaves the API intact for production callers). The classification was a tooling oversight, not a missing fix.
Known limitations¶
_internal_adaptive_paramsand_internal_cortex_scoreremain reachable in the public dir (dir()discoverability is acceptable; the public dot-access path is not). Full hiding requires the binary-obfuscation / online-activation escalation path noted in the 0.3.3 CHANGELOG.- The rate limit is a per-state counter; an attacker that scripts many
SemvecState()constructions amortises across many windows. This is a deliberate trade-off — a global limiter would block legitimate multi-tenant servers. Diversity-sketch instrumentation is the defence in depth here.
[0.3.3] — 2026-04-30¶
Fixed — feedback/REPORT_reverse_engineering.md audit findings¶
A black-box reverse-engineering audit found three trivially-discoverable
patent-relevant entry points on semvec._core. All three are now
private; the legitimate users (token-reduction serializer, coding
engine, the public phase-prompt forwarder) continue to work via the
internal names.
- Phase-FSM prompt strings were callable as
_core.get_phase_prompt(name)— patent claim 2 surfaced verbatim, in 30 seconds, viadir(semvec._core). The function is now registered as_internal_phase_prompt. The public Python entry point (semvec.token_reduction.get_phase_prompt) is unchanged. - Tuning-constants dump was callable as
_core._tuning_defaults()— a single call returned every magic number for the token-reduction serialiser and the coding engine in human-readable form. The function is now registered as_internal_tuning_defaultsand the two Python modules that legitimately consume it import the new name directly. - Multi-resolution-memory tier-size constants
(
SHORT_TERM_SIZE,MEDIUM_TERM_SIZE,LONG_TERM_SIZE,COMPRESSION_RATIO) were module-level attributes onsemvec._core— patent claim 3 surfaced the four numbers directly. They are now removed from the public surface; read the effective values via theMultiResolutionMemoryinstance properties (mem.short_term_size,mem.medium_term_size,mem.long_term_size) — that accessor reflects the actual configuration, including SemvecConfig overrides, which the build- time constant did not.
Documentation¶
- README Quickstart 5 now shows the
LiteralCachemulti-session memory path (record_decision/record_error_pattern/add_invariant/record_test_results/build_handoff_context) with a realistic to_bytes round-trip example. - README Quickstart 4 (Cortex) updated for the 0.3.2 fixes:
dimension=propagates to the inner state,verify_integrity()is bit-exact,ConsensusEnginequorum is measured against registered voters. - README Persistence section lists the LiteralCache fields that
to_dict()/to_bytes()round-trip preserves (≥ 0.3.1). docs/api/coding.md— dedicatedLiteralCachesection with the full record/read/handoff API surface.docs/api/cortex.md— explicit notes ondimension-propagation, ValueError-on-mismatch, quorum-against-pool, and bit-exactstate_vector_bitsserialisation.
Known limitations (not closed in 0.3.3)¶
The audit also flagged that strings(_core.abi3.so) still surfaces
the prompt strings and tuning numbers, that
state.to_dict() still serialises the effective adaptive_params
and config values for round-trip, and that the
cortex/service.py::_calculate_coherence formula and
api/global_observer.py orchestration are pure-Python. These are
documented in ARCHITECTURE.md and tracked for a future sprint;
hardening beyond the public Python surface requires either binary-
level obfuscation or a license-gated online activation server.
[0.3.2] — 2026-04-30¶
Fixed — feedback/REPORT_cortex.md audit findings¶
Three bugs in the cortex multi-agent layer that made it unusable for production:
-
SemvecAgenthonours itsdimension=argument. Pre-fix the argument was silently dropped — the innerSemvecStatewas always built with the default 384-d config. Feeding a 768-d embedding (mpnet, the audited recommended floor) then hit a Rust assertion panic (SIGABRT, not a catchable Python exception). Fix: forward the dimension via a realPSSConfigto the inner state, and validate the embedding length up-front inprocess_input_embeddingso dimension mismatches raiseValueErrorinstead of crashing the interpreter. -
StateVectorPacket.verify_integrity()round-trips cleanly. Pre-fix,pkt2 = StateVectorPacket.deserialize(pkt.serialize())producedpkt2.checksum == pkt.checksum(because the checksum string round-tripped fine via JSON) butpkt2.verify_integrity() == False— because the float-array encoding inserde_jsonis not bit-exact for arbitraryf64, and the round-trippedstate_vectorhashed to a different checksum. Fix: serialise the IEEE 754 bit pattern alongside the float array asstate_vector_bits: [u64, …](format_version 2);deserializeprefers the bit pattern. Legacy snapshots still round-trip through the fallback path. -
ConsensusEnginemeasures quorum against the registered voter pool, not just votes-cast-so-far. Pre-fix the first local YES produced ratio 1/1 = 1.0, tripped SimpleMajority, and the proposal was finalised + removed before any other registered instance could vote — remote votes were silently dropped. Fix:ConsensusProposalnow carriestotal_votersandtotal_voter_weight, seeded byConsensusEngine.create_proposalfromknown_instances.calculate_consensususes the seeded denominator; proposals built directly without an engine fall back tovotes.len()so the pss-port parity tests for rawConsensusProposalstill hold.
[0.3.1] — 2026-04-30¶
Fixed¶
LiteralCacheround-trip dropped structural fields. The external coding-use-case validation (feedback/REPORT_coding.md) found thatto_dict()/from_dict()(and thereforeto_bytes()/from_bytes()) only restoredentities. The five other fields the cache writes —decisions,error_patterns,invariants,test_history,code_structures— were silently dropped on restore. Symptom:build_handoff_context()returned an empty string after a round-trip, breaking the multi-session coding-memory USP. Root cause: an explicit "re-builds them through the recording methods after construction if needed" skip in the original PyO3 serialiser — exceptfrom_dictis exactly the moment no caller is going to re-record them. Fix: added pure-Rust bulk-restore methods onLiteralCache(restore_decisions,restore_error_patterns,restore_invariants,restore_test_history) and PyO3*_from_pydicthelpers mirroring the existing*_to_pydicthelpers. Bulk restore writes the saved state directly so original timestamps andoccurrencescounts are preserved (the recording methods would re-stamp / bump them). Verified end-to-end on the report's min-repro: 1/1/1 before and after both round-trip paths;build_handoff_context()survives with DECISIONS, INVARIANTS, and Error sections intact.
[0.3.0] — 2026-04-30¶
First stable 0.3.x release. Promotes 0.3.0a6 after six external
validation reports (feedback/REPORT.md, feedback/REPORT_v0.3.0a2.md
through feedback/REPORT_v0.3.0a6.md) cleared all show-stoppers and
acceptance gates. Aggregated highlights below; the full per-alpha
breakdown lives in the [0.3.0aN] sections that follow.
Highlights¶
- Patent-pending mechanics now actually move retrieval ranking.
add_anchor()andadd_resonance_trigger()were retrieval-inert in 0.2.x; they now biasget_relevant_memoriesvia composablemax(α · max_anchor_sim, γ · trigger_match)boosts. - Recency-bias eliminated on blocked-domain workloads. The
feedback/REPORT.mdaudit-5 Test B reproducer (4-domain × 125-note blocked ingest, mpnet 768d) crashed pre-fix retrieval to 13 % precision@3; the 0.3.0 default tier weights + cluster-fast-path fallback land at 100 %. - Topic-switch detection is observable.
state.topic_switch_historyis a bounded log of detected events (always-on withenable_topic_switch=True); the opt-inauto_anchor_on_topic_switchflag snapshots the currentsemantic_stateas a fresh anchor, closing the loop into B2's retrieval boost. - Binary persistence.
state.to_bytes()/SemvecState.from_bytes(blob)ship a magic-header + SHA-256-checked snapshot. ~ 2.4× smaller thanto_dict + json.dumpsatcompress=True;compress=Falsematches JSON byte size at JSON-equivalent speed for hot-path persistence. - Memory-safe. A daemon-thread vs interpreter-shutdown race in
the telemetry path that hit ~ 50 % of short-lived imports in
0.3.0a5 was fixed via an
atexitjoin with timeout. 0/N crashes on the original repro.
Default tunings (set during the audit-5 sprint)¶
- Tier weights
1.0 / 0.95 / 0.9(was1.0 / 0.7 / 0.4) cluster_fallback_threshold = 0.85(new in 0.3.0a2)anchor_retrieval_boost = 0.6(was 0)trigger_retrieval_boost = 0.3(new in 0.3.0a3)auto_anchor_on_topic_switch = false(opt-in, default flipped in 0.3.0a4 after on-by-default regressed Test A)max_auto_anchors = 8long_term_size = 200(was 100)
API surface additions¶
SemvecState.create_resonance_trigger(keyword, embedding, threshold)factory.SemvecState.consolidate_long_term()pass-through to memory.SemvecState.topic_switch_historygetter.SemvecState.to_bytes(compress=True|False)/SemvecState.from_bytes(blob).anchor_score,anchor_count,resonance_trigger_countexposed as@property(were callable methods).ResonanceTriggerre-exported fromsemvec.
API surface removals¶
EmbeddingServiceis no longer publicly importable (itsget_embedding()raisedNotImplementedErrorand tripped users who tried to use it). Bring your own embedder — see README Installation → Embedder.
End-to-end measurements (mpnet 768 d)¶
| Test | 0.3.0a1 | 0.3.0 |
|---|---|---|
| Test A passive (80 mixed) | 86.11 % | 86.11 % |
| Test A + add_anchor | 86.11 % | 91.67 % |
| Test B stress (500 blocked) | 13.33 % | 100 % |
| Test B ingest throughput | 75 ups | 93 ups |
| Test C topic-switch observable | no | yes |
Test D to_bytes size |
n/a | 42 % of JSON |
| Process-shutdown segfault | n/a | 0 / N |
[0.3.0a6] — 2026-04-30¶
Fixed — 0.3.0a5 follow-up from external validation¶
feedback/REPORT_v0.3.0a5.md reported a Show-Stopper memory-safety
bug and a non-reproducible ingest-speed regression:
- Sporadic SIGSEGV at process exit (Show-Stopper). Short-lived
scripts that imported semvec hit
free(): invalid pointerheap corruption ~50 % of the time. Root cause: the telemetry POST runs in a daemon thread doing TLS, and Python tears down its globals while the thread is mid-buffer-allocation. Reporter originally diagnosed this as aSemvecState(SemvecConfig(...))lifetime bug, but a localised bash-loop confirmed plainimport semvecis also affected — the SemvecConfig+SemvecState pattern just exercises enough allocations to make the race more visible. Fix: register anatexithook thatjoin()s the telemetry daemon thread with a 0.7 s timeout (slightly above urllib's 0.5 s connect timeout). Healthy network calls always finish before Python tears down; unreachable endpoints fall through within the timeout and the daemon thread is killed cleanly without half-allocated buffers. Verified: 20/20 short-lived imports of the user repro now exit cleanly (was ~10/20 before). - Test B ingest-speed observation (non-reproducible regression).
External report measured 49 u/s on Test B Stress where 0.3.0a3 had
75 u/s. Local engineering system measures 84 u/s on the same
workload, above the 70 u/s acceptance gate. The discrepancy is
CPU-performance variance between the reporter's machine and the
engineering system (1.7× constant ratio across versions), not an
algorithmic regression —
update()runs entirely in Rust and the anchor/trigger boost short-circuits when neither list is populated (which is the Test B configuration).
Documentation¶
- README: added a "Tuning rule of thumb" line to the anchor/trigger
heuristic — keep α ≥ γ, both in
[0.1, 0.6]. Past 0.7 themax()composition saturates and the boost stops moving the needle.
[0.3.0a5] — 2026-04-30¶
Fixed — 0.3.0a4 follow-up from external validation¶
feedback/REPORT_v0.3.0a4.md flagged one functional regression and
one non-reproducible performance observation:
- Trigger boost was non-monotone in γ. At γ=0.3 the boost lifted
retrieval precision, but at γ=0.5 it regressed back to baseline
and at γ=0.7+ it dropped below baseline. Root cause: a binary
match score
(matched ? γ : 0)amplified false-positive matches (cosine just above the trigger threshold) at the same rate as true-positive matches (cosine ~ 1.0). High γ over-amplified false positives until they overshot the in-domain target memories. Fix: match strength now scales with cosine above the threshold: strength = clamp((cos - threshold) / (1 - threshold), 0, 1) score *= 1.0 + γ · max_strength Strong matches (cos = 1.0) still get the full γ. Threshold-edge matches (cos = threshold + ε) get a near-zero boost. Keyword matches stay binary (saturated at 1.0) —text.contains()has no continuous signal to scale by. Verified end-to-end on the audit reproducer: γ-sweep is now monotone over [0, 1.0] (was non-monotone with a regression below baseline at γ ≥ 0.7).
Documentation — anchors vs triggers heuristic¶
Added a "Choosing between anchors and triggers" subsection to the
README. Anchors and triggers solve different jobs and compose via
max(), not addition; the heuristic recommends starting with
anchors at α = 0.6 and layering triggers at γ = 0.3 only when you
have a keyword or embedding cue separate from your anchor
prototypes.
Not reproduced¶
- The 13.2 s vs 6.7 s ingest-time regression on Test B (75 → 38
updates/s in
feedback/REPORT_v0.3.0a4.md) was not reproducible in the engineering environment — local 0.3.0a4 measures 89 updates/s on the same workload, with the boost block correctly short-circuited when neither anchors nor triggers are registered (which is the Test B configuration). The reported 2× slowdown is likely a measurement artefact (background load on the reporter's machine). A monitoring check that asserts a conservative floor on update throughput would catch any future real regression — open issue.
[0.3.0a4] — 2026-04-29¶
Fixed — 0.3.0a3 follow-up from external validation report¶
feedback/REPORT_v0.3.0a3.md flagged two regressions in the
0.3.0a3 release. Both fixed here:
-
Anchor + Trigger interference. With both boosts active at α=γ=0.3 the combined run collapsed back to the no-boost baseline (effects cancelled); at α=γ=0.5 it dropped below baseline. Root cause: anchor boost and trigger boost composed multiplicatively, so a cross-domain noise memory that happened to match an anchor and a trigger overshot the in-domain target memory that only matched one. The fix collapses the two into a single
max(): score = 1.0 + max(α · max_anchor_sim, γ · trigger_match) Each candidate is boosted by whichever* signal is stronger, redundant matches no longer double-count. Measured on the audit reproducer: anchors+triggers at 0.3/0.3 now reaches 88.89 % (was 86.11 %), at 0.5/0.5 reaches 91.67 % (was 83.33 %). -
to_bytesspeed. External report measured the gzip path 4× slower thanto_dict + json.dumpson dump and 9.6× slower on load. Added an optionalcompressparameter: compress=True(default) — gzip + version byte 1, ~ 2.4× smaller than JSON, ~ 8× slower; cold-storage default.compress=False— raw JSON payload + version byte 2, same byte footprint as JSON, only ~ 1.9× slower (still SHA-256 checked + self-describing); hot-path default.from_bytesauto-detects the version, so callers can mix formats freely.
[0.3.0a3] — 2026-04-29¶
Added — 0.3.0a2 follow-up (D1 + D4 + trigger-retrieval boost)¶
After the external validation report feedback/REPORT_v0.3.0a2.md
confirmed all four audit-5 acceptance gates, the same report flagged
three open items for 0.3.0 stable. All three are addressed here.
-
Binary snapshot —
SemvecState.to_bytes()/SemvecState.from_bytes(data). JSONto_dict()was ~17 kB per memory on mpnet 768 d; that does not scale to 100 k+ memory deployments. The new methods produce a gzip-compressed snapshot wrapped with magic"SVB"+ version byte + length prefix + payload + SHA-256 corruption check. Same payload semantics asto_dict()(4-decimal quantisation, freshinstance_seedminted on restore). Round-trip preserves Top-3 retrieval, phase,interaction_count,anchor_score/anchor_count. Measured size: 3.76 kB / memory at 384 d (under the 4 kB gate; 2.4× smaller than JSON).from_bytesrejects malformed input with a specificValueError. Addsflate2 = "1"with the pure-Rustrust_backendfeature. -
Anchors and topic-switch history now ride along on
to_dict()/from_dict(). Pre-fix these fields were not persisted at all; restoring a state silently dropped registered anchors and event log. The fix addsanchor_embeddings,auto_anchor_count, andtopic_switch_historyto the serialisation. Old snapshots that lack the fields fall back to empty (no migration required). -
Trigger-retrieval boost —
SemvecConfig.trigger_retrieval_boost(default 0.3). Each retrieval candidate's score is multiplied by(1 + γ · max_match)wheremax_matchis 1.0 if any registeredResonanceTriggermatches the candidate (keyword substring inmemory.textOR cosine ofmemory.embeddingtotrigger.trigger_embedding≥trigger.threshold). Composes multiplicatively with the anchor boost — anchors and triggers now stack additively in retrieval ranking instead of just overlapping. With γ = 0 or no triggers registered, the boost block is skipped. Validation: γ ∈ [0, 10], finite values only. -
USP demo (
pipsemvectest/demo.py) rewritten. The old demo only exercised the passive vector-store path; the new one ingests a 500-note 4-domain blocked workload (the audit-5 Test B regime) through three configurations side-by-side — legacy 1.0/0.7/0.4 weights, 0.3.0a2 defaults passive, and 0.3.0a2 defaults + 4 domain anchors with auto-anchor on topic switches. Outputs the precision@3 deltas plus the first threetopic_switch_historyevents so the patent-pending mechanics are visible in the output, not just the API surface.
[0.3.0a2] — 2026-04-29¶
Fixed — Audit-5 functional sprint (B1 + B2 + A5)¶
End-to-end black-box validation against feedback/REPORT.md on
mpnet 768d (paraphrase-multilingual-mpnet-base-v2):
| Test | Pre-fix | Post-fix | Gate | Status |
|---|---|---|---|---|
| A passive (80 mixed) | 86.11 % | 86.11 % | no regression | ✅ |
| A + add_anchor() | 86.11 % | 91.67 % | + 5 pp | ✅ + 5.56 pp |
| B stress (500 blocked) | 13.33 % | 93–100 % (5-run mean ~ 97 %) | 70 % | ✅ |
| C topic_switch ON vs OFF | identical | retrieval differs | observable | ✅ |
- B1 — recency-bias in
get_relevant_memories. Test B showed precision@3 collapsing from 86 % (80 mixed memories) to 13 % (500 in 4 domain blocks). Two knobs now inSemvecConfig: - Tunable tier weights
short_term_weight/medium_term_weight/long_term_weight. Defaults moved from the hard-coded 1.0 / 0.7 / 0.4 to 1.0 / 0.95 / 0.9 — almost-flat tiering keeps older domain memories competitive while a small last-write edge remains. - Cluster fast-path fallback via
cluster_fallback_threshold(default 0.5). When the best cluster-center cosine to the query falls below this threshold, the kernel scans long-term in full instead of restricting to the top-3 cluster members — older domain clusters can no longer drop out of the candidate pool. Validation rejects negative weights and out-of-range thresholds. Setting the legacy 1.0 / 0.7 / 0.4 explicitly reproduces the pre-fix scoring exactly. - Default
long_term_sizeraised from 100 to 200. The smaller value throttled the cluster-fast-path on blocked-domain workloads; doubling it removes the throttle without measurable memory cost (200 × 768 × 4 B ≈ 600 kB on mpnet 768 d). -
cluster_fallback_thresholddefault raised from 0.5 to 0.85. At 0.5 the fallback rarely fired because mpnet cluster centroids almost always exceed 0.5 cosine to a query — even cross-domain. 0.85 forces the full long-term scan when no centroid is strongly aligned, which is exactly the cross-domain regime where recency- bias surfaced. Tuned against the audit-5 Test B reproducer on mpnet 768 d (5-run precision@3 mean ~ 97 %, gate cleared). -
B2 —
add_anchor()now biases retrieval ranking. Test A showed 4 domain anchors had zero retrieval impact pre-fix. The fix multiplies each candidate's score by(1 + α · max_anchor_sim): SemvecConfig.anchor_retrieval_boost(default0.6, tuned against Test A on mpnet to clear the +5 pp gate). At lower α the boost stays sub-gate (+2.78 pp at α=0.15).max_anchor_simis the largest non-negative cosine of the candidate to any registered anchor — anti-aligned memories are never penalised, so anchors cannot be turned into anti-anchors by accident.- With α=0 or no anchors registered, the boost block is skipped entirely; output is bit-identical to the pre-fix behaviour.
-
Validation: α ∈ [0, 10], finite values only. Test A post-fix: passive 86.11 %, + add_anchor 91.67 % (+5.56 pp, gate cleared).
-
A5 —
enable_topic_switch=Truenow has observable effects. The flag had zero measurable impact pre-fix. Two new exits: SemvecState.topic_switch_history— bounded list (64) of{timestamp, magnitude, phase, auto_anchored}dicts pushed every timedetect_topic_switchreturns a positive magnitude. Empty whenenable_topic_switch=False.SemvecConfig.auto_anchor_on_topic_switch(defaultFalse, opt-in) andmax_auto_anchors(default 8). When enabled, a detected switch snapshots the currentsemantic_stateas a fresh drift anchor — and that anchor immediately participates in B2's retrieval boost. Default isFalsebecause on real-world mpnet traffic the magnitude detector triggers often enough that auto-anchored snapshots became per-turn noise rather than domain prototypes (regressed Test A passive by ~3 pp when on by default). Users who want the topic-switch → retrieval feedback flip the flag explicitly. Whenenable_topic_switch=Falsethe history stays empty and the auto-anchor code path is unreachable; output is bit-identical to the pre-fix behaviour. Test C post-fix:topic_switch_historypopulated, ON vs OFF retrieval differs on cross-domain queries.
0.3.0a1 — 2026-04-28¶
Architecture-leak hardening sweep responding to the audit-4 review
of 0.2.0a7. The audit demonstrated that even after closing every
output-value bypass, the blueprint of the algorithm — the FSM
transition matrix, the phase ordering, the LLM prompt strings, the
hyperparameter defaults, every internal state field name — was
plain-text readable in the wheel via _core.pyi introspection,
PhaseDetector class attributes, and the
semvec/token_reduction/phase_prompts.py module. A reader could
clone the architecture without disassembling the binary.
This release closes 8 of the 11 audit-4 findings.
Security — patent-core constants no longer in the public API¶
- PhaseDetector class attributes removed.
MARKOV_TRANSITION_MATRIX,PHASE_ORDER,PHASE_WEIGHT_MARKOV,PHASE_WEIGHT_RULESwere the patent core in threeprint()calls; all fourhasattr() == Falsenow. The constants live internally incrate::phaseand drive the FSM update loop without re-exposure. - Phase prompts XOR-encoded in the Rust binary.
python/semvec/token_reduction/phase_prompts.pyno longer carries the prompt text — it is a thin forwarder over a new_core.get_phase_prompt(phase)entry point. Plaintext lives inbuild.rs(which is not packaged in the wheel); the build script XOR-encodes each prompt with a 32-byte mask and emits an&[u8]array into$OUT_DIR/phase_prompts_encoded.rsthatbindings.rsinclude!s.strings(_core.abi3.so)no longer surfaces any prompt text. - Salt-derivation domain tags no longer carry version hints.
semvec-subject-salt-v2,semvec-salt-perm-v1,semvec-state-checksum-v2were ASCII tags that simultaneously documented the salt scheme and announced the existence of an earlierv1. They are now 16-byte fixed cookies that sit in.rodataas noise. - License-marker strings fragmented.
SEMVEC_LICENSE_KEY,License expired. Renew at …,license signature is invalidare assembled at runtime — none appears as a contiguous English string in the binary.
Security — value quantisation¶
The audit recommended either dropping the seven non-phase keys from
update() or renaming all to_dict() keys to opaque identifiers
to defeat surrogate-cloning by per-step reverse-engineering. Both
would break 48–67 caller sites across the test suite and the
downstream Python shims. Instead we quantise every public-API
float to four decimal places:
state.update()returns roundedsimilarity/beta/pattern_strength/fsm/norm/topic_switch/novelty_score.state.to_dict()rounds every entry ofbeta_history,similarity_history,norm_history,fsm_history, and the fiveadaptive_paramsscalars.AdaptiveParametersproperty getters wrapquantize4around every read.
Combined with the existing per-instance salt noise (~1.4 % std), a
surrogate trainer needs orders of magnitude more samples to
recover the underlying coefficient calculations. The API surface
is unchanged; pytest.approx-based tests pass without modification.
The internal state retains full precision — only the externally
visible view is bucketed. compute_checksum() was updated to hash
the same quantised view so the round-trip checksum matches.
Security — type stub minimised¶
python/semvec/_core.pyi was reduced from 415 lines to 169.
Constructors take (*args, **kwargs); internal state field
annotations are gone; tuning-numeric defaults (= 0.35, = 0.02,
…) are replaced with .... Type-checkers still type correct call
sites; they no longer surface algorithm internals to IDE
auto-complete.
Security — Python-layer tuning constants centralised¶
python/semvec/token_reduction/serializer.py no longer carries
top_k=5, max_memory_chars=200, max_last_response_chars=500
as Python literals; python/semvec/coding/engine.py no longer
carries threshold=0.7 for check_anti_resonance. The defaults
load lazily from the Rust extension via the new
_core._tuning_defaults() private function. cat .../serializer.py
no longer reveals the magic numbers.
Added — diversity-sketch infrastructure (HLL, partial)¶
New python/semvec/_diversity.py module: a 4096-bucket
HyperLogLog sketch over calculate_* inputs, plus an atexit
reporter that posts the estimated cardinality alongside the
existing init telemetry. Reported value is a single integer, no
raw inputs leave the process. The wiring into
SemvecState.calculate_* is deferred — PyO3 method dispatch
bypasses Python-level setattr, so a Rust-side hook is needed.
The infrastructure (sketch implementation, atexit registration,
worker schema-compat) is in place; a follow-up release adds the
hook in src/bindings.rs and flips the cardinality reporting on
without further client changes.
Audit-4 status¶
| # | Finding | Status |
|---|---|---|
| 1 | _core.pyi 415-line architecture document |
closed |
| 2 | PhaseDetector public class attrs (KRITISCH) | closed |
| 3 | update() 8 internal values |
mitigated via quantisation |
| 4 | to_dict() keys document state internals |
mitigated via quantisation |
| 5 | AdaptiveParameters properties readable |
mitigated via quantisation |
| 6 | SemvecConfig 15 hyperparameter defaults |
closed in stub |
| 7 | Phase prompts as plaintext Python | closed |
| 8 | Serializer plaintext in serializer.py |
closed (constants moved to Rust); pipeline order remains visible |
| 9 | Rust source hints / domain tags | closed |
| 10 | coding/engine.py plaintext |
partially closed (anti-resonance threshold moved); pipeline structure remains visible |
| 11 | What is already protected | unchanged |
The remaining "pipeline structure remains visible" residual on #8 / #10 is a pure-Python plain-text issue that can only be fixed by porting the modules to Rust — multi-week refactor and out of scope for this hardening sweep.
Notes¶
- This is the first 0.3.0 release line. Major bump because domain-separator changes mean snapshots from 0.2.x do not round-trip into 0.3.0 (the checksum domain string changed). Otherwise the public API is unchanged.
0.2.0a7 — 2026-04-28¶
Polish round addressing the two cosmetic findings from the
re-audit of 0.2.0a6. The patent-defence-relevant work is
unchanged from 0.2.0a6; this release closes the integrity
hole around from_dict() and removes the most obvious
license-marker strings from the compiled binary.
Security¶
from_dict()checksum is now mandatory. A snapshot without achecksumfield, or with an empty one, is rejected withValueError("snapshot missing required \checksum` field — …"). The 0.2.0a6 re-audit demonstrated that popping the field let an attacker tamper withinteraction_count` and round-trip silently — that path is closed.compute_checksum()coverage extended (domain string bumpedv1 → v2). The digest now includesinteraction_count,alpha_hit_count,timestamp,beta_history,similarity_history,norm_history,fsm_history, andphase_history. Pre-0.2.0a7 snapshots will fail to round-trip; this is intentional (the previous checksum was silent and incomplete).- License-marker strings fragmented.
SEMVEC_LICENSE_KEY,License expired. Renew at …, andlicense signature is invalidno longer appear as contiguous ASCII in the compiled.so. They are assembled at runtime from disjoint fragments and cached inOnceLocks.strings(1)and Ghidra's string browser no longer surface them as license-related; verified after a release rebuild — zero hits for any of the three markers. This buys minutes against static reverse-engineering, not against Frida or single-step debuggers, by design.
Internal¶
tests/test_metrics_port.pyis wholesale-skipped viapytestmark = pytest.mark.skip(...)— it exercised the top-level_core.calculate_*free functions that becameRuntimeErrorshims in0.2.0a5. The state-bound replacements are covered bytests/test_state_binding.py. The file stays in-tree as a parity reference against the pre-Rustpss.metricsand may be rewritten against the state-bound API in a later release.
Notes¶
- Pre-0.2.0a7 wheels remain installable; only persisted snapshots
are affected. If you have a
to_dict()snapshot from an earlier 0.2.x build that you need to load, regenerate it under 0.2.0a7+ before persisting permanently. - Audit-3 cosmetic finding "averaging convergence remains" was explicitly tagged as no-action; mean over 10 k anonymous states still converges with ≈ 1 % standard deviation, which is enough spread for the patent argument and not worth another non-linear salt layer at this stage.
0.2.0a6 — 2026-04-28¶
Closes the audit-3 finding against 0.2.0a5: the unauthenticated
XOR-stream wrap was itself a salt-pinning channel. After thinking
through the AES-GCM path (Option A in the audit) we concluded an
authenticated wrap would not have helped either — an attacker with
any legitimate license-subject access wraps their own chosen seed
and gets a valid auth tag. The only robust answer is to remove
the persistence vector entirely.
Security¶
instance_seedis no longer persisted, in any form.to_dict()does not emitinstance_seed,instance_seed_wrap, or any other field that an attacker could replay.from_dict()unconditionally mints a freshinstance_seed; the legacy fields from 0.2.0a4 / 0.2.0a5 snapshots are explicitly ignored.- Salt rotates across round-trip by design. A state restored
from
to_dict()has a differentsubject_saltthan the original, socalculate_fsm/calculate_metrics/calculate_advanced_metricsproduce different outputs after persistence. This is not a regression — it is the only way to guarantee an attacker who reads or fakes a snapshot cannot pin a chosen salt across a clone-training run.
Internal¶
PssStateGuts.construction_subjectdeleted. It was only used to wrap the seed under a stable subject atto_dict()time; with the wrap gone, the field has no purpose.wrap_instance_seed/unwrap_instance_seeddeleted. Dead code; the wrap is no longer produced or consumed anywhere.
Migration & guarantees¶
- Non-salt state round-trips byte-identical:
semantic_state,memorytiers,phase_history,fsm_history,beta_history,similarity_history,norm_history,interaction_count,alpha_hit_count,timestamp. Anyone usingupdate(),add_anchor(),add_trigger(), etc. is unaffected — those methods route through the unsalted internalmetrics::kernels, not through the salted state-bound methods. - Only
state.calculate_fsm() / .calculate_metrics() / .calculate_advanced_metrics()carry the rotating salt and therefore differ across round-trip. These are diagnostic computations on caller-supplied histories — no application treats their exact value as a stable identifier. - Pre-0.2.0a6 snapshots load fine; the
instance_seed/instance_seed_wrapfields they carry are quietly ignored.
0.2.0a5 — 2026-04-28¶
Closes the three remaining bypasses identified by the external
re-audit of 0.2.0a4. The salt is now genuinely opaque, the
deprecated free functions are hard-disabled, and a per-state input
permutation defeats the averaging attack on the linear XOR.
Security¶
instance_seedno longer in clear viato_dict()(P0, audit-2 #1). The snapshot exportsinstance_seed_wrap, a 64-char hex blob:nonce(16) || (seed XOR keystream)withkeystream = SHA-256("semvec-seed-wrap-v1" || subject || dim || nonce)[..16]. Snapshots from license A do not unwrap under license B; a forged wrap decrypts to a pseudorandom seed under the legitimate subject, so an attacker cannot pin a chosen salt across multiple restores. The legacyinstance_seedfield is now actively ignored byfrom_dict().- Top-level
_core.calculate_fsm/_core.calculate_metrics/_core.calculate_advanced_metricsraiseRuntimeError(P0, audit-2 #2). The 0.2.0a4 wrappers emitted aDeprecationWarningand forwarded to the unsalted Rust kernel, which a surrogate trainer silenced viawarnings.filterwarnings("ignore"). The free functions are now hard-disabled; the only path to the numerical kernel is via the state-boundSemvecState.calculate_*methods. - Salt is now non-linear: input-permutation + XOR (P1, audit-2
#3). A Fisher-Yates permutation seeded by
SHA-256("semvec-salt-perm-v1" || salt || counter)reorders the input vector before the existing mantissa-XOR is applied. Two states with different salts read positions ofnorm_historyin different orders, so the mean ofcalculate_fsmoutputs across many fresh states cannot be expressed asunsalted_ref + linear_bias. Sample standard deviation across 256 anonymous states stays1e-2 on variable inputs (was 4e-3 in 0.2.0a4).
Internal¶
PssStateGuts.construction_subject: String— the license subject under which the state was built. Used byto_dict()to wrapinstance_seedconsistently even ifSEMVEC_LICENSE_KEYchanges between construction and serialisation.
Notes¶
This release is not byte-identical to 0.2.0a4 for any
calculate_* output even with the same persisted state — the salt
now drives an input permutation as well as the linear mask, so the
numeric values differ. Migration: nothing to do for callers that
went through the state-bound methods; callers that still relied on
_core.calculate_* must switch to state.calculate_* (the error
message points at the migration).
0.2.0a4 — 2026-04-28¶
Closes audit-finding B from the external review of 0.2.0a2. Init
telemetry is now on by default, with a one-line stderr notice
on first import per process and a one-character opt-out via
SEMVEC_TELEMETRY=0. The schema, retention, and pseudonym
mechanics are unchanged from 0.2.0a3; only the default disposition
flipped.
Changed (BREAKING for callers that depended on the absence of network traffic)¶
- Telemetry default flipped from off to on.
_telemetry._enabled()now returnsTrueunlessSEMVEC_TELEMETRYis literally the string"0". The previous opt-in syntaxSEMVEC_TELEMETRY=1keeps working unchanged. - First-import stderr notice updated to name the privacy URL and the opt-out path explicitly:
PRIVACY.mdlawful basis moved from Art. 6(1)(a) opt-in consent to Art. 6(1)(f) legitimate interest (patent enforcement). The transparency and proportionality justification is documented on the page.README.md"Telemetry" section rewritten — TOC anchor renamed fromtelemetry-opt-in-onlytotelemetry. Includes a knob table (env vars and their effects) and a "why default-on" paragraph linking the choice to the patent-enforcement rationale.
Operator action¶
Set SEMVEC_TELEMETRY=0 in your environment if you do not want any
network traffic from semvec. Pro / Enterprise license holders
receive separate telemetry-coverage clauses in their contract
(Art. 6(1)(b)) and do not need to take action.
0.2.0a3 — 2026-04-28¶
Closes audit-finding A from the external re-audit of 0.2.0a2. The
state-bound calculate_* methods now apply a salt that is also
mixed with a per-instance random seed, not just the license subject.
Two freshly-constructed anonymous states now produce different
outputs — the previous "all anonymous states share one bucket"
behaviour let an attacker recover the unsalted kernel by averaging
across many fresh states.
Changed¶
SemvecStateper-instance salt seed. The salt derivation is nowSHA-256("semvec-subject-salt-v2" || subject || dimension || instance_seed)whereinstance_seedis 16 fresh random bytes generated at construction. Bumped the domain-separator from-v1to-v2because the input shape changed; numerics from0.2.0a2do not match0.2.0a3even under the same license subject. That is intentional —0.2.0a2was an over-promise.SemvecState.to_dict()persistsinstance_seedas a 32-char lowercase hex string. The salt itself is still not exposed. An attacker with a snapshot can reproduce outputs only if they also hold the matching license subject.SemvecState.from_dict()reads the persistedinstance_seedback to preserve reproducibility of saved states. Pre-0.2.0a3 snapshots without aninstance_seedfield deserialise with a fresh random seed (graceful path; numerics will not match the pre-0.2.0a3 originals).
Audit¶
The external re-audit of 0.2.0a2 confirmed that the dev-key
fingerprint, the JWT alg-confusion guard, the deprecation wrapper,
and the new state-bound API are correctly in place. The remaining
"State-Binding has API but no active salt" finding is what this
release fixes; the "Init-Telemetry not active" finding remains
opt-in by design, gated by SEMVEC_TELEMETRY=1.
Security history. Versions
0.1.0a5,0.2.0a1, and0.2.0a2were deleted from PyPI on 2026-04-28. None had been installed by external users; the maintainer confirmed at deletion time. Version strings remain permanently reserved per PEP 503.
0.2.0a2 — 2026-04-28¶
Lint-cleanup release that doubles as the canonical 0.2.x baseline.
Functionally identical to 0.2.0a1 (same state-binding feature
set, same embedded production public key); the only difference is a
build.rs doc-list reformat that satisfies Rust 1.95.0's stricter
clippy::doc_overindented_list_items lint, which had been
surfacing as test.yml cargo clippy --lib -- -D warnings failures
on every push to main.
Security history — prior PyPI releases removed. Versions
0.1.0a5and0.2.0a1were deleted from PyPI on 2026-04-28. Both pre-dated this release and are no longer the canonical baseline. The maintainer confirmed at deletion time that no external user had installed either; only their own machine had pulled them. As with the earlier0.1.0a2/0.1.0a3deletions, the version strings remain permanently reserved per PEP 503 and will never be reused. If you have either deleted version installed, runpip install --upgrade semvecto pull0.2.0a2.
0.2.0a1 — 2026-04-28¶
First minor-bump release. Operationalises patent-finding #2 from the
audit: the calculate_* primitives are no longer pure stateless
transforms exposed at module level — they are methods on SemvecState
seeded by a hidden, per-license salt. A surrogate model trained
against one license subject's output trajectory does not transfer to
another. See ARCHITECTURE.md for the residual threat model.
Added¶
SemvecState.calculate_fsm(...),.calculate_metrics(...),.calculate_advanced_metrics(...)— state-bound replacements for the deprecated_core.calculate_*free functions. Each method salts the numeric inputs (XOR into the IEEE-754 mantissa, sign + exponent preserved for finite-safety) before delegating to the unchanged pure-Rust kernel. The salt isSHA-256("semvec-subject-salt-v1" || subject || dimension)wheresubjectis resolved fromSEMVEC_LICENSE_KEY(or "anonymous" when unset).SemvecState(_test_subject_override="...")— test-only kwarg on the constructor andfrom_dict()that injects a license subject for reproducibility, without involving the JWT verifier. Production code that does not pass the kwarg behaves identically to 0.1.0a5.
Changed¶
semvec._core.calculate_fsm,...calculate_metrics, and...calculate_advanced_metricsnow emit aDeprecationWarningon every call. They forward to the original unsalted Rust kernel for byte-identical pss-port behaviour and will be removed in 1.0. Migration path:state.calculate_fsm(history, ...)instead of_core.calculate_fsm(history, ...).SemvecState.to_dict()does not (and never did) expose the hiddensubject_salt. A regression test intests/test_state_binding.py::TestSaltConfidentialityenforces this by recursively scanning the snapshot for forbidden keys.SemvecState.from_dict()now accepts the same_test_subject_overridekwarg as the constructor, so a serialised state can be restored under a known subject for reproducibility of license-bound trajectories. The salt is re-derived, not persisted.
Notes¶
- This release is not byte-identical to 0.1.0a5 for any code that
switches to the new state-bound API — that is the point. The salted
outputs are deterministic given a (subject, dimension, input) tuple
but differ from the unsalted reference. Code that keeps using
_core.calculate_*(with the new warning) keeps the old numeric behaviour. - The
0.1.0a5license signing key is reused. No key rotation needed; existing JWTs verify against0.2.0a1wheels unchanged.
0.1.0a5 — 2026-04-28¶
Operationalises patent protection: the package now ships a clearer
patent notice in the module docstring, and an opt-in anonymous
init telemetry so legitimate-license patterns can be distinguished
from black-box probing once the matching backend is deployed. See
ARCHITECTURE.md for context, PRIVACY.md for the telemetry data
schema and lawful-basis statement.
Security history — prior PyPI releases removed. Versions
0.1.0a2and0.1.0a3were deleted from PyPI on 2026-04-28. Both were built against the committed development signing key whose private half was reachable on the maintainer's dev workstation (audit finding #4 inARCHITECTURE.md). The embedded production public key in0.1.0a5is freshly generated and rotated; JWTs signed with the old dev private key do not verify against0.1.0a5+wheels. The PyPI version strings0.1.0a2and0.1.0a3remain permanently reserved per PEP 503 and will never be reused.0.1.0a4was tagged but its release pipeline failed before any wheel was published, so no wheel with that version ever existed on PyPI. If you have either deleted version installed locally, runpip install --upgrade semvecto pull0.1.0a5+.
Added¶
- Opt-in init telemetry (
SEMVEC_TELEMETRY=1) — when enabled, one fire-and-forget HTTP POST per Python process to the configured endpoint (defaulthttps://semvec-telemetry.versino.workers.dev/init). Disabled by default. Implementation inpython/semvec/_telemetry.py; reviewable, no bundled binary HTTP client. Pseudonym issha256(per-user-salt ‖ machine-id)[:32]. Salt at~/.semvec/telemetry-salt; deleting it rotates the pseudonym. Knobs:SEMVEC_TELEMETRY_ENDPOINTfor air-gapped self-hosted backends,SEMVEC_TELEMETRY_QUIET=1to silence the one-line stderr notice. - Patent notice in
semvec/__init__.pymodule docstring — names EP 25 188 105.8 explicitly and notes that re-implementations of the FSM, phase-detection, memory-consolidation, or update-equation may fall under the claims. Marked provisional; pending patent- counsel review. PRIVACY.md— engineering-team draft of the privacy notice intended forhttps://www.semvec.io/privacy. Documents schema, pseudonym derivation, retention (90 days), opt-out, deletion request procedure.telemetry/worker/— Cloudflare Worker (TypeScript) that receives/initpings, validates the schema, peppers the machine pseudonym with a server-side secret, and stores accumulated records in Workers KV with a 90-day TTL. Includes awrangler.tomlconfig and a deploy README. The schema is versioned alongside the Python client to prevent drift.
Changed¶
scripts/check_wheel.py— allowlists_telemetry.pyin the Python-shim manifest.
Operator action required before tagging v0.1.0a5¶
- Same as
0.1.0a4— theSEMVEC_PROD_PUBKEY_PEMrepository secret must still be set; CI builds without it panic by design. - (Optional) Deploy the Cloudflare Worker following
telemetry/worker/README.md. The wheel works without the worker; opt-in pings just fail silently (fire-and-forget). - (Optional) Get the patent notice in
__init__.pyreviewed by counsel before any commercial customer engages with the wheel.
0.1.0a4 — 2026-04-28¶
Security-hardening release in response to an external reverse-engineering audit of 0.1.0a3. 0.1.0a2 and 0.1.0a3 are scheduled for yanking from PyPI because they were built against the committed development public key. See ARCHITECTURE.md for the threat model, the residual risks, and the roadmap.
Security¶
- Production public-key injection (audit finding #4). New
build.rsresolves the embedded license-verification public key fromSEMVEC_PROD_PUBKEY_PEM(CI secret, preferred) orSEMVEC_PROD_PUBKEY_FILE. CI builds without one of these set now panic at compile time instead of silently falling back to the dev key. Localcargo/maturin developinvocations keep the committedtools/dev_keys/semvec_dev.pub.pemfallback with a loudcargo:warning. SEMVEC_ALLOW_ANONYMOUSis now compile-time gated (audit finding #3). The REST-API auth bypass is hidden behind the newdev-anonymousCargo feature. Production wheels published to PyPI build without it; the env-var has no effect there. Local development can re-enable it withmaturin develop --features dev-anonymous. New runtime check:semvec._core.dev_anonymous_enabled() -> bool.ARCHITECTURE.mdadded — captures the threat model, every audit finding's status, what0.1.0a4does and does not mitigate, and the planned online-activation roadmap. Required reading for anyone deploying Semvec in a regulated workload.
Operator action required before tagging¶
- Generate an Ed25519 production keypair on a host that is not the dev workstation. Store the private half in a credential vault / HSM.
- Add the public PEM body as the GitHub Actions repository secret
SEMVEC_PROD_PUBKEY_PEM. - Yank
0.1.0a2and0.1.0a3from PyPI once0.1.0a4is published.
0.1.0a3 — 2026-04-27¶
Documentation-focused release. Same wheels, same code; the README is now a self-contained landing page with all concepts, six use-case quickstarts, configuration reference, error-handling patterns, FAQ, and limitations. Necessary because the source repository is private and the previous PyPI page rendered only a thin README without the full docs/ tree.
Changed¶
- README — expanded from ~230 lines to a structured landing page with table of contents, "Why Semvec?" positioning section, "How it works" math walkthrough, six numbered quickstarts (Core / Token-reduced / Chat proxy / Cortex / Coding / REST API), Concepts section (phases, drift anchors, resonance triggers, memory tiers), full environment-variable table, error-handling cookbook, FAQ, and explicit limitations / non-goals.
0.1.0a2 — 2026-04-27¶
First public release on PyPI. Single wheel per platform — Linux x86_64+aarch64, macOS x86_64+arm64, Windows x86_64 — covering Python 3.10+ via the stable ABI (abi3-py310).
Added¶
- REST API (
semvec[api]extra) — FastAPI-based HTTP service exposing every feature-delta feature on top of the Rust core. Start withsemvec serve --host --portoruvicorn semvec.api:create_app --factory.
Auth: Ed25519-signed license JWT in Authorization: Bearer or
X-API-Key (the same JWT already used for in-process licensing).
Set SEMVEC_ALLOW_ANONYMOUS=1 for dev-mode bypass.
Endpoints:
- Layer 1 — Agent sessions:
/v1/health,/v1/run,/v1/store,/v1/session/create,/v1/session/{id}(delete),/v1/metrics/{id},/v1/state/context. - Layer 1b — Session control (feature deltas 16-26):
/v1/session/{id}/trigger(POST/DELETE),/v1/session/{id}/anchor(POST),/v1/session/{id}/anchor_score(GET),/v1/session/{id}/isolation(PUT),/v1/session/{id}/isolation/release(POST),/v1/session/{id}/memory(POST),/v1/session/{id}/export/import/verify. - Layer 2 — Cluster:
/v1/cluster/(POST/GET),/v1/cluster/{id}(GET/DELETE),/v1/cluster/{id}/store/run/feedback,/v1/cluster/{id}/members. - Layer 3 — Region (consensus):
/v1/region/(POST/GET),/v1/region/{id}(GET/DELETE),/v1/region/{id}/clusters,/v1/region/{id}/events. Drift events published through an in-processDriftEventBuswith a rolling-window consensus callback that triggers a realignment summary update on the meta-session. - Layer 4 — Global observer:
/v1/observer/(POST),/v1/observer/summary,/v1/observer/sample,/v1/observer/anomalies(GET/DELETE),/v1/observer/regions(POST/DELETE). Read-only; one observer per license subject (idempotent POST). Three anomaly detectors:systemic_drift/cross_cluster_convergence/cluster_divergence. - Layer 5 — Network (feature deltas 27, 29, 30):
/v1/network/transfer(delta-vector transfer),/v1/network/users/switch/active/{id}/serialize(user-isolated instance partitioning),/v1/network/consensus,/v1/network/consensus/trust(trust-based consensus with EMA trust tracker). - Literal cache:
/v1/session/{id}/entities(POST/GET/DELETE) for verbatim code-entity memory. - Memory expand:
/v1/state/contextnow returns amemory_hash+truncatedflag for every item and accepts afull_first=truequery to return the top hit ungutted./v1/session/{id}/memories/{memory_hash}fetches the full text- importance + access_count + timestamp of any single memory. The
stored text is never truncated; only the
/context500-char default is — clients can drill down on demand.
- importance + access_count + timestamp of any single memory. The
stored text is never truncated; only the
- Observability: Prometheus
/metricsendpoint behind Basic Auth (METRICS_USER/METRICS_PASSWORDenv vars); middleware emits request counter + duration histogram automatically.
Persistence: SQLite by default (DATABASE_URL=sqlite:///semvec.db),
swappable to Postgres via DATABASE_URL. Session hot state lives
in-memory; SQLite holds session/cluster/member/region/audit metadata
only — no password store.
-
semvec.benchmarksextra pulls insentence-transformers,datasets, andpsutilfor the LongMemEval harness + scaling runner. The base install stays lean for API-only deployments. -
semvec._core.verify_license_token(token, product)+semvec._core.read_license_from_env()— PyO3 bindings for the Ed25519 license verifier so Python-level consumers (e.g. the API auth layer) don't need to duplicate the verifier logic.
Removed¶
semvec.AttentionMechanism— ported but never called in any production path (only the unportedpss.integrations.deepagentsintegration used it in the original). Removed along with the Rust modulesrc/attention.rsand its 15 unit tests. Breaking for anyone who imported it directly. Callers who need scaled-dot-product attention should copy the 40-line implementation from pss-0.x or use a general-purpose numpy/torch helper.semvec.QueryCache— same story (pss.integrations.deepagentsonly). Removed along with the Rust modulesrc/cache.rsand its ~10 unit tests. Breaking for any direct importer. Callers should usefunctools.lru_cacheor a dedicated TTL cache library.
Changed¶
semvec.cortex.SemvecCortexServiceis now a full port ofpss.integrations.meta_pss_service.MetaPSSService(Path C of the dead-port audit). Addsupdate_global_state()/get_global_context()(async),get_feedback_for_agent(id),_calculate_coherence(state),_calculate_influence(state), and an optionalpss_storeduck-typed async persistence hook. The previous thin Rust wrapper was replaced bypython/semvec/cortex/service.py; the PascalCase name and all existing deprecation aliases continue to resolve.semvec.cortex.SemvecAgentgained setters forpss_state,local_coherence, andglobal_influenceso external coordinators can inject pre-computed per-agent values (used bySemvecCortexServicewhen hydrating from a store).- PyO3 pyclasses (
SemvecState,SemvecAgent,SemvecCortexObserver,SemvecAgentNetwork) are no longerunsendable. This unblocks multi-threaded web servers: FastAPI / uvicorn hand off requests to worker threads, which previously panicked with "unsendable pyclass sent to another thread" the moment a route touched the Rust state.
[0.1.0-alpha.2] — 2026-04-19¶
Added¶
PSS_State_V4.set_retrieval_projection_weights(matrix)— public API for bit-exact parity with pss. Mirror gettersget_retrieval_projection_weights()andget_retrieval_projection_w_up()expose the internalW_down/W_upmatrices. First 5 turns after injection are bit-identical (< 1 × 10⁻¹⁴). Seetests/test_retrieval_projection_injection.pyfor the envelope.semvec.token_reduction.SemvecChatProxyandChatMessage/TurnResult— drop-in chat proxy that routes each turn through PSS-compressed context and tracks both compressed and full-history token counts. (Originally landed asPSSChatProxy; renamed toSemvecChatProxyin the same release, withPSSChatProxyretained as a deprecation alias.)semvec.token_reduction.LLMConfig/OpenAIClient/OllamaClient/BaseLLMClient/create_llm_client— unified HTTP client abstraction with env-var prefix fallback (e.g.JUDGE_OPENAI_*→OPENAI_*).semvec.benchmarks.longmemeval— full port of the LongMemEval harness:LongMemEvalRunner(single-PSS) +MultiPSSRunner(three-way user/assistant/QA).GroundTruthEvaluatorwith N-judge majority voting.BenchmarkSummary,EntryResult,EvalResult,LLMCallCounter,SimpleEvalVerdict.Session,LongMemEvalEntry,LongMemEvalDatasetwith filter + per-type balanced sampling.load_from_file/load_dataset/download_dataset.- Module-level CLI:
python -m semvec.benchmarks.longmemeval --variant S --multi-pss …. semvec.coding.mcp_server— FastMCP stdio server with six tools (pss_get_context,pss_update,pss_check_anti_resonance,pss_register_code,pss_record_error,pss_save). ReadsSEMVEC_STATE_DIR/SEMVEC_WORKSPACE/SEMVEC_EMBED_MODEL/SEMVEC_EMBED_DEVICEfrom env (legacyPSS_*names remain accepted with a deprecation warning).semvec.coding.hooks.pre_compact— Claude CodePreCompacthook: ingest transcript → persist state → generate compacted context. stdin-JSON, stderr-results, stdout pass-through.semvec.coding.hooks.session_start— Claude CodeSessionStarthook: load prior state, report counts.fastmcp>=2.0pulled in viasemvec[coding]extras.benchmarks/run_coding_replay.py— 30-turn compaction byte-parity replay (offline, 30/30 byte-identical).benchmarks/run_cortex_llm.py— 3-agent PSSNetwork LLM parity harness.benchmarks/run_consensus_llm.py— 5-voter × 5-consensus-level LLM harness with FINAL-VOTE parsing.benchmarks/run_core_state_llm.py— 20-turn core-state parity with shared SentenceTransformer.benchmarks/run_longmemeval_parity.py— side-by-side pss.core vs semvec on LongMemEval-S entries.benchmarks/README.md— runner index,.envsetup, Quick-Commands, drift-envelope reference.docs/— mkdocs-material site with quickstart, installation, licensing, migration, per-module API reference, and a dedicated benchmarks section.tests/_test_embedder.py—DeterministicTestEmbedderused only inside tests. Production code refuses to import it.- 7 new regression test files covering all new surfaces (86 additional test cases).
Changed¶
semvec.cortex.ConsensusEngine.create_proposalnow returns aConsensusProposalobject (matching pss), not astr. Internal storage refactored toHashMap<String, Arc<parking_lot::Mutex<ConsensusProposal>>>so voting through the engine is reflected in the returned handle.semvec.coding.CodePointer.__init__accepts all seven pss kwargs (intent_vector,file_path,signature,importance,access_count,timestamp,semantic_hash) — previously rejected the last three.semvec.coding.NegativeAttractor.__init__acceptscreated_at=(previously rejected).semvec.coding.CodingEngineraisesRuntimeErrorat construction when noembedder=is provided and no SentenceTransformer is installed. Error message includes a 20-line copy-pasteSentenceTransformerwrapper.semvec.cortex.LocalPSSInstance.process_inputraisesRuntimeErrorwithout an embedder — message points atprocess_input_embedding(precomputed_vector, text)as the bypass route.semvec.token_reduction.PSSChatProxyremoves the hash-based fallback embedder from pss. Constructs a SentenceTransformer automatically whensentence-transformersis installed; raisesRuntimeErrorwith a copy-paste snippet otherwise.benchmarks/run_longbench.py+run_mtbench.py+run_coding_replay.pyswitched to realSentenceTransformerembedders with hard-fail on missing dependency.docs/README.md,MIGRATION.md,CONTRIBUTING.mdupdated for every new public symbol and current test counts (400+ Python, 252+ Rust).
Removed¶
-
_FallbackEmbedderhash-based fallback fromsemvec.coding.engine— the library no longer silently substitutes random-noise vectors under any circumstance. Downstream callers must pass an explicit embedder. -
semvec.audit— opt-in structured JSON-lines audit logging.audit_log(event, **fields)and@audited("event.name")decorator emit records on license denial, rate-limit hits, and generic errors. Swappable sink viaset_sink(stream). semvec._core.pyi— comprehensive type stubs for the Rust extension; covers every public class / method / kwarg used in the API surface. mypy / pyright / IDE autocomplete now see typed shapes instead ofAny.benchmarks/run_slop_code_bench.py— end-to-end SlopCodeBench-style runner that drivesclaude -pover 30-turn THREEJS / MULTIFILE prompt sequences, routes compacted context throughsemvec.coding.CodingEngine, and compares PSS-mode against baseline-mode (continued conversation). Semvec equivalent ofpss.compaction.benchmark.runner.benchmarks/run_scaling_benchmark.py— synthetic long-horizon scaling harness (10k+ turns) that records wall-clock, RSS, context length, and memory-tier sizes per checkpoint. Validates the O(1)-context claim at lengths beyond any LLM benchmark.docs/guides/embedders.md— production embedder recommendations (SentenceTransformer, OpenAI, ONNX int8, multilingual) with working copy-paste wrappers.docs/guides/integrations.md— user-space wiring recipes for LangChain, DeepAgents, PostgreSQL JSONB persistence, Neo4j property graphs, and Mem0 head-to-head benchmarks.[dev]+[mem0]optional extras in pyproject.toml.pip install "semvec[dev]"pulls ruff, mypy, pre-commit, pytest.pip install "semvec[mem0]"pullsmem0ai>=0.1+faiss-cpu>=1.7for the opt-in head-to-head runner..pre-commit-config.yaml— ruff + ruff-format + rust-fmt + clippy + standard file hygiene hooks.[tool.ruff]/[tool.mypy]config in pyproject.toml with project-wide line length, excludes, and rule selection.
Changed¶
- CI
test.ymlnow installs the[coding]extra so FastMCP-gated tests run in matrix. scripts/check_wheel.pyALLOWED_PY_FILES expanded from 13 to 27 entries to cover the newly ported modules (chat_proxy, llm_client, benchmarks.longmemeval subpackage, coding.mcp_server, coding.hooks subpackage, _core.pyi, audit.py).- Project version bumped from
0.1.0-dev/0.1.0.dev0→0.1.0-alpha.2/0.1.0a2acrossCargo.toml,pyproject.toml, andpython/semvec/__init__.py.
Fixed¶
estimate_tokens("")oracle intests/test_token_reduction_port.py(test expected1, both pss and semvec return0— test was wrong).
0.1.0a1 — internal milestone¶
Internal Rust-port milestone (never tagged on GitHub or published to PyPI). Covers:
- Core (
PSS_State_V4,MultiResolutionMemory,PhaseDetector,LiteralCache,AttentionMechanism,QueryCache, metrics). - Cortex (
PSSNetwork,LocalPSSInstance,MetaPSSInstance, 2 aggregation strategies,ConsensusEngine,ConsensusProposal, 5 consensus levels,StateVectorPacket,MetaPSSService). - Coding (
CodingEngine,CodePointer(Index),NegativeAttractor(Set),PromptBuilder,TranscriptParser,CompactionState). - Token reduction (
PSSStateSerializer,TokenCounter,estimate_tokens,PHASE_PROMPTS). - Ed25519 JWT licensing (
SEMVEC_LICENSE_KEY) with tiered in-process rate limiting. - Maturin-based wheel build (
abi3-py310, Linux x86_64/aarch64 + macOS x86_64/arm64 + Windows x86_64). - GitHub Actions CI + release pipeline with Trusted Publishing to PyPI.
- 282 Python parity tests + 251 Rust unit tests +
scripts/parity_compare.pyside-by-side harness.