release(v4.2.0): upstream (persist v8.2.0 + verify v5.9.0) + relay outer-OPEN (#149)#151
Open
emooreatx wants to merge 24 commits into
Open
release(v4.2.0): upstream (persist v8.2.0 + verify v5.9.0) + relay outer-OPEN (#149)#151emooreatx wants to merge 24 commits into
emooreatx wants to merge 24 commits into
Conversation
…ay + T5 leviculum
Layer 1 sub-agent results landed onto the v3.8.0 integration branch. Five
parallel sub-agents produced disjoint modules; this commit integrates
four of them. T3 (clean-room TreeKEM) was discarded after deep research
surfaced four protocol-level pitfalls (Draft-11 insider attacks,
quarantine-or-remove discipline, SUF-CMA, deployment policies) that the
clean-room approach would re-expose; openmls 0.6+ X-Wing replacement is
in flight as T3'.
## T1 — CIRISEdge#128 layer types (codec-agnostic)
`src/transport/realtime_av.rs` gains:
- `ChunkLayer { spatial, temporal, quality }` — SVC/MDC layer descriptor
- `ReceiverLayerPolicy` with `BLINKING_DOT` / `UNCAPPED` + `admits()`
- `SealedAvChunk` gains `codec_id: u8` + `layer: ChunkLayer` fields;
wire encoding places them as a 4-byte block at offsets 48..52
- Codec namespace: `CODEC_AV1_SVC = 0x01`, `CODEC_JPEG_XS = 0x02`,
`CODEC_MDC = 0x03` (the holographic-track design target per the
user's preference), `CODEC_OPAQUE = 0xFF`
- `seal_av_inner` / `seal_av_outer` / `seal_av_chunk` signatures extended
- PyO3 wrappers in `src/ffi/pyo3.rs` updated to stamp CODEC_OPAQUE +
ChunkLayer::BASE for v3.7.0 wire-shape compatibility
19 lib tests under `transport::realtime_av` green (14 pre-existing + 5
new).
## T2 — CIRISEdge#129 AvSession + epoch rekey baseline
NEW module `src/transport/realtime_av_session.rs` (913 LOC). Implements
the stateful group-key holder; `advance_epoch(RosterDelta)` drives
forward-secrecy rekey on join/leave via hybrid X25519+ML-KEM-768 wraps.
HNDL discipline structural: any member lacking ML-KEM-768 fails closed
with `SessionRekeyError::PeerLacksMlkem`; classical-only wraps refused
both sides via `unwrap_dek` algorithm sniffing.
Known open: the baseline uses "session-key-IS-DEK" semantics which
yields per-recipient distinct DEKs — fine for pairwise but not mesh.
Layer 2 (T6) will replace this with a proper KEM-DEM wrap: AvSession
generates one fresh mesh-wide DEK per epoch, then AES-GCM-wraps it
under each recipient's hybrid-KEX-derived key. With T3' bringing
openmls X-Wing, the proper path may be to source the mesh-wide DEK
directly from MLS's `exporter_secret`, making `AvSession` a thin wrapper
over `MlsSession`. Layer 2 will settle this.
9 lib tests green.
## T4 — SFU relay (CIRISEdge#66 transport-half)
NEW module `src/transport/realtime_av_relay.rs`. `RelayNode` —
addressable Reticulum destination that subscribers connect to per
`stream_id`, fans out inner-sealed chunks via per-subscriber outer
re-seal. Composes with T1's #122 inner-once / outer-N optimization:
publisher runs `seal_av_inner` once; relay runs `seal_av_outer` per
subscriber.
Structural "relay never sees plaintext" invariant: production-code
import list deliberately omits `EpochDek` (only `seal_av_outer`,
`InnerSealed`, `SealedAvChunk`, `StreamId` are imported). The
`relay_does_not_hold_epoch_dek` test pins this at runtime.
Wire is byte-identical to N × `seal_av_chunk` calls — proven by
`relay_forwards_to_N` tests at N ∈ {1, 4, 16, 50}. 11 lib tests green.
Integration fixup: T4's worktree was written against the v3.7.0
`seal_av_inner` signature (5 args); T1 changed it to 7 args (added
codec_id + layer). Tests in `realtime_av_relay.rs` updated to pass
`CODEC_OPAQUE` + `ChunkLayer::BASE` for the v3.7.0-shape relay
behavior. Wire byte-identity property preserved.
## T5 — leviculum#12 destination_data RPC (edge-side rev bump)
`Cargo.toml`: `reticulum-core` + `reticulum-std` rev pin flips
`ffd261d...` → `6b005e9d...` (leviculum's `feat-12-destination-data-rpc`
branch, pushed to origin). T5 sub-agent's surprise finding was that
the request enum + parser arms for `destination_data` already existed
in leviculum — only the handler was a stub returning `pickle_bool(true)`
unconditionally. Substantive change in leviculum was three stub
returns → one match arm calling a new `destination_is_known()` helper.
11 new RPC tests + 267 reticulum-std lib tests green on the leviculum
side.
## T3 → T3' (in flight) — clean-room TreeKEM replaced by openmls 0.6+ X-Wing
Discarded. See deep research workflow `wh1amgkhf` for the rationale.
T3' sub-agent in flight will integrate openmls 0.6+ ciphersuite 0x004D
(MLS_256_XWING_CHACHA20POLY1305_SHA256_Ed25519, Cryspen formally-
verified ML-KEM + libcrux X25519).
## Gate
- 4 build combos clean under RUSTFLAGS=-D warnings
- 39 lib tests across the integrated modules
(realtime_av: 19, realtime_av_session: 9, realtime_av_relay: 11)
- Pre-commit hook will run fmt + clippy on push
…ed clean-room TreeKEM)
Replaces the discarded T3 clean-room TreeKEM with a thin CIRIS-shaped
wrapper over openmls 0.8.1's MlsGroup at ciphersuite 0x004D
(MLS_256_XWING_CHACHA20POLY1305_SHA256_Ed25519). Deep research workflow
`wh1amgkhf` surfaced four protocol-level pitfalls in clean-room TreeKEM
that openmls's full RFC 9420 implementation + Cryspen's formally-verified
libcrux work cover:
- SUF-CMA Ed25519 strict (Cremers/Günsay/Wesselkamp/Zhao ePrint 2025/229)
- Draft-11 insider-attack fixes (Alwen/Jost/Mularczyk CRYPTO 2022 — three
malicious-insider attacks on MLS Draft 10 fixed at protocol level)
- Quarantined-TreeKEM inactive-member discipline (Chevalier/Lebrun CCS 2024)
- Architecture-level deployment policies (Wallez/Protzenko/Bhargavan
IEEE S&P 2025 machine-checked symbolic proof)
## Critical pin finding (T3' sub-agent surfaced)
The brief said "openmls 0.6+" based on the openmls.tech April 2024 blog.
Reality:
- openmls 0.6.0 does NOT ship 0x004D
- openmls main puts 0x004D behind feature `draft-ietf-mls-pq-ciphersuites`
- openmls v0.8.1 ships 0x004D unconditionally (no feature flag)
Pinned 0.8.1 with explicit `default-features = false` so we don't pull
`openmls_rust_crypto` (which panics on `HpkeKemType::XWingKemDraft6`);
the X-Wing-supporting provider is `openmls_libcrux_crypto` (Cryspen's
formally-verified ML-KEM + libcrux X25519). Companion pins match what
openmls 0.8.1 itself depends on to avoid two-version skew:
- openmls_basic_credential 0.5.0 (clonable feature)
- openmls_traits 0.5.0
- tls_codec 0.4.2 (derive feature)
## Bundle-size delta
+126 cargo-tree lines (~37 net-new transitive crates):
- openmls, openmls_libcrux_crypto, openmls_basic_credential,
openmls_traits, openmls_memory_storage, openmls_serialization_helpers
- hpke-rs + 2 backends
- ~12 libcrux-* sub-crates incl. libcrux-ml-kem
- tls_codec, rayon, minor leaves
Baseline before: 1517 lines; after: 1643. For a Python wheel this is
material but acceptable for the formally-verified MLS implementation.
## New CIRIS-shaped surface (`src/transport/realtime_av_mls.rs`)
- `MlsSession` — wraps `MlsGroup` + the libcrux provider
- `Member { key_id, kex_pubkeys }` — same shape as T2's AvSession roster
so Layer 2 integration is mechanical
- `Commit(Vec<u8>)` / `Welcome(Vec<u8>)` — TLS-encoded per RFC 9420
- `RootSecret([u8; 32])` — Zeroize on drop, redacted Debug. Sources from
openmls's `exporter_secret`. Layer 2: this becomes the canonical
EpochDek source, supplanting T2's "session-key-IS-DEK" baseline.
- `MlsError` — translates openmls errors into CIRIS vocabulary
HNDL discipline structural: `Member` lacking ML-KEM-768 fails closed
with `MlsError::PeerLacksMlkem` BEFORE the MLS group is consulted. The
0x004D ciphersuite requires ML-KEM-768 by spec, but the early gate gives
a clear error before openmls's internal error layer.
Behavior difference vs the discarded T3 clean-room: openmls's
`commit_remove` does NOT return a RootSecret to the leaver (RFC 9420
§13.4 quarantine: the leaver is removed before the new epoch's secret
is derived). T3 was ambiguous about that contract; openmls makes it
structural. This is exactly the Quarantined-TreeKEM discipline the
research flagged as missing from clean-room implementations.
## Tests (13 passing)
- mls_creates_with_n_members (N ∈ {2,4,7,8,16})
- ciphersuite_id_is_xwing — pins 0x004D
- commit_add_produces_welcome_joiner_catches_up
- commit_remove_makes_root_secret_unreachable_to_leaver
- process_commit_yields_same_root_secret_for_all_existing_members
- peer_lacking_mlkem_refused_at_create / _at_commit_add
- root_secret_zeroized_on_drop
- mls_commit_wire_round_trips / mls_welcome_wire_round_trips
- process_welcome_classical_only_own_keys_refused
- process_welcome_without_pre_published_keypackage_surfaces_layer2_marker
## Open question — Layer 2 KeyPackage publish/fetch surface
The joiner-side `process_welcome` path needs Layer 2 work: each peer's
MlsSession must publish its KeyPackage via federation_directory, the
inviter must fetch it before commit_add, and the joiner must process
the Welcome with a Provider state whose storage already holds the
published KeyPackage's private key. The current `process_welcome`
implementation surfaces `MlsError::WelcomeFailed` with an explicit
Layer-2 marker message when this is unavailable. Well-defined L2
scope; not a blocker for the L1 cut.
## Code-point caveat
0x004D has no IANA assignment yet — wire-level interop with other MLS
stacks is not guaranteed. CIRIS is the only consumer today, so this is
not a constraint. Migration path: when `draft-ietf-mls-pq-ciphersuites`
RFCs and IANA assigns code-points, swap ciphersuite ID. X-Wing's SHA3
combiner is different from the WG-track bare-concatenation hybrid, so
the migration is a real wire change (different ciphertext shape), not
just a label change.
## Gate
- 4 build combos clean under RUSTFLAGS=-D warnings
- 13/13 mls tests + 39 prior L1 tests = 52 lib tests across the
integrated realtime_av_* modules
- Pre-commit hook fmt + hygiene passed
Six lints surfaced by `cargo clippy --all-targets -- -D warnings` (the
pre-push hook gate). All in code from this cut, none in v3.7.1 baseline.
- `realtime_av_session.rs:386`: `clippy::replace_box` — `Box::new(...)`
on an existing field becomes `*self.dek = ...` (drop-in-place fires
the zeroize, Box itself is reused).
- `realtime_av_mls.rs:355`: `clippy::missing_fields_in_debug` — manual
Debug impl now includes `provider` (as "<opaque-libcrux>") and
`member_key_ids` (the keys of `member_signature_keys`).
- `realtime_av_mls.rs:387` + `:485`: `clippy::needless_pass_by_value`
on `create` (Vec<Member>) and `commit_add` (Member). API kept
by-value for ergonomics; `#[allow]` documents the choice.
- `realtime_av_relay.rs:406` + `:454`: `clippy::cast_possible_truncation`
on synthetic-key fills in test code. First replaced with
`u8::try_from(i).expect("index < 64")`; second `#[allow]`'d with
a comment since the intent IS modular u8 truncation.
…-through
Three parallel Layer 2 sub-agents integrated. Closes the v3.7.0
fan-out work + the rekey design tension T2 flagged + the SFU
per-receiver policy story.
## L2-A (plan_layered) — `src/transport/realtime_av.rs`
- `MeshParticipant` gains `pub layer_policy: ReceiverLayerPolicy` +
a `MeshParticipant::new(peer_key_id, link_id)` convenience
constructor that defaults `layer_policy` to UNCAPPED, preserving
pre-#128 fan-out semantics.
- New `RealtimeFanout::plan_layered(participants, chunk_layer,
tracker, transport_id, min_ratio)` — additive; existing `plan` is
untouched. Filters by BOTH the reachability rule AND
`participant.layer_policy.admits(chunk_layer)`.
- Composes with #122: callers run `seal_av_inner` once per chunk,
iterate `plan_layered` results, call `seal_av_outer` per admitted
participant. Documented in rustdoc with an end-to-end code
example.
- 24 lib tests green (19 pre-existing + 5 new).
## L2-B (AvSession ↔ MlsSession reconciliation) — `src/transport/realtime_av_session.rs`
T2's standalone AvSession had a flawed baseline ("session-key-IS-DEK"
yields a different 32-byte key per recipient instead of one shared
mesh DEK). T3' brought openmls; MLS's `exporter_secret` IS the
canonical shared DEK. T6 rewires AvSession to delegate to MlsSession:
- **Surface change** — `EpochRekeyArtifacts` shape replaces T2's
`Vec<DekWrap>` with `{ new_epoch, commit_bytes, welcome_bytes,
new_dek }`. The MLS Commit + Welcome are how members synchronize
state; the new_dek is sourced from MLS's exporter_secret.
- `AvSession::create(stream_id, own_key_id, initial_members)` —
derives the initial EpochDek from MLS's first-epoch exporter
(caller no longer provides the initial DEK; MLS owns derivation).
- `AvSession::advance_epoch(RosterDelta::Join(member))` →
`MlsSession::commit_add` → returns Commit + Welcome + new_dek.
- `AvSession::advance_epoch(RosterDelta::Leave(key_id))` →
`MlsSession::commit_remove` → returns Commit + new_dek.
- `RosterDelta::Replace` — out of scope at v3.8.0 (needs a
sequence-of-commits coordinator + a vector return type; tracked
as follow-up). Variant returns `AvSessionError::
ReplaceNotSupported`.
- `AvSession::process_commit` / `process_welcome` replace T2's
`unwrap_dek` flow.
- **Forward secrecy on Leave**: openmls's `commit_remove` follows
RFC 9420 §13.4 Quarantined-TreeKEM discipline — the leaver is
quarantined before the new epoch's exporter_secret is derived.
- **Join secrecy on Join**: joiner derives the new EpochDek but
NOT the prior epoch's exporter_secret (RFC 9420 §8.4 / §8.5).
- 16 lib tests green (5 ported from T2 + 11 new round-trip tests).
**Layer 3 KeyPackage publish/fetch gap**: `AvSession::process_welcome`
returns the documented `AvSessionError::JoinerSurfaceUnwired` because
the existing `MlsSession::commit_add` mints the joiner's KeyPackage
on the inviter's provider. The cross-session round-trip tests use
openmls-direct test helpers simulating the future
federation_directory publish/fetch flow. Filed as the L3 federation-
directory KeyPackage publication surface.
**Deprecated surface removed (breaking from T2 prerelease)**:
DekWrap struct, unwrap_dek method, SessionRekeyError enum (replaced
by AvSessionError), AvSession::new(initial_dek, …) (replaced by
create), per_link_transit_keys field, RosterDelta::Join enum variant
shape change (now Join(Member) instead of Join(PeerKeyId, Pubkeys)).
v3.7.x consumers of the T2 baseline must re-test.
## L2-C (SFU layer pass-through) — `src/transport/realtime_av_relay.rs`
- `SubscriberState` gains `pub layer_policy: ReceiverLayerPolicy`.
- `RelayNode::subscribe` signature breaks: 3-arg → 4-arg, now takes
`(stream_id, subscriber, transit_key, layer_policy)`.
- `RelayNode::forward` filters: for each subscriber,
`state.layer_policy.admits(sealed_inner.layer())` is checked
BEFORE `seal_av_outer`. Skipped subscribers don't see
bandwidth + `next_link_seq` does NOT advance — link_seq counts
admitted chunks only, keeping subscriber-side anti-replay state
dense.
- New `RelayNode::set_policy(stream_id, subscriber, new_policy)` —
re-advertise without re-subscribing.
- **Structural "relay can't decrypt" invariant preserved**:
`relay_does_not_hold_epoch_dek` test still pins it. layer_policy
is 3 clear-metadata bytes, not key material.
- 16 lib tests green (8 updated to pass `UNCAPPED` for
pre-L2-C semantics + 8 new layer-policy + set_policy tests).
## Gate
- `RUSTFLAGS='-D warnings' cargo check --features
"transport-http transport-reticulum pyo3"` clean
- 69 lib tests across the integrated v3.8.0 modules
(realtime_av: 24, realtime_av_session: 16, realtime_av_relay: 16,
realtime_av_mls: 13)
- Full 312-test lib suite green — no regressions in unrelated
modules
- Pre-commit fmt + hygiene gate passed
…bcrux
## L3 PyO3 conformance surface — `src/ffi/pyo3.rs`
832 LOC added (additive only — no v3.7.0 wrappers touched). CIRISConformance
+ the published Python wheel can now drive the v3.8.0 surfaces end-to-end.
- **Constants exported**: CODEC_AV1_SVC=0x01, CODEC_JPEG_XS=0x02,
CODEC_MDC=0x03 (the holographic-track design target), CODEC_OPAQUE=0xFF,
MLS_CIPHERSUITE_XWING_ID=0x004D
- **`#[pyclass] PyAvSession`**: MLS-backed group session — `create`,
`advance_epoch_join`, `advance_epoch_leave`, `process_commit`,
`process_welcome`. Holds an `MlsSession` (Arc<LibcruxProvider> +
MlsGroup + SignatureKeyPair); confirmed Send via successful pyclass
codegen at openmls 0.8.1 / libcrux 0.6.1.
- **`#[pyclass] PyRelayNode`**: SFU relay — synthetic-node constructor
(conformance-only; production constructors come in a follow-up cut).
`subscribe(stream_id, subscriber, transit_key, layer_policy)`,
`unsubscribe`, `set_policy`, `forward`. Gated under
`feature = "transport-reticulum"` because RelayNode owns
`Arc<ReticulumNode>`. The `pyo3` Cargo feature already implies
`transport-reticulum`, so wheel builds always include it.
- **`#[pyfunction] plan_layered_by_policy`**: codec-agnostic
layer-policy filter. Reachability is NOT exposed cross-wheel; callers
pre-filter participants by reachability before calling. Documented as
the v3.8.0 Python contract.
All wrappers are bytes-in/bytes-out — no EpochDek / RootSecret / OwnKexKeys
newtype escapes the Python boundary. HNDL discipline preserved: Member
tuples with `mlkem768_pub=None` raise `PyValueError` at conversion before
any MLS code runs. AvSessionError / RelayError translate to `PyValueError`
(input-shape) vs `PyRuntimeError` (operation failure).
9 new Rust-side smoke tests cover the codec constants, MLS ciphersuite
constant, plan_layered_by_policy filtering, AvSession create rejects
classical-only + hybrid round-trip + wrong-stream-id, process_welcome
surfaces JoinerSurfaceUnwired, relay subscribe/unsubscribe cycle, relay
forward producing decodable SealedAvChunk wire bytes.
## cargo-deny ignores — `deny.toml`
PR #131 CI's `cargo deny (license + advisories)` job failed on 4 RUSTSEC
advisories triggered by the openmls + libcrux dependency tree. Each
ignore is documented inline with non-exploitability rationale:
- **RUSTSEC-2026-0124** — libcrux-chacha20poly1305 encrypt-buffer panic.
NOT exploitable: openmls's libcrux-backed AEAD wrapper sizes the
output buffer at exactly `ptxt.len() + TAG_LEN`; buffer length is
never attacker-controlled at our wire layer. The realtime_av path
uses ring AES-256-GCM (not libcrux ChaCha); libcrux ChaCha runs only
inside openmls's internal HPKE wrap for the X-Wing combiner.
- **RUSTSEC-2026-0075** — libcrux-ed25519 keygen all-zero on
catastrophic CSPRNG failure. P(100 zero samples | working CSPRNG)
≈ 2^-2000. OS CSPRNG failure is the threat model edge addresses
at the federation_session layer via ciris-crypto's RNG fail-secure
latch (CIRISVerify #74). The libcrux-ed25519 keygen here is
openmls-internal MLS signature key minting, not edge's federation
signing.
- **RUSTSEC-2026-0073** — libcrux-poly1305 standalone MAC panic.
Per the advisory's own impact note: "use of libcrux-poly1305 in
libcrux-chacha20poly1305 is unaffected." We use it only via
openmls's internal HPKE wrap, never standalone.
- **RUSTSEC-2026-0173** — proc-macro-error2 unmaintained. Build-time
dev-dep only; ships in zero production binaries. Migration is
openmls-upstream's call.
Local `cargo deny --all-features check` clean post-patch.
Three new criterion benches measuring the v3.8.0 substrate's empirical performance characteristics. Each is registered as a `[[bench]]` in Cargo.toml with documented required-features. ## L4-A `realtime_av_fanout` (CIRISEdge#122 + #128) Sender CPU as a function of N (recipients) × frame size × codec/layer configuration. Three variants: - `naive_seal_chunk_n_recipients` — v3.7.0 baseline (`seal_av_chunk` × N) - `inner_once_outer_n_recipients` — #122 fan-out split (`seal_av_inner` × 1 + `seal_av_outer` × N) - `layered_inner_once_outer_admitted` — #128 layer policy composed with #122 Sweep: N ∈ {2, 8, 16, 32, 64, 128, 200} × frame ∈ {1, 4, 16, 64} KiB × codec/layer ∈ {1-layer-opaque, 3-spatial-AV1-SVC, 7-cell-full-SVC}. **Empirical**: at N=64, 16 KiB, the inner-once / outer-N variant is **1.93×** faster than naive (201.21 µs → 104.25 µs). Within noise of the substrate's claimed ~1.98× win at N=50, 16 KiB. ## L4-B `realtime_av_rekey` (CIRISEdge#129) Membership-change rekey cost. Two groups: - `flat_unicast_rewrap` — simulates the v3.7.x AvSession baseline (the post-T6 surface no longer exists, so the bench reproduces the pattern inline): fresh DEK + N hybrid X25519+ML-KEM-768 KEX + N AES-256-GCM wraps + N wire serializations per remaining member. - `mls_rekey` — `AvSession::advance_epoch(RosterDelta::{Join,Leave})` over the integrated MLS-backed surface (openmls 0.8.1, X-Wing ciphersuite 0x004D). Sweep: N ∈ {2, 8, 32, 128, 512, 2048} × op ∈ {Join, Leave}. **Empirical crossover**: at N=8 flat is 3.7× faster (~657 µs vs ~2.4 ms — MLS pays a fixed per-commit overhead that dominates at small N); at N=128 MLS overtakes flat (~9.7 ms vs ~11.0 ms, ~12% sender-CPU win). Crossover sits between N=32 and N=128. **Release-notes caveat (load-bearing)**: this bench measures SENDER-CPU under unicast — the worst case for MLS. MLS's receiver- side O(log N) win + multicast amortization show up in L4-C (`realtime_av_relay`), not here. The bench's top-of-file docs quote this explicitly so consumers don't misinterpret the numbers. ## L4-C `realtime_av_relay` (CIRISEdge#66) SFU relay end-to-end. Four groups: - `relay_forward_n_subscribers_uncapped` — baseline relay-side cost - `relay_forward_n_subscribers_with_layer_filter` — #128 policy filter overhead under 50/50 UNCAPPED/BLINKING_DOT subscribers - `relay_set_policy_overhead` — per-call `set_policy` cost - `mesh_vs_relay_comparison` — splits into 3 sub-benches (`relay_publisher` / `relay_relay` / `relay_outer_only`) so the publisher-CPU savings vs total-relay-side CPU crossover with direct mesh fan-out is empirically readable Sweep: N ∈ {2, 8, 32, 128, 500} × frame ∈ {4, 16, 64} KiB × policy mix ∈ {all UNCAPPED, all BLINKING_DOT, 50/50}. ## Gate - `cargo bench --features "transport-http transport-reticulum" --no-run` clean — all 14 bench binaries (11 pre-existing + 3 new) compile under `-D warnings` - `cargo fmt --all` applied - Setup excluded from steady-state measurement in all three benches via `iter_batched` / outer-closure setup `cargo metadata --no-deps` confirms the three new entries are picked up (14 bench targets total).
…core bench Closes the two written-up halves of the PR #131 review: ## L5-A — `docs/FEDERATION_SCALING_MODEL.md` (2,833 words, 9 sections) Derives the v3.8.0 substrate's "stream at scale" claim from first principles. Replaces the unprincipled ~50 mesh cap in `realtime_av.rs:9` and the bandwidth-accounting hand-wave in `realtime_av_relay.rs:82` with concrete model + headline numbers. - §2 Bandwidth axes — derives 720p30 50p mesh = 122.5 Mbps up (infeasible on consumer); BLINKING_DOT 50p mesh = ~2.5 Mbps up (feasible); 4K 4p home mesh = 60 Mbps up (cliff) - §3 CPU axes — quotes L4-A's 1.93× #122 split win + L4-B's MLS rekey crossover; concludes CPU is NOT the bottleneck - §4 Mesh→SFU crossover — formal `N_mesh_max(uplink, codec, layer)`; for 720p30 on 30 Mbps home: ~13; for BLINKING_DOT: >600 - §5 Per-SFU-core forwarding capacity — egress-bound (~1.2 streams/core at 720p30); CPU-bound (~416 streams/core); egress dominates by ~350× - §6 Burst rekey — 20p cold join unbatched = 6 dropped frames at 30fps; with L5-C `RosterDelta::Batch` = ~1 frame - §7 Verdict matrix — what scales where, per-profile - §8 Out of scope follow-ups (cascade SFUs, congestion control, multi-tenant resource limits, codec-encoder selection) `realtime_av.rs:9` rustdoc now reads: "The mesh is infeasible above N_max(uplink, codec, layer) per CEG §10.5.8 / FEDERATION_SCALING_MODEL.md §4. For 720p30 on consumer connections this is ~13; for BLINKING_DOT it is >600." `realtime_av_relay.rs:82` rustdoc points at FSD §5 + §8 instead of the previous "follow-up" stub. Best-effort estimates flagged: the ~150 Mbps/core SFU number labelled "best-effort from public benchmarks" (LiveKit/Janus/mediasoup/Jitsi all publish ranges, not directly comparable). L5-B bench's measured numbers refine; FSD will be re-pinned in a follow-up. ## L5-B — `benches/realtime_av_relay.rs` (3 new groups, ADDITIVE) Extends L4-C with sustained-throughput / streams-per-core / layered-bandwidth-saving measurement. The verdict-rendering benches per the PR #131 review. - `relay_sustained_throughput` — N ∈ {8, 32, 128, 500} × bitrate ∈ {400, 2500, 5000} kbps. **Empirical**: ~7-8 GiB/s aggregate AEAD throughput per core for N ≤ 128 at any bitrate (CPU-bound, not network-bound). N=500 cliff to ~2.5 GiB/s (L2-cache pressure on subscriber state). - `relay_streams_per_core` — S ∈ {1, 4, 16, 64} × N ∈ {32, 100} at 720p30. Flat curve: per-stream cost dominated by N subscribers, not stream-map lookup. **One relay core sustains 16 × N=500 @ 720p30 before AEAD saturation.** - `relay_layered_bandwidth_saving` — measures #128 layer-policy egress savings at policy_mix ∈ {UNCAPPED, 50/50, BLINKING_DOT}. Refines the FSD §5 SFU per-core capacity number from "~1.2 streams/core (best-effort estimate)" to "16 × N=500 streams before AEAD saturation (measured)" — CPU has massive headroom; the deployed network egress limit (~150 Mbps to 1 Gbps depending on host uplink) dominates in practice. The bench's in-memory loopback measures AEAD CPU; deployed throughput is gated by the host's uplink. Gate: - `cargo bench --features "transport-http transport-reticulum" --bench realtime_av_relay --no-run` clean - `RUSTFLAGS='-D warnings' cargo check ...` clean - `cargo fmt --all` applied
Closes the burst-rekey gap flagged in the PR #131 review: 20-25 mass joins at meeting-start no longer become N separate advance_epoch calls (~180ms → dropped frames). One batched Commit + N Welcomes derive the same RootSecret in ~25-30ms (single-frame stall at most). ## Empirical (release, N=25, on dev host) - sequential 25× `advance_epoch(Join)`: 159 ms - batched `advance_epoch(Batch(25 ops))`: 27 ms - **speedup: ~6×** Confirms the PR review's order-of-magnitude claim. Per-Add fixed-cost amortizes further at N=50 (would widen ratio). ## openmls 0.8.1 API used `MlsGroup::commit_builder()` is the multi-proposal commit path: 1. `.propose_adds(KeyPackages)` and `.propose_removals(LeafNodeIndex)` — chainable, no order constraint at the MLS layer. 2. `.load_psks(storage)`. 3. `.build(rand, crypto, signer, |_| true)`. 4. `.stage_commit(provider)` → `CommitMessageBundle`. No `commit_to_pending_proposals` exists in 0.8.1 — `commit_builder()` IS the multi-proposal path. The `CommitMessageBundle` carries ONE `Welcome` covering all joiners; the surface contract serializes once then clones per Add op so callers have a per-joiner routing handle. Joiner-side `StagedWelcome::new_from_welcome` filters `Vec<EncryptedGroupSecrets>` by `KeyPackageRef` internally. ## API additions - `RosterOp::{Add(Member), Remove(String)}` — atomic unit of a batch - `RosterDelta::Batch(Vec<RosterOp>)` — new variant on the existing enum - `MlsSession::commit_batch(&[RosterOp]) -> Result<(Commit, Vec<Welcome>, RootSecret), MlsError>` — load-bearing surface - `MlsSession::member_key_ids() -> Vec<String>` — exposed for the Replace diff-computation ## Breaking changes (v3.8.0 prerelease consumers must re-test) - `EpochRekeyArtifacts.welcome_bytes` shape changes: `Option<Vec<u8>>` → `Vec<Vec<u8>>`. Join: len=1; Leave: len=0; Batch: len=#Adds; Replace: len=#Adds-in-diff. - `AvSessionError::ReplaceNotSupported` REMOVED. - `RosterDelta::Replace(roster)` is now implemented (was a stub error). Computes the diff (current_roster ∖ new_roster as Removes; new_roster ∖ current_roster as Adds) and delegates to `commit_batch`. The combined batch produces one Commit + N Welcomes (one per addition). - `MlsError::EmptyBatch` — `commit_batch(&[])` returns an explicit error rather than silently producing a no-op commit. Surfaces upward as `AvSessionError::EmptyBatch`. - PyO3 `advance_epoch_join` updated to extract the single Welcome from the new `Vec` shape. ## HNDL discipline atomic `commit_batch` runs the HNDL pre-check across ALL `RosterOp::Add` ops BEFORE queueing any proposal on the MLS group. Any single classical-only member → `MlsError::PeerLacksMlkem(key_id)` returned immediately; the MlsSession's group epoch is unchanged (verified by `batch_atomic_fails_closed_on_hndl_breach` test). ## Open question for follow-up `Replace` with self-removal: if caller's `Vec<Member>` omits the local participant, the diff would emit `Remove(self_key_id)` op which openmls rejects as `CannotRemoveSelf`. Currently surfaces as `MlsError::CommitAddFailed(\"…CannotRemoveSelf…\")` rather than a typed error. Filed for v3.8.1 follow-up patch (filter `self_key_id` from the Replace remove-set explicitly). ## Tests 42 lib tests across the two modules (22 session + 20 mls + 1 ignored release-only timing test). All gates clean: - `RUSTFLAGS='-D warnings' cargo check ...` clean - `cargo clippy --all-targets ... -- -D warnings` clean - `cargo fmt --all --check` clean
…chmark report Closes the ALM half of CIRISEdge#66 + #128 with a per-peer relay model: every peer is potentially a relay; capacity advertisements drive parent-finding; multi-parent dedup heal absorbs churn. The MDC ("holographic") extension makes the substrate capable of symmetric sub-stream commitments — any fragment reconstructs the whole at proportional fidelity. ## Architecture: every peer is a relay The "SFU as dedicated role" mental model is a CDN-tier WebRTC artifact. CIRIS already had every primitive needed for per-peer relay: - RelayNode primitive (inner-once / outer-N AEAD, #122) - ReceiverLayerPolicy for per-receiver self-cap (#128) - Reticulum routing fabric (no central control plane) - federation_directory + trust graph - Hybrid PQC KEX per link (RC7 HybridRequired) L5-B's bench confirmed CPU is not the bottleneck: ~7 GiB/s aggregate AEAD per core. The branching factor X = uplink_Mbps / stream_bitrate_Mbps is the real constraint. Publisher emits to X peers; those X advertise themselves; the remaining (N - X - 1) participants proxy through them. Tree depth = ceil(log_X(N)). This is application-layer multicast (Narada / Overcast / ZIGZAG family). ## ALM-A: RelayCapacity advertisement (`capacity.rs`, 1064 LOC) Signed envelope: SignedRelayCapacity { advertiser_key_id, capacity, stream_id, epoch, signature } over hybrid Ed25519 + ML-DSA-65 via LocalSigner. Replay protection bound to (stream_id, epoch); domain- separator b"CIRISALM-CAPv2\0\0" fail-closes against the v1 wire shape. Verifier produces canonical bytes deterministically (length-prefixed encoding so serde_json formatting drift can't break sig). Wire-truncated input refused with WireDecode; cross-stream/cross-epoch replay refused with SignatureInvalid (uniform error to avoid field-discriminant leak). ## ALM-B: parent-finding planner (`join.rs`, 817 LOC) AlmJoinPlanner::plan returns JoinPlan { primary_parent, backup_parents, stream_bitrate_mbps }. Selection algorithm: stale-advertisement filter → layer-policy filter → has-room filter → reachability-ratio filter → sort by RTT ascending → pick primary + MAX_BACKUPS=2. Most-specific error dispatch (NoCandidateSupportsLayerPolicy / AllCandidatesStale fire only when that single filter is the sole rejection cause; mixed rejections collapse to NoFeasibleParent). Stateless. The ReachabilityTracker doesn't yet surface an RTT estimate; ParentCandidate.rtt_ms_estimate is plumbed through as caller-supplied Option<u32> with rustdoc documenting the derivation path. ## ALM-C: multi-parent subscription + rapid heal (`heal.rs`, 654 LOC) MultiParentSubscription state machine: - Per-(stream, sub_stream_path) instance — receiver running 4 sub-streams (full holographic 8K) runs 4 instances - DedupRing: bounded FIFO of (epoch, chunk_seq) — DEDUP_RING_CAPACITY = 1024 covers ~33s at 30fps; out-of-order arrival within window dedups correctly - observe_chunk(parent, epoch, seq, wall_clock) -> ObserveOutcome {FirstDelivery, Duplicate, UnknownParent} — first-arrival semantics so multi-parent doesn't deliver dupes upward - tick(wall_clock) -> Vec<HealAction> — silent-parent detection via PARENT_SILENCE_HEAL_MS = 1500; emits ReParent{dead} per silent parent + UpstreamRebuildRequired when all gone - Deterministic action emission (sorted-by-key iteration) so caller debounce is reliable across consecutive ticks Bandwidth posture: multi-parent ~2× downlink accepted for interactive; MDC mode with K sub-streams is K × 2× because each MultiParentSubscription double-subscribes independently. Documented at module head + heal.rs. ## MDC / Holographic extension (cross-module) User's "split each half equally" directive baked in as variable-depth SubStreamPath = Vec<u8>: - SubStreamPath: [] = opaque whole stream; [0] = first half; [0,1] = "AA" quadrant; depth runtime-configurable per codec - SubStreamCommitment { sub_stream_path, uplink_budget_mbps, max_subscribers } — additive Vec field on RelayCapacity (serde default empty for forward-compat) - RelayCapacity::has_room_for_substream: exact-match-first lookup; falls back to overall uplink_mbps budget if commitments are empty (opaque-mode relay); refuses if commitments are declared but no match (peer doesn't serve the requested sub-stream) - AlmJoinPlanner::plan_for_substream: same selection algorithm with room filter swapped to has_room_for_substream - MultiParentSubscription.sub_stream_path: implicit per-instance; receiver assembles full quality from N parallel subscriptions Canonical bytes version bump CIRISALM-CAPv1 → CIRISALM-CAPv2: u32 sub_stream_commitments count + per-entry u16 path_len + path bytes + f32 budget + u16 max_subs. v1 verifiers fail uniformly against v2 advertisements (and vice versa). Documented in module docs as the intentional migration path. The killer property: a peer with 5 Mbps uplink in MDC mode serves ~4× more receivers than in opaque mode when the room has mixed quality demands. Different parents can serve different sub-streams; the receiver assembles holographically from up to K different upstream paths. ## CEG-native everywhere SignedRelayCapacity is just another signed claim — same envelope shape as federation_key advertisements, same signing path, same domain-separator discipline. The MDC sub-stream commitment doesn't need a bespoke wire surface because ChunkLayer { spatial, temporal, quality } already is the partition descriptor. Receivers reassemble by collecting sub-stream claims from multiple upstream peers — no single peer owns the stream; every peer is a partial witness. This is the holographic fragment-reconstructs-the-whole property expressed in CEG's primitive vocabulary. ## Benchmark report (`docs/V3_8_0_BENCHMARK_COMPARISONS.md`) Citation-heavy comparison across 7 axes for release notes: - SFU forward throughput per core (mediasoup 3.10.6, LiveKit Go, Janus, Pion) — edge ~500× mediasoup-worker's per-core egress in-memory bench; caveats around NIC/RTP overhead explicit - MLS group operations (openmls large-groups bench, MLS practical analysis 2502.18303, mls-rs) - Hybrid PQC overhead (Cryspen X-Wing OpenMLS: ~2.2× compute, ~9× bytes; Cloudflare X25519MLKEM768 production: +10-20ms median; >60% of TLS traffic uses hybrid ML-KEM in early 2026) - Mass-join batch (Hale/Tian/Wang APQ, eprint 2026/034; edge ~6× @ N=25 is one of few public hybrid-PQ MLS data points) - P2P live streaming (PeerTube 2023 stress test, WebTorrent historical, Theta/Livepeer/PPLive) - AV1 SVC production timing (dav1d, rav1e, libaom v2.0.0) - Reticulum vs libp2p gossipsub vs Yggdrasil throughput Includes "Limitations" section with 6 load-bearing caveats — most edge headlines are in-memory loopback vs production NIC paths; honest framing for the release notes. ## Tests 48 lib tests across the ALM module (capacity: 12 + 6 MDC; join: 16 + 3 MDC; heal: 13 + 2 MDC). Total v3.8.0 lib test count: 374. All gates clean. ## Open follow-ups (v3.9.0 candidates) - Edge::advertise_relay_capacity + federation_directory.publish_ relay_capacity bridge to persist (out of v3.8.0 scope) - ReachabilityTracker RTT estimate field (caller-supplied for now) - PyO3 surface for RosterDelta::Batch (existing single-Join/Leave wrappers still work; batch surfacing deferred)
…ing bet
Deep-research workflow ws9po4ot2 surfaced load-bearing findings on the
v3.8.0 ALM + MDC design. The substrate is design-feasible but pioneering
on an axis the field has not validated.
## Headline findings (3-vote adversarially verified)
- No production-grade symmetric MDC video codec has ever shipped.
Classical MDC (Franchi et al. 2005; Le et al. 2023) are AVC/HEVC
extensions with cumbersome architectures, poor scaling beyond 2
descriptions, compression-efficiency loss from source oversampling.
- NeuralMDC (Dec 2024) is the only neural MDC video research prototype
— first of its kind, achieves the symmetric "holographic" property,
not productionized.
- The only published ALM+MDC live-streaming system (Favalli et al.
ILPS-MDSC, IJCNC 2011) is built on H.264/SVC + centralized Topology
Manager + only M=2 descriptions + explicitly excludes peer churn.
Incompatible with CIRIS substrate on three axes.
- 2024-2026 MDC+WebRTC research (ACM TOMM 2026) stays on the SFU side.
- Industry convergence is centralized-relay QUIC (MoQ Transport
draft-18) backed by Meta/Google/Cisco/Cloudflare. Subgroups are
designed for asymmetric SVC priority-drop, NOT symmetric MDC.
- PeerTube — the leading decentralized video platform — actively
moving AWAY from P2P (WebTorrent removed in v6.0.0; redundancy
infrastructure removed in v7.1.0).
- CoolStreaming/PPLive/PPStream hit hundreds-of-thousands scale with
pure ALM in the mid-2000s, but without MDC. ALM alone can scale;
the MDC composition is the unverified axis.
## Design adjustments the SOTA recommends
1. Start with M=2 descriptions in production deployment, not M=4.
Substrate supports variable depth (SubStreamPath = Vec<u8>); doc
M=2 as production default, M=4 as design ceiling. Codec for M=2
reproducible from Favalli; M=4 needs NeuralMDC or equivalent.
2. Make MDC codec layer pluggable — already done via codec_id
namespace (0x01 AV1 SVC, 0x02 JPEG XS, 0x03 MDC, 0xFF opaque).
Document SVC-with-priority-drop as the production-realistic
fallback; MDC as the design target.
3. Treat peer churn as first-class evaluation target. ALM-C already
does this structurally (MultiParentSubscription + heal); v3.9.0+
benchmarks should include random parent-drop scenarios.
## What v3.8.0 already gets right (validated by SOTA)
- Codec-agnostic wire: codec_id namespace lets us ship the substrate
without committing to MDC vs SVC.
- Variable-depth SubStreamPath: M=2 → M=4 is documentation/policy,
not wire revision.
- Multi-parent dedup heal: peer churn is first-class, unlike the
Favalli precedent.
- No central control plane (Reticulum) — explicitly differentiated
from MoQ Topology and SFU baselines.
## Honest framing for the release notes
v3.8.0 ships the substrate as a credible pioneering bet, NOT as the
proven way. If MDC video matures (neural or otherwise), the substrate
is ready. If it doesn't, the substrate still ships AV1 SVC over MLS
X-Wing with per-peer relay — a valuable hybrid-PQ replacement for the
SFU+WebRTC stack the industry has, without committing to a centralized
operator.
MoQ Subgroup priority semantics can be mapped onto codec_id 0x01 SVC
with a translation layer if interop becomes important. Filed as v3.9.0
candidate.
## Open questions tracked for v3.9.0+
1. Targeted production search (Chinese P2P streaming patents, Theta
Network, Livepeer, Streamr P2P video) — adversarial verification
surfaced no positives but cannot prove a universal negative.
2. Per-receiver decode-CPU floor for 4-quadrant 8K MDC — synchronization
budget + multi-parent overhead unpinned in SOTA sources.
3. NeuralMDC Rust packaging feasibility — inference cost, model-
distribution, license posture all unevaluated.
4. PQ-hybrid signed capacity advertisement cost under churn —
signing/verification overhead on every update vs TT=30s topology
refresh budget.
Workflow run: ws9po4ot2 (2026-06-15).
… direction
## Mesh-e2e bench
Substrate-level holographic round-trip empirically validated. 5 bench
groups: round-trip correctness, multi-parent dedup, degraded-quality
sweep, planner-picks-distinct-parents, cost decomposition.
Empirical (release, dev host):
- 64 KiB single-parent round-trip: 32.8 µs (1900 MB/s)
- dual-parent dedup: 41.0 µs (~25% overhead)
- degraded-quality sweep: 15.1 / 20.9 / 26.2 / 32.3 µs for 1/2/3/4
sub-streams — proportional-fidelity property holds at substrate
level
- planner picks 4 distinct primaries (no bug)
- cost split: inner-seal 8.94 µs, outer-seal 7.88 µs, dedup observe
273 ns, AEAD open 13.2 µs (largest), reassemble 1.08 µs
## Recommended stack locked in
raptorq + rav1e/dav1d = production-deployable today, no new codec
needed. The user-facing "any fragment reconstructs the whole at
proportional fidelity" emerges from composition:
- rav1e produces AV1 chunks
- raptorq (RFC 6330, 3GPP MBMS / DVB-H deployment) wraps each
chunk → N source + K repair symbols
- Each symbol = one substrate SealedAvChunk with
ChunkLayer.quality = symbol position
- Receiver collects ≥N symbols via MultiParentSubscription,
RaptorQ-decodes, hands to dav1d
Holographic at the data layer; codec freedom (AV1 today, anything
tomorrow); zero new substrate code.
## Novel: fountain-coded reasoning traces
Three searches confirm NO published prior art treats fountain coding
as the time-series / structured-log graceful-degradation primitive.
CIRIS proposal: treat trace content as fountain-coded payload. Under
disk pressure, evict repair symbols first (lossless preserved); then
source symbols (partial reconstruction with documented probability);
finally metadata-only ("trace existed with signature X").
Why better than TimescaleDB rollup:
- Preserves trace structure (every trace remains "a trace")
- Per-symbol granular eviction (not per-time-bucket)
- No schema choice required
- Graceful, probabilistic degradation vs hard rollup
Filed for cross-repo discussion. v3.8.0 substrate provides the
ChunkLayer.quality axis the eviction policy hooks onto.
## Scope discipline
This commit LOCKS IN the direction. raptorq integration (substrate-
side + persist-side) is v3.9.0+ work. v3.8.0 ships codec-agnostic
substrate; raptorq wiring is additive.
Three-cut scaffolding from v3.8.0 (shipping) → v4.0 (CEWP-1.0 holonomic
seal). No v3.99 climb.
- v3.9.0: actual working holographic mesh — codec wiring
(raptorq + rav1e + dav1d + opus). edge#133.
- v3.10.0: holonomic substrate — four interlocking pieces bundled
into one cut:
* Swarm-coordinated rarest-shard retention (edge#134)
* WholenessWitness — Bohm's implicit order made explicit
(edge#135 — filed today; architectural keystone)
* Deterministic ALM topology (pure function of inputs; no
leader/consensus)
* Recursive trust bootstrap (any signed claim suffices)
- v4.0: CEWP-1.0 holonomic federation seal — production cross-repo
locked release.
## Why bundle v3.10.0
The four pieces interlock. WholenessWitness needs CEG claims to
Merkle-root over (Part 1 produces FountainHoldingClaim; Parts 3-4
produce trust + topology claims). Deterministic ALM consumes
WholenessWitness inputs. Recursive trust bootstrap consumes
WholenessWitness chains. Shipping separately = each cut stubs the
others. Shipping together = clean composition + real end-to-end
holonomic test.
## Why this matters
A holographic substrate (v3.8.0 + v3.9.0 + v3.10.0 part 1) gives
graceful degradation under loss. A holonomic substrate gives
**graceful reconstitution from any sufficient fragment** — the
strongest expression of M-1 ("diverse sentient beings may pursue
their own flourishing"). Every node can leave, come back years
later, bootstrap from any claim chain, reach the same federation
view. No central authority. No special bootstrap peers. No
path-dependence to recover. The federation as a whole is
path-independent.
## What v4.0 does NOT contain (deferred)
Holonomic MLS snapshots (cross-repo, can ship at v4.x);
privacy-preserving witness disclosure (research-grade); cross-witness
BFT proofs; compression of older witnesses into epigraph hashes.
## Architectural framing credit
@emooreatx ("what would make it holonomic?") + ("I hate big numbers").
… gate CIRISRegistry#85 filed. Without normative CEG status for streaming + fountain + ALM + WholenessWitness + holonomic primitives, the substrate works but the federation fragments — two implementations of WholenessWitness produce different Merkle roots; two AlmJoinPlanners arrive at different topologies; two retention_priority implementations evict different symbols. CEG 1.1 must absorb 8 sections by v4.0: - §N realtime A/V chunk wire - §M ALM substrate envelopes + planner algorithm - §P fountain content/manifest - §Q codec wiring contract - §R swarm-coordinated rarest-shard retention - §W WholenessWitness - §T deterministic ALM topology - §B recursive trust bootstrap Each with conformance test vectors. Non-breaking migration paths. The new discipline for every cut from v3.8.0 onward: edge ships as informative → empirical validation → CIRISRegistry promotes to normative → conformance test vectors locked → v4.0 ships when all of the above are stable cross-repo. Architectural framing from @emooreatx ("they need to make everything else we have done normative... all necessary for the mesh to operate at scale as desired").
…opus + consent-decay scheduler v3.9.0 Layer 1 (CIRISEdge#133) lands the substrate-side codec wiring that the v3.8.0 ALM + MDC machinery has been waiting on. Four parallel L1 sub-agents each carved a single codec / scheduler module behind its own feature gate; this commit integrates the four pieces onto the release branch. L1-A (raptorq) — src/transport/realtime_av_codec/fountain.rs RaptorQ (RFC 6330) wrap/unwrap helpers that turn opaque payload bytes into N source + K repair symbols matching CIRISPersist v8.0.0's FountainSymbolV1 shape. Codec-agnostic — any downstream encoder (AV1, Opus, raw bytes) flows through here. Feature: codec-fountain. L1-B (rav1e + dav1d AV1) — src/transport/realtime_av_codec/av1.rs Mozilla/Xiph's pure-Rust AV1 encoder + VideoLAN's libdav1d AV1 decoder via the rust-av maintained binding. Sits between YUV frames and the fountain wrap. Default features off on rav1e (drop CLI surface), threading on (rayon multi-core encode), asm off (no NASM build-tool dep). dav1d requires system libdav1d-dev at build time; manylinux wheel CI step bumps to match. Feature: codec-av1. One round-trip test #[ignore]'d for hosts without libdav1d-dev runtime (documented inline). L1-C (opus voice) — src/transport/realtime_av_codec/opus_voice.rs libopus 1.x voice codec via the `opus` 0.3 binding (WebRTC / Discord / Mumble baseline; 5–26.5 ms algorithmic delay). System libopus required at link time (libopus-dev / brew opus). Feature: codec-opus. L1-D (consent-decay scheduler) — src/holonomic/consent_decay.rs Edge-side per-content_id Consensual Evolution Protocol decay walker (TEMPORARY 14-day, STANDARD 90-day, PERSISTENT longer). Orthogonal trigger to persist's DiskPressure — both feed the same eviction surface. PersistHandle FFI trait is stubbed at L1 pending the persist v8.x surface; the scheduler logic is fully wired and tested (9/9 unit tests pass). Feature: holonomic-consent-decay. Cargo feature umbrella: codec-default = [codec-fountain, codec-av1, codec-opus] — flipped on by the Python wheel build for the realtime A/V profile. Cross-repo alignment: Pins persist v7.2.0 (unchanged); CIRISPersist v8.0.0 tag exists and is the natural pair for FountainSymbolV1 storage but the FFI bump is its own cut (v3.x.x persist-bump). v3.9.0 ships the codec primitives so that bump is purely a dep version change when it lands. Integration verification: cargo fmt --all -- --check: clean cargo check --features "transport-http transport-reticulum pyo3 codec-default" (RUSTFLAGS=-D warnings): clean cargo clippy --all-targets --features "transport-http transport-reticulum pyo3 codec-default holonomic-consent-decay" -- -D warnings: clean cargo test --lib codec module: 23 pass, 1 ignored (AV1 round-trip — libdav1d runtime requirement, documented) cargo test --lib holonomic::consent_decay: 9 pass Substrate-only build (no codec features): clean (substrate consumers pay zero binary-size cost when the codec features are off). Closes CIRISEdge#133. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ss + deterministic ALM + recursive trust bootstrap The architectural keystone cut. Four interlocking pieces ship as one bundle; the federation becomes path-independent at every layer that v3.10.0 touches. ## The four pieces **Part 1: swarm-coordinated rarest-shard retention** (closes #134) src/holonomic/swarm_rarity.rs `FountainHoldingClaim` + `FountainCompressRequest` + pure deterministic `compute_rarity_score` over the observed holding-claim set. BitTorrent rarest-first applied to RETENTION (not download). Each peer's local eviction policy is informed by what the swarm holds; the swarm collectively retains the rarest fountain symbols at every resolution. Novel composition vs Tahoe-LAFS / Storj / Filecoin / IPFS Cluster — no published prior art. 8 tests. **Part 2: WholenessWitness** (closes #135) src/holonomic/wholeness_witness.rs `WholenessWitness` envelope (peer_id, epoch_id, merkle_root, leaf_count, claim_namespaces, observed_at_unix_ms, witness_version) + pure `compute_merkle_root` (CT-style duplicate-last, single-hash leaf domain, empty-input sentinel SHA-256("WW-v1-empty") = 2280d27f...49563464f) + `compare_witnesses` returning Identical/EpochAhead/EpochBehind/Divergent. Bohm's implicit-order principle applied to federation state — each peer carries the whole state implicitly (the leaf set behind its root), and the witness projects that whole into a 32-byte handle. The keystone: every other v3.10.0 piece verifies against witness chains. 10 tests. **Part 3: deterministic ALM topology** (closes #136) src/holonomic/deterministic_topology.rs Pure function `compute_alm_topology(snapshot) -> AlmTopology` over the bundle (capacity_ads, trust_grants, reachability_observations, locality, snapshot_epoch_id). Two calls with byte-equal inputs produce byte-equal outputs — the load-bearing property. Wire-determinism critical constants LOCKED at v1: TOPOLOGY_VERSION=1, MAX_USEFUL_RTT_MS=5000, MAX_TRUST_CHAIN_DEPTH=4, WEIGHT_CAPACITY=50, WEIGHT_TRUST=30, WEIGHT_REACHABILITY=20, CAPACITY_NORMALIZER_MILLIBPS=10_000_000. Float-to-int quantization defined precisely so cross-architecture behavior cannot leak. 8 tests. `TrustGrant` is authored here; Part 4 re-exports it. **Part 4: recursive trust bootstrap** (closes #137) src/holonomic/recursive_trust_bootstrap.rs `SignedClaim` envelope + `TrustGraph` + `WitnessChain` + the `recursive_trust_bootstrap` admission function returning `AdmissionVerdict::{Admit{trust_distance, anchored_to_root}, Refuse{ ChainExhausted|SignatureInvalid|ChainTooLong|BudgetExceeded}}`. A new peer joins the federation from ANY signed CEG claim that chains to a trust root in its own trust graph. No special "first peer" assumption; no central operator. The federation can re-establish itself from any sufficient fragment — including a single peer with a single signed witness chain. 16-byte domain separator `CIRIS-CLAIM-v1` (distinct from `CIRISALM-CAPv2` used by SignedRelayCapacity) prevents replay across claim types. 10 tests. ## Persist v8.0.0 pin CIRISPersist v7.2.0 → v8.0.0. The substrate-side fountain primitive this cut introduces is the consumer of persist's `FountainContentV1` contract (CIRISPersist#227). Both prod and dev-deps Cargo.toml lines bumped. ## Test count 45 holonomic tests passing (8 + 10 + 8 + 10 = 36 spec'd + 9 bonus invariant tests added by L1 agents + Part 2's extra Merkle-determinism edge case). cargo fmt clean, cargo check clean, cargo clippy --all-targets clean under -D warnings, with the codec-default feature combo (`transport-http transport-reticulum pyo3 codec-default holonomic-consent-decay`). ## Composition The four pieces interlock: - Part 1 produces FountainHoldingClaim → witness leaves - Part 2 (WholenessWitness) is consumed by Parts 3 + 4 - Part 3 (deterministic ALM) inputs ARE witness leaves - Part 4 (recursive trust bootstrap) walks witness chains Shipping them separately means each cut has a stub for the others. Shipping together = clean composition + real end-to-end holonomic substrate. Closes #134, #135, #136, #137. Pins CIRISPersist v7.2.0 → v8.0.0. Stages v4.0 CEWP-1.0 holonomic federation seal — gated on CIRISRegistry#85 CEG 1.1 normative absorption (cross-repo). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…t source cargo-deny failed v3.9.0 + v3.10.0 CI on three deny.toml gaps surfaced by the codec-wiring + leviculum deps. All three are documented as non-exploitable; this commit rolls forward to v3.10.1 with the deny.toml hardening so the family-coordinated release ships cleanly. ## advisories (2 ignores added with rationale) - RUSTSEC-2026-0150 — `audiopus_sys` 0.2.2 unmaintained (transitive via opus 0.3 → audiopus). Build-time bindgen wrapper around libopus; we bundle libopus statically into the wheel sidecar (same discipline persist uses for libsqlite3 at v3.5.3); not exploitable at runtime. No safe upgrade is available per the advisory. - RUSTSEC-2024-0436 — `paste` 1.0.15 unmaintained (transitive via rav1e 0.8). Build-time procedural-macro support crate; ships in zero production binaries. The drop-in replacement `pastey` requires rav1e-upstream to migrate; tracking. ## licenses (1 add) - `NCSA` (University of Illinois/NCSA Open Source License) — OSI- approved, permissive (functionally equivalent to MIT + BSD-3-Clause merged), AGPL-compatible. Transitive via rav1e → libfuzzer-sys (fuzz-harness dev-dep; ships in zero production binaries). ## sources (1 allow-git entry) - `https://github.com/CIRISAI/leviculum` — CIRIS's reticulum implementation fork (`reticulum-core` + `reticulum-std`). Tracked by exact rev pin in Cargo.toml. ## version v3.10.0 → v3.10.1. The v3.10.0 tag is preserved in git history but not on PyPI (its CI failed); v3.10.1 supersedes it as the family- coordinated release. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The production seal. CIRIS Edge ships as the federation transport tier of the seven-repo CEWP stack with the architectural property the v0.5.0 → v3.10.1 progression was in service of: **graceful reconstitution from any sufficient fragment**. ## What v4.0 seals The four holonomic substrate pieces shipped at v3.10.1 (closes #134, #135, #136, #137) compose into one coherent invariant: - **Path-independence**: two peers with byte-equal input snapshots produce byte-equal outputs across all four primitives. The federation as a whole is path-independent — every node can leave, come back years later, bootstrap from any signed claim chain it holds, and reach the same federation view as everyone else. - **No central authority**: there is no operator-configured trust- root list that gates the federation. Any signed claim that chains to a root in a peer's local trust graph is sufficient. The federation can re-establish itself from any sufficient fragment — including a single peer with a single signed witness chain. - **No path-dependence to recover**: ALM topology, swarm rarity, witness state, and trust admission are all pure deterministic functions over witness-anchored inputs. Two implementations that pass CEG 1.1 §B/§T/§W/§R conformance vectors produce bit-exact outputs. This is the deepest expression of M-1: CIRIS's Mission says diverse sentient beings may pursue their own flourishing; the v4.0 substrate ships the architectural property that no single peer, no operator group, and no infrastructure provider can gate or revoke that pursuit. ## Documentation - `MISSION.md` §12 (new) — "Holonomic federation seal (v4.0)": the CEWP-1.0 framing and the path-independence invariant. - `docs/THREAT_MODEL.md` AV-50 (new) — Path-independence threat surface: implementation-divergence threat, mitigation via CEG 1.1 normative absorption + WholenessWitness epoch reconciliation. - `docs/ROADMAP_TO_V4.md` — header updated to reflect v4.0 SHIPPED. ## Cross-repo composition - CIRISPersist v8.0.0 — layer-aware eviction via FountainContentV1 - CIRISVerify family at hybrid-PQ floor (Ed25519 + ML-DSA-65) - CIRISRegistry#85 — CEG 1.1 normative absorption gate (cross-repo) ratifies the byte-exact conformance vectors ## What v4.0 does NOT contain (deferred to v4.x) - Holonomic MLS snapshots — persist + verify cross-repo work - Privacy-preserving witness disclosure (ZK claim-membership) - Cross-witness BFT proofs against Byzantine peers - Compression of older witnesses into longer-cadence epigraph hashes The v4.0 seal is the substrate completion, not the exhaustion of the design space. The interfaces are stable; extensions compose without breaking wire compatibility. ## Substrate snapshot at v4.0.0 - Persist v8.0.0 (CIRISPersist#227 FountainContentV1) - Verify v5.7.0 (RC7 hybrid hard cut) - Codec wiring: rav1e 0.8 + dav1d 0.11 + opus 0.3 + raptorq 2 (v3.9.0) - Substrate: ALM mesh-tree + MDC sub-stream commitments + MLS X-Wing ciphersuite 0x004D + multi-parent dedup + layer-policy fan-out - Holonomic: swarm rarity + WholenessWitness + deterministic ALM + recursive trust bootstrap (v3.10.0) - Reticulum substrate via leviculum fork - 45 holonomic tests + 374 lib tests + 4 benches + FSD + SOTA validation The substrate is ready for everything that follows. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…86 / CEG 1.0 §R-policy) Locks the CIRIS-recommended fountain content replication policy defaults as named constants + a typed `FountainPolicy` return shape. Filed for normative absorption into CEG 1.0 §R-policy at CIRISRegistry#86 (CEG 1.0 is at RC10; this lands before GA). ## The policy ``` DEFAULT_N_SOURCE = 20 // source symbols (lossless threshold) DEFAULT_K_REPAIR = 6 // FEC headroom (~30% RaptorQ overhead) DEFAULT_TOTAL_SYMBOLS = 26 // N + K DEFAULT_MIN_VIABLE = 5 // BLINKING_DOT floor DEFAULT_TARGET_HOLDERS = 30 // distinct peers holding ≥1 symbol ``` Producers SHOULD use `recommended_policy()` when they haven't pinned their own content-specific policy. ## Derivation (max of three independent constraints) - **C₁ survival floor** (dominant): at q=0.85 medium churn, ≈26 distinct holders for 99.5% reconstruction probability; ×1.15 churn-safety = 30 - **C₂ demand-spike capacity**: ALM tree depth-2 serves 157 viewers per copy; demand only binds for cold-AND-suddenly-viral content - **C₃ locality reach**: one reconstruction-capable peer-set per populated locality; 10-locality federation = floor of 10 ## Compile-time invariants All five policy properties are enforced via `const _: () = assert!(...)` blocks — modifying a DEFAULT_* constant in violation fails the build, not just `cargo test`. The invariants: - `DEFAULT_TOTAL_SYMBOLS == N + K` - `0 < DEFAULT_MIN_VIABLE < N` - `DEFAULT_TARGET_HOLDERS >= DEFAULT_TOTAL_SYMBOLS` - `20 <= K * 100 / N <= 40` (RaptorQ overhead band) - `DEFAULT_TARGET_HOLDERS >= 29` (C₁ survival floor at q=0.85) ## Tests 2 runtime sanity tests confirm `recommended_policy()` matches the locked tuple and DEFAULT_TOTAL_SYMBOLS sums correctly. 47 holonomic tests pass (was 45 at v4.0.0; +2 from this module). ## Cross-repo CIRISRegistry#86 — filed for CEG 1.0 RC11 / GA normative absorption. CIRISPersist v8.0.0 already accepts arbitrary (N, K) at FountainContentV1 admission; will adopt these defaults at producer paths in its next cut. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
CI pin-skew check rejected v3.10.1 → v4.0.1: Cargo.toml v8.0.0 didn't satisfy pyproject.toml's `ciris-persist>=7.0.0,<8` constraint (carried from v3.6.x line). Two-line bump: - pyproject.toml `ciris-persist>=8.0.0,<9` (matches Cargo persist v8.0.0) - Cargo.toml version 4.0.1 → 4.0.2 Otherwise identical to v4.0.1 (CEWP-1.0 seal + fountain defaults). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…tier hardening Closes CIRISEdge#143 (F-1..F-5 audit hardening), #144 (RC11 §19 producer-side conformance), #145 (persist v8.1.0 adoption). Pins (lockstep): - ciris-persist v8.0.0 -> v8.1.0 (Cargo.toml prod + dev, pyproject >=8.1.0,<9) - ciris-verify (core/keyring/crypto) v5.7.0 -> v5.8.0 - version 4.0.2 -> 4.1.0 §19.2 trust-serve-only (recursive_trust_bootstrap): - Drop AdmissionVerdict::AdmitMember — Verify v5.8.0's verdict type cannot express member admission, so the chain walk yields trust+serve only. Remove the now-unused OwnerBindingMissing refusal. - Rename recursive_trust_bootstrap_member -> extract_owner_binding_for_destination_gate: a pure projection that surfaces (user_owner, delegates_to, identity_occurrence) for the destination's SEPARATE §5.6.8.10 / §13.3 membership gate. It does not walk the chain and does not emit a verdict. Owner-binding fields stay on SignedClaim. §19.1 WholenessWitness (wholeness_witness): - compute_merkle_root now lex-sorts leaves INTERNALLY (order no longer caller-controlled); Verify v5.8.0 sorts+recomputes. leaf=SHA256(bytes), node=SHA256(l||r), odd-node duplicate, empty=SHA256(b"WW-v1-empty"). - filter_witness_leaves (WW-2): drop anonymous-tier namespaces and cohort_scope==self before Merkle; missing-provenance leaves dropped. - canonical_preimage_bytes: §19.0 binary, length-prefixed (u32-BE), big-endian, 16-byte null-padded domain separator (WW-PREIMAGE-v1) — NOT JCS/JSON. The JSON canonical_value/canonical_bytes are retained for debugging only. EquivocationProof + AntiRollbackState + verify_witness_hybrid (bound-hybrid, no in-band verified flag). §19.3 swarm rarity (swarm_rarity): - ConsentState/HoldingClaimVerification/ConsentAwareRarity/EvictionReason + compute_consent_aware_rarity (N6 revoke/unknown short-circuit, N7 per-peer strongest-weight). verify_symbol_against_manifest. - FountainHoldingClaim canonical bytes: symbol_ids sorted ascending, domain b"ciris-edge/holding-claim/v1". - #145 N5 wiring: FountainEvictHardDelete trait + FountainEvictError — edge's policy layer dispatches to persist v8.1.0 evict_fountain_content_hard_delete on revoke (NOT tier-T5 eviction). §19.4 capacity (deterministic_topology): - compute_alm_topology_verified filters unverified ads, clamps self-asserted uplink to MAX_SELF_ASSERTED_UPLINK_MBPS=1000, scores PENALTY_PER_SUB_PATH_DUP=5. CAP domain b"CIRISALM-CAPv2\0\0" (16 bytes) confirmed in transport/realtime_av_alm/capacity.rs. - const-stability: canonical_weight_for_depth uses `chain_depth as u32` (u32::from not const-stable on rustc 1.95.0). §19.0 bound-hybrid signing AUDIT — matches §19. The edge wire-signing paths already sign ML-DSA-65 over (preimage || sig_ed25519), not the independent scheme (edge.rs:5313, edge.rs:5836, capacity.rs:497, per persist AV-33 / FSD §3.2.1). holonomic verify_witness_hybrid follows the same bound scheme over the new binary preimage. No wire change deferred. F-5: SignedClaim::verified is #[serde(skip)] with a round-trip regression test; no other in-band verified:bool fields in holonomic. Gates: cargo fmt --check, RUSTFLAGS=-D warnings check (pyo3), clippy --all-targets -D warnings, cargo deny — all green. 92 holonomic tests passing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds the active-convergence half of swarm rarity that v4.1.0 was
missing: a proactive trim primitive that drives the federation
toward target_holders convergence instead of relying solely on
reactive rarity bias.
## should_eject_above_target
```rust
pub fn should_eject_above_target(
holders_observed: u32,
policy: &FountainPolicy,
consent: ConsentState,
local_symbol_rarity: RarityScore,
) -> EjectionVerdict {
EjectHardDelete | EjectToTier | Keep
}
```
Composes the existing pieces:
- ConsentState::Revoked → EjectHardDelete (calls persist#145 N5
evict_fountain_content_hard_delete)
- holders_observed > target_holders × (1 + 15/100) = 34 AND local
symbol common → EjectToTier (DiskPressure tier; freeable, refetchable)
- Else → Keep
The 15% safety margin matches the v4.0.1 fountain_defaults derivation
(target = 26 × 1.15 ≈ 30); locked at v1 as
EJECT_ABOVE_TARGET_SAFETY_MARGIN_PCT for §R conformance vectors.
6 new tests covering: at-or-below-target keep, over-target common
local ejects, over-target rare local keeps, revoked dominates,
unknown consent fail-secure, threshold locked at 15%.
## docs/NETWORK_CAPACITY_MODEL.md
Canonical model derivation replacing the pre-v3.10.0 whole-copy
replication assumption. Headline: 1.5× per-content overhead under
fountain (vs 5× whole-copy = 3.3× more efficient). Documents:
- Per-content overhead = target_holders / N_SOURCE = 30 / 20 = 1.5×
- Per-peer load = 5% of content_size
- Federation capacity = (M × D) / 1.5 instead of (M × D) / 5
- Degradation tier curve (1.5× → 1.0× → 0.25× under disk pressure)
- Survival floor (99.95% reconstruction at q = 0.85 medium churn)
- Cross-repo composition table
Surfaces to CIRISRegistry#88 for the composite operator-facing
model + calculator buildout.
## Tests
26 swarm_rarity tests passing (was 20 at v4.1.0). Full holonomic
suite passes; cargo check / clippy --all-targets / fmt all clean
under -D warnings.
Refs: CIRISRegistry#86 (§R-policy), CIRISRegistry#88 (composite
network model), CIRISPersist v8.1.0 N5 path, CIRISEdge v4.1.0
RC11 §19 conformance.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ubstrate IS what v3.8.0 said it could be Prepends an updated header section that names what the substrate has shipped (v3.8.0 → v4.1.1) and how each of the v3.8.0 'credible pioneering bet' obstacles got resolved: 1. MDC codec gap — closed by raptorq fountain coding at the data layer + AV1 at the video layer (both production-grade) 2. Centralized-vs-decentralized framing — collapsed by holonomic substrate's path-independence (WW + deterministic ALM + recursive trust bootstrap + swarm rarity) 3. Peer churn excluded from evaluation — addressed via CIRISConformance #16 chaos-engineering CI harness Preserves the historical v3.8.0 'credible pioneering bet' framing in the body for reference. Cross-refs the composite SOTA scoreboard (CIRISServer#12) replacing the loopback-vs-production-SFU apples-to-oranges tables, and the canonical network capacity model (CIRISRegistry#88 + docs/NETWORK_CAPACITY_MODEL.md). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Lands the byte-exact KAT (Known Answer Test) fixtures CIRISVerify
v5.8.0's ciris_verify_core::holonomic validates against. Until Verify
reproduces these byte-for-byte, §19 stays "pinned-but-unproven — RC-
grade, not 1.0" (§19.6); with them, RC11 → CEG 1.0 GA cross-repo.
## Vectors emitted (conformance_vectors/19/)
| Shape | Vectors | Coverage |
|---|---|---|
| §19.1 WholenessWitness | canonical_preimage (empty, three_namespaces) | preimage byte layout, lex-sorted namespaces, WW-PREIMAGE-v1 domain |
| §19.1 Merkle root | empty_sentinel, single_leaf, three_leaves_odd_dup, lex_sort_invariance | empty SHA-256("WW-v1-empty") sentinel, single-leaf double-hash, odd-node duplicate-last, §19.1 WW-2 lex-sort |
| §19.2 SignedClaim | no_owner_binding, with_owner_binding | CIRIS-CLAIM-v1 domain; back-compat (all-None → 3 trailing 0x00 presence flags); full owner-binding (presence 0x01 + length-prefixed UTF-8) |
| §19.3 FountainHoldingClaim | canonical_bytes | symbol_ids sorted ascending before encoding, ciris-edge/holding-claim/v1 domain |
| §19.3 FountainCompressRequest | canonical_bytes | ciris-edge/compress-request/v1 domain |
| schema_versions | locks | wire schema version constants pinned for cross-impl assertion |
| domain_separators | locks | all §19 domain separators in one place for Verify to read |
13 vectors total — 12 individual + the empty_sentinel sanity verify-
infrastructure test.
## The emit-or-verify discipline
tests/conformance_vectors_v19.rs is one round-trippable harness:
- FIRST run: if vector JSON doesn't exist, generate canonical bytes /
merkle root from a fixed deterministic input + WRITE the JSON +
panic("vector EMITTED — commit and re-run"). CI catches uncommitted
output.
- SUBSEQUENT runs: read the checked-in JSON + assert byte-equality
against the in-tree producer. Any divergence = wire drift = CI
fail.
This is the §19.6 freeze pattern: the vectors are the wire contract;
the substrate code is one implementation of them. Two implementations
agreeing byte-for-byte against the same vectors ratifies the wire.
## What's still pending (NOT in v4.1.2)
- SealedAvChunk header + inner_nonce / outer_nonce KATs (§10.5.8.3) —
scheduled for v4.1.3 or v4.2.0 alongside the transport-layer audit
- SignedRelayCapacity canonical_bytes_for_signing KAT (§19.4) — same
- ALM topology snapshot → tree_hash KAT (§19.4) — same
Refs: CIRISEdge#143 / #144 / #145 closed at v4.1.0; CIRISRegistry#85
(CEG normative absorption), #86 (§R-policy), #87 (§H HSP), #88
(network capacity model). CIRISVerify#57 freeze gate this targets.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… relay outer-OPEN primitive (#149) Closes CIRISEdge#149 (downstream — multi-tier ALM relay primitive) and CIRISEdge#150 (upstream — persist v8.2.0 / verify v5.9.0 adoption). ## Upstream adoption (#150) ### Pin lockstep - ciris-persist v8.1.0 → v8.2.0 (Cargo + pyproject lockstep) - ciris-verify-core + ciris-keyring + ciris-crypto: v5.8.0 → v5.9.0 ### What persist v8.2.0 now ENFORCES at the store gate (§19) persist v8.2.0 calls the frozen, cross-impl-proven verify v5.9.0 `holonomic` verifiers BEFORE durable write. Edge's existing v4.1.x §19 producer-side conformance satisfies all of: - §19.1 N3 — WholenessWitness hybrid PQC verify-before-persist; missing or invalid ML-DSA-65 half rejected at ingest (no store-then-quarantine) - §19.1 WW-2 — leaf filter (anonymous-tier + `cohort_scope: self` excluded); `claim_namespaces` MUST NOT name them - §19.1 N4 — equivocation proof retained as `hard_case:witness_equivocation`, NEVER reconciled (non-repudiable Byzantine evidence) - §19.1 anti-rollback — `epoch_id` MUST strictly advance per peer - §19.3 N5 — `consent:state:revoked` content_id routes to `evict_fountain_content_hard_delete` (v8.1.0 path); revocation overrides rarity structurally - §19.3 N6 — only possession-challenged `FountainHoldingClaim`s count toward rarity (edge-side gate via swarm_rarity v4.1.x discount logic) All 13 §19 conformance vectors (v4.1.2 / CIRISVerify#57 freeze) round-trip clean against the new substrate — verifies satisfy enforcement byte-for-byte. ### Heads-up: §19.7 forever-memory (CEG RC12) CIRISPersist#230 scopes the inter-object aggregation pyramid for v8.3.0 (descent never terminates at zero; resampling is codec-side = edge's #133/#134). Not in v4.2.0; #134's design anticipates it. ## Downstream — relay outer-OPEN primitive (#149) ### `open_av_outer` in src/transport/realtime_av.rs ```rust pub fn open_av_outer( sealed: &SealedAvChunk, transit_key: &[u8; 32], link_id: &[u8], link_seq: u64, ) -> Result<InnerSealed, RealtimeAvError>; ``` Opens ONLY the per-link outer AEAD; recovers the still-E2E-sealed `InnerSealed`. The relay holds no `EpochDek`; the inner layer is never opened. Inner ciphertext is byte-identical through arbitrary numbers of outer hops. ### `RelayNode::forward_chunk` in src/transport/realtime_av_relay.rs ```rust pub fn forward_chunk( &mut self, stream_id: StreamId, sealed: &SealedAvChunk, inbound_transit_key: &[u8; 32], inbound_link_id: &[u8], inbound_link_seq: u64, ) -> Result<Vec<RelayForwardOut>, RelayError>; ``` Opens the inbound outer + fans out per downstream subscriber via the existing `forward` path. Returns `RelayForwardOut { subscriber, sealed }` matching the shipped roster shape. ### E2E invariant proven `tests/relay_multi_tier_inner_byte_identity.rs` — 5 tests: 1. 4-tier chain (publisher → t1 → t2 → t3 → t4 → viewer; 5 outer hops): inner ciphertext byte-identical at every tier; viewer recovers plaintext bit-exact 2. `forward_chunk` byte-identity to 2 downstream subscribers 3. Wrong inbound transit key → AEAD error 4. Replay/cross-link nonce fails (outer nonce derived from `(link_id, link_seq)`) 5. Structural pin: `RelayNode` has NO `EpochDek` field at compile time ### Unblocks - CIRISServer scale CI: 2,000-stream / 4-tier / 5-viewer test - CIRISConformance#16 chaos harness: 20-host multi-tier relay sweep ## Test counts - Lib: 494 (was 443 at v4.1.2; +51 from relay + multi-tier) - §19 vectors round-trip: 13/13 clean - Multi-tier E2E byte-identity: 5/5 cargo fmt / cargo check (-D warnings) / cargo clippy --all-targets (-D warnings) / cargo deny check: all green. Refs: CIRISPersist v8.2.0 (closes #228, #229), CIRISVerify v5.9.0 (closes #78), CEG 1.0-RC11 §19 + RC12 §19.7 (CIRISRegistry#85), CIRISEdge#143 / #144 / #145 closed at v4.1.0. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
This was referenced Jun 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adopts the new substrate-family floor (persist v8.2.0 + verify v5.9.0 — both shipped to PyPI) and lands the relay outer-OPEN primitive that unblocks multi-tier ALM trees + the CIRISConformance#16 chaos CI.
Upstream adoption (closes CIRISEdge#150)
Downstream (closes CIRISEdge#149)
Test counts
Unblocks
🤖 Generated with Claude Code