Changelog

Public changelog rendered from the project’s CHANGELOG.md. Sprint sign-off documents carry the detailed change story; this page is the one-line index.

EPIC — Changelog

All notable changes to EPIC. Follows Semantic Versioning. One-line entries per merged PR, each pointing at the sprint sign-off that closed the work — the detailed change story lives in the sprint document, not here (sprints/CLAUDE.md §4).

Format reference: Keep a Changelog, with section ordering Added / Changed / Deprecated / Removed / Fixed / Security.


[Unreleased]

Phase 10 — Federated sign-in: 🔧 IN PROGRESS — planned 2026-09-14, opened 2026-09-19 at Sprint 10.1. ADR-068 (Proposed) and the six sprint documents under sprints/phase-10/. Documentation only; no code or deployment change.

❄️ SUBSTRATE-CORRECTION FREEZE — FerraVault — 2026-06-01. Sprint 1.7 retracts every prior CHANGELOG framing that describes EPIC's epic-vault adapter as "an HTTP client" / "hyper-based" / "talks to the /v1/secret/{key} endpoint" / "uses the X-Vault-Token header" / "long-polls /v1/revision for hot-reload events". FerraVault does not have any HTTP/REST surface. FerraVault is gRPC-over-TLS-1.3-only via the official ferravault-client tonic crate, with Vault.KvGet / KvPut / KvDelete / Watch (server-streaming) RPCs and bearer-token or mTLS-bind authentication. Sprint 0.5 shipped the wrong substrate (hand-rolled hyper + JSON envelopes against URL paths that don't exist on FerraVault's wire); Sprint 1.3 + Sprint 1.4 extended it; Phase 1 sign-off (phase-1-final, 2026-05-31) and the Ferrum substrate-retrofit tag (phase-1-final-ferrum-retrofit, 2026-05-31) both built on it. The only reason CI shipped green is that epic-vault was exercised exclusively against wiremock stand-ins and the in-memory MemKeyVault fake — no integration test ever booted a real ferravault daemon. Caught 2026-05-31 during the post-Sprint-1.6 cross-project audit; rework lands in Sprint 1.7. ADR-025 ratifies the corrected substrate. The standing project rule is recorded in project_ferravault_is_grpc_only. Until Sprint 1.7 closes, no CHANGELOG entry below that describes epic-vault, VaultClient, the watch path, or any FerraVault-shaped wire as "production-grade" or "complete" stands — the wire never matched the real FerraVault. Sprint 1.7 also bumps the dev-stack PostgreSQL pin from postgres:16-alpine to postgres:18-bookworm per the operator's "Postgres should at least be 18" directive and the ChatXP reference architecture.

❄️ SUBSTRATE-CORRECTION FREEZE — Ferrum — 2026-05-31. ✅ LIFTED 2026-06-01. During Sprint 2.2 close-out the operator caught a structural defect that invalidated a portion of the post-Sprint-0.5 work-product: every Ferrum*-named EPIC adapter (epic-store::ferrum::FerrumClient, FerrumCache, FerrumMeter (Sprint 0.5 + Sprint 2.2 copies), FerrumRateLimiter, FerrumStream, the Sprint 2.2 Aggregator, and every LayeredStore/LayeredSessionLookup that wraps them) was implemented against the Rust redis crate speaking RESP3 to a redis://... URL — but Ferrum does not speak RESP3 at all. Ferrum is its own native data store with its own binary protocol (FBP — Ferrum Binary Protocol — over QUIC or Unix socket) and its own official Rust client (ferrum-client / ferrum-client-tokio). Sprint 0.5 shipped the wrong substrate; Phase 1 sign-off built on it; Sprint 2.2 inherited and amplified it. Resolution: the rework split into Sprint 1.6 — Ferrum substrate retrofit (Phase 1 retrofit, closed 2026-05-31, tag phase-1-final-ferrum-retrofit) + Sprint 2.0 — Ferrum substrate + meter rework (Phase 2 prerequisite, closed 2026-06-01). Both sprints have now closed; every Ferrum* adapter, epic-store::FerrumStream, the epic-meter::FerrumMeter producer, the epic-meter::Aggregator consumer + tick, and the meter integration suite all run against the real Ferrum FBP wire surface (verified by cargo test -p epic-meter --test integration_aggregator — 7/7 against a live ferrum-server child process). The Sprint 2.2 doc remains in the tree as a historical record; the load-bearing sign-off for the epic-router + epic-meter pipeline is Sprint 2.0 §6. See Sprint 2.2 §8 Freeze block for the per-item disposition table and the project_ferrum_is_not_redis memory for the standing rule. Phase 2's Sprint 2.1 (epic-ferron-client) was unaffected — it talks to Ferron (the LLM inference server), not to Ferrum.

Slot purchasing & access levels ✅ COMPLETE 2026-09-19. Tag: slot-purchasing-final (unsigned annotated — commit signing remains indefinitely deferred per root CLAUDE.md §9.1), at b18e5c5, the commit SP.4 and the sign-off landed on main. Named rather than numbered per ADR-064: the next free number is claimed by the federated sign-in plan, so a number here would collide with a phase that has not opened. SP.1 closed 2026-09-14, SP.4 2026-09-16, SP.2 2026-09-17, SP.3 (gating sign-off) 2026-09-19. 34 §2 items across the three production sprints, all ✅ Delivered; zero four-field deferrals. Three audits ✅ PASS: quality (workspace line 88.33 % — above the 80 % floor, above Phase 9's 86.51 %, and the highest recorded here; 2 889 tests / 137 binaries / 0 failed), security, architecture 77/77. Definitive ci.sh --full exit 0 (3 441 passed / 0 failed across 245 suites; 68m 17s; fuzz 6/6; conformance openai + anthropic over both plaintext and TLS; Stage-8 bench gate 0 regressed, 13 stable vs local-3826c6c). Two ADRs Accepted: ADR-067 (superseding four ADR-007 decisions) and ADR-069 (superseding ADR-067 §1 and §3 in part). The thesis: EPIC's commercial model became enforced rather than described — billing was a subscription writing period dates onto the organisation, payment webhooks could never find an organisation, and every signup was FoundingMember with seat_count = 0, so every organisation passed every gate. A slot is now a row with a kind, four rights, a status and an end; each right is decided by its own count; and the edge refuses a key whose account holds no slot carrying API access. What the gate found is the part worth remembering: five real defects, all fixed in-sprint, and three of them share one shape — a check that had been returning the right answer for the wrong reason. The ADR-026 pipeline-invariant command named a file that had stopped containing the import it polices; a dual-stack listener test asserted "both sockets are bound" about a socket that had never bound; and an epic-meter coverage figure had been carried by two prior phase audits as a live-substrate measurement gap when the substrate suites were running all along. The tracker's L1 (live merchant) is deliberately not closed by this phase — no merchant is live.

Phase 2 — Edge MVP, EPIC native ✅ COMPLETE 2026-06-02. Tag: phase-2-final (unsigned annotated — commit signing remains indefinitely deferred per DESIGN.md §20; a signed phase-2-final-signed tag will be created against the same history when signing eventually schedules). Sprint 2.1 closed 2026-05-31. Sprint 2.2 ❄️ FROZEN 2026-05-31 → Superseded by Sprint 2.0 2026-06-01. Sprint 2.0 closed 2026-06-01. Sprint 2.3 closed 2026-06-01. Sprint 2.4 closed 2026-06-02. Sprint 2.5 (gating sign-off) closed 2026-06-02. 67 distinct work-contract items across 5 sprints, all ✅ Delivered. Zero deferrals after the sign-off remediation pass — the original aarch64 link-stage four-field block was retired (root cause was a missing linker = "aarch64-linux-gnu-gcc" in /.cargo/config.toml; both binaries now link clean for aarch64-unknown-linux-gnu and execute under qemu-aarch64-static), and the original "tokenizer byte-parity property test" carry-forward was narrowed to cross-tool byte parity at the shim layer because the remediation pass added an edge-side tokenizer determinism gate covering everything testable inside Phase 2. One cosmetic carry-forward remains (epic-ferron-client/phase-2-final/ baseline rename → next cache-warm window). Three audits ✅ PASS: quality, security, architecture 27/27. Full cargo geiger workspace sweep at sign-off remediation confirms zero unsafe in EPIC-owned src/ except the documented Secret::<T>::into_inner ManuallyDrop discipline in epic-crypto. Five ADRs Accepted/Revised: ADR-009, ADR-010, ADR-011, ADR-021 (revised), ADR-026. Phase 1 — Control plane MVP ✅ COMPLETE 2026-05-31 (originally on substrate-incorrect Sprint 0.5 Ferrum stand-in; corrected via Sprint 1.6 + Sprint 1.7 → tags phase-1-final-ferrum-retrofit + phase-1-final-substrate-corrections).

Phase 8 — Coding-harness enablement (native tool-calling) ✅ COMPLETE 2026-07-03. Tag: phase-8-final (unsigned annotated — commit signing remains indefinitely deferred per DESIGN.md §20). Sprints 8.1 + 8.2 + 8.3 (gating sign-off) all closed 2026-07-03. 26 §2 task IDs across the two production sprints, all ✅ Delivered; zero four-field deferrals. Three audits ✅ PASS: quality (workspace line 85.16 %; epic-protocol::native::chat 100 %), security (tool surface held as an untrusted-input boundary; no server-side tool execution; the schema-bomb depth bound source-verified and made mechanical in ci.sh Stage 1), architecture 61/61. Definitive ci.sh --full exit 0 (2,620 tests / 209 suites / 0 failed; fuzz 6/6; Stage-8 bench gate 0 regressed vs phase-4-final; conformance openai==2.44.0 15/15 + anthropic==0.116.0 11/11 as Stage-4 gates). All three exit drills run for real against the live workstation Ferron tool build: the three-surface tool round-trip (14/14; every reassembled argument string valid JSON — the XGrammar-2 constrained-decoding guarantee) and a real Cline CLI agentic loop (grammar-constrained read_files → local execution → role:"tool" → answer). Two gate-caught defects fixed in-sprint, not deferred: the dev-fixture ADR-052 detok gap, and the tool-stream EOS special-token leak — fixed both sides (EPIC's ADR-052-revised scrub_special_markers defense-in-depth + Ferron's root-cause id-level strip shipped as v1.2.1 in response to EPIC's FR, validated live post-fix). Phase 8 benches frozen under phase-8-final/. Two ADRs Accepted: ADR-054, ADR-055; ADR-052 revised (special-token wire contract extended to the tool-content channel). Next: Phase 9 — Launch (see sprints/LAUNCH-READINESS-TRACKER.md).

Phase 3 — Shims (OpenAI + Anthropic SDK compatibility) ✅ COMPLETE 2026-06-07. Tag: phase-3-final (unsigned annotated — commit signing remains indefinitely deferred per DESIGN.md §20; a signed phase-3-final-signed tag will be created against the same history when signing eventually schedules). Sprint 3.1 + 3.2 closed 2026-06-02; Sprint 3.3 + 3.4 closed 2026-06-07; Sprint 3.5 (gating sign-off) closed 2026-06-07. 63 §2 task IDs across the four production sprints, all ✅ Delivered; zero four-field deferrals. Three audits ✅ PASS: quality, security, architecture 34/34. Conformance gates green (openai==2.41.0 11/11; anthropic==0.107.0 7/7). The plaintext JWT signing key is gone from epic-control process memory (ADR-030). The project's longest-running carry-forward (T-1.4.16 visual-regression, deferred Phase 1 → 2 → 3) retired in full. Five ADRs Accepted: ADR-027, ADR-028, ADR-029, ADR-030, ADR-031. Next: Phase 4 — Abuse + diagnostics + voting.

Added

  • The public pages get a light mode and the dashboard's two switches (2026-10-22, #395; Melvyn's request, stacked on #373). The public bar carries an animations button (a red pause while things move, a green play once they are still) and the dashboard's dark/light dot, the same buttons as the signed-in header's, beside Login. Pause stops every animation on the page, the robots' walks, the star glow and its scroll drift, the shooting stars and the badge tilt; Play resumes them, and outranks the system's reduced-motion setting. This is #355 finding 3's pause control (WCAG 2.2.2); a page with no script, and so no switch, is now still rather than running the robots' idle loops forever. The light mode puts the page on a daylight sky with the same stars as faint specks in the logo palette (tools/make-starfield.py's day copy), the logo colours in darker shades where text needs AA (tools/palette-contrast.py now checks the light palette and the switches), and the bar's and the comparison table's EPIC mark in the dashboard's dark grey (tools/make-header-mark.py writes both inks). A first visit is dark, whatever the device asks, and its motion follows prefers-reduced-motion; both choices are remembered in the browser's localStorage, never sent to the server, applied before the first paint by one small hashed inline script (ADR-083, Accepted 2026-10-24). The signed-in dashboard goes the other way: a member who never chose a base sees the light one (migration 0095 makes it the column default too), and the Profile tab's colour box gains "Follow my device's light/dark setting", off by default, applied before the first paint by a hashed script at the top of the body; the header's dark/light button turns it off (ADR-073 §7). The signed-in header's animations switch takes the same play/pause disc, both its switches become toggle buttons with aria-pressed, and their forms lose a stray 1 rem margin, so the dashboard's pair is spaced like the public bar's; nothing else on the dashboard changes. Against baselines recorded before the change, every pixel that moved on the 29 baselined pages is inside the header bar.

  • Robots on the landing page (owners' landing round, 2026-10-20). The dashboard's robot cast plays four short scenes on /, each under the words it acts out: under the hardware footnote Pip lifts GPU cards out of a crate in its beam into a server under test while Strider checks them in; beside the electricity rate Max keeps feeding coins into the meter of a humming local rig, the meter turning at the visitor's own rate; under the principles robots drop their votes in a ballot box and Clerk writes each one down; under the join form a queue waits at a velvet rope Max lets them through one at a time, and once per visit Glitch tries to push in and is sent away. Same drawings and the same Actor director as the dashboard (assets/scenes/actors.js), served as the same-origin /assets/js/landing-scenes.js; decorative (aria-hidden, nothing focusable), in fixed-height bands of their own (no layout shift, nothing covered, clipped stages), still under prefers-reduced-motion or with no script, and asleep off screen. The robots' stylesheet is split so the landing carries only what it draws: cast.css (the drawings and moves, shared), scenes.css (the dashboard's scenes, moved verbatim) and landing-scenes.css. The visual-regression set gains landing_full, the whole landing page top to bottom under reduced motion, stitched from viewport screenshots so either engine can take it (FULL_PAGES); its baseline is recorded after merge.

  • epic-secret-sync exports metrics and is alerted on (2026-10-16, issue #224). The reconciler had no metrics surface: a renewed client certificate FerraVault refused, a run of failed reconciles or a Watch that never reconnected showed only in its log, and 30 days after a missed renewal every synced Secret went stale with no alert. It now serves /metrics + /healthz on a required, dual-stack metrics_bind (as epic-vault-audit does), exporting:

    • Watch health, reconcile outcomes and the last clean full resync;
    • identity renewals and refusals, and the certificate's expiry. The only label is the allow-listed outcome. A PodMonitor scrapes it, and the network policy admits only the scrape. The shared identity alerts (EpicVaultIdentityRenewalRefused, …NotRenewed, …ExpiryImminent) now match its series, and two new tickets cover it: EpicSecretSyncNotConverging and EpicSecretSyncWatchDown. All are promtool-tested. The FerraVault gate now checks its certificate's renewal window against the expiry rules. Nothing evaluates alerts yet (#157).
  • The coin desks race, talk prices and move the bar (owners, 2026-10-14). Under "Slot deposits received" and "Slots sold" the line is now a race: robots come in at speeds of their own, first come first served; now and then a faster one overtakes in front of or behind a slower one, and now and then Strider flies over the line on his rocket pack (neither often; the desk stands further right to leave room). At the desk Clerk asks "How many deposits?" or "How many slots?", the robot answers, and Clerk names the total: the deposit per slot, or the price per slot by the size of the purchase (single for 1, duo for 2, multi for more). A real rise in the box's figure is what the next robot says; Strider then flies up to the bar above the scene and pulls its end to the new figure (or pushes it back for a fall), the box's count changing with it. Coins carry a dollar sign; the slots-sold desk's three coins are gold; coins and papers are drawn in the robots' hands and move with them (Tread reaches out further with a paper, tilted). The desk wears the EPIC wordmark on its panels and a large one on a banner on two poles behind Clerk. The deposit per slot is set with the slot prices at /app/admin/slot-prices (AUD 50 to start, never more than the multi price; migration 0084, audited with the prices).

  • The project stages crew, all the way to the servers and on to the houses (owners, 2026-10-14). The crew starts at the first stage not yet done, whatever it is called, and works spot by spot along the strip of stages to the server rack: Strider breaks the ground with his jackhammer and steps backwards out of the way; Max, beside him, steps in and digs the spot out with a new spade (has-shovel, front on, is-shoveling: the blade into the ground as his knees bend) in three spadefuls onto a heap that grows with each; Tread scoops the heap up in one go (has-load, is-scooping, is-carrying) while Pip lays the cable, the solid cable running out at Pip's own pace; Tread tips the earth back over it (is-dumping) and drives back and forth to pack it down. Clerk watches from the side, writing on his notepad or swapping it for the megaphone in his top hat (the hat hops off his head to meet his hand, is-swapping), shouting at the crew and pointing. At the rack the cable runs up into its bottom-left corner, the crew stands clear, the lights come on for a moment, and they start again. The dashed route now turns up into the rack's corner, and a second line leaves its other corner for three houses (a cottage, a tall house, a bungalow) under "Full launch", turning up into each; once "Servers installed" is done the crew works its way to the houses instead, lighting each one's windows. The rack stays under "Servers installed" and the houses under "Full launch" wherever those stages move; when the crew's target is the last stage, a stretch of plain ground follows it for the crew to stand on. The strip scrolls along with the crew unless the member has scrolled it in the last 15 s. The props and moves are copied from the robot designs, where they were added the same day.

  • The launch scenes, round 2 (owners, 2026-10-14). The dig: Strider replaces Max as the repairer, flying over Tread on his rocket pack with his legs dangling; Tread faces left where the give-away bar's green meets its red, digs into the green and throws onto a pile behind him on the red; Strider flies to the pile, picks up a clod, flies back and drops it into the hole, and when the green has no room left for him he keeps to the pile and flies up from there. Tread's scoop now reaches down into the ground in every scene he digs in. The desks: the line pays one at a time, each robot first saying how many it is after (the box's real rise when its figure has moved, otherwise a number like the interest visitors'); with its paper it walks on past the desk and off the far end, and comes back in from the start with new coins after a while of its own, so the order keeps changing — there is always someone besides Clerk. The carnival: Max carries a two-handed sledgehammer on his shoulder (a new has-sledge prop for him, with an is-sledging swing), walks between the buttons half as fast again, and hits a button from the side he walked in from — from the left going right, from the right going left — standing far enough off that the value he lights up is never behind him. Each button stands on its column's foot, its base 3px below it; the buttons, values and likeliness numbers are centred on their columns (they had sat 1% to the right); the values are 1.4rem, pop in and grow to 2rem before they fade; and the chart is taller, reaching about as far down as the entry form beside it. The question: Clerk now keeps the question mark and writes the numbers on a notepad, and Strider is one of the visitors; a visitor arriving while another is with Clerk waits in line behind it, and the line moves up. Clerk gains a notepad and pencil (has-pad, has-pencil, written on with is-writing) and a megaphone (has-megaphone, shouted through with is-shouting). One GET /app/launch/figures (the interest total and entries, slots sold, slots secured by a deposit; no-store, signed-in members only) now feeds every scene, replacing the interest-only endpoint; each browser remembers the last figures it saw.

  • The interest scenes, reworked (owners, 2026-10-14). Strider now stands by the question mark facing the viewer; visitors come from either end, he walks to their side of the question mark, points at it, turns back, the visitor says a number (1 to 5 nineteen times in twenty, otherwise up to 100) and he writes it on a clipboard; the visitor goes back or walks on at random, others wait their turn, never more than four robots on the bar. The numbers are real when the tracker moves: the scene compares the total with the last one this browser saw and asks GET /app/launch/figures every 20 s while the page is open; a rise is what the next visitor says, a fall brings Glitch (now in the site's cast, only ever for this) with the drop in red. On the chart, the buttons sit at the columns' feet with the numbers underneath, Max walks the floor in front of them, and a column shows its value at its top once its light gets there.

  • The robots wear the EPIC wordmark (owners, 2026-10-14). The header's wordmark, traced into one vector symbol (ms-epic in the robots' shared defs; the server-rack "I" redrawn clean), printed on Clerk (a new top hat with the wordmark on its crown, a small badge on his vest), Strider (his shoulder, up his rocket pack, and the gold clasp of a new belt) and Max (across the top of his shield, the disc moved down below it, and on his shoulder). Every wordmark on a side view turns back the right way round when the robot faces left; on Max it goes round his front edge, cut at his outline. Strider's legs lean in a little in the front view and his hips sit inside his belt. The designs are the owners' approved standard of 2026-10-14.

  • The question and the carnival in the interest tracker (owners' site-text round 1). Over the interest bar, Strider keeps a 3D question mark just before the middle; at random intervals Max, Tread, Pip or Clerk wander in from the left — never more than two at once — Strider jumps and points at the question mark, the wanderer stops by him, they look at each other, a bubble shows a random number from 1 to 100, and it walks on, fading in and out at the bar's ends. Under the likeliness chart, a button under each column and Max with his hammer: he walks to a column with slots in it and hits its button, and the column, dimmed while the game runs, lights from the bottom up to its height and fades. The dev server's new --sample-interest flag seeds eight answers so the chart (held back until five members have answered) can be seen.

  • The cable crew under Project stages (owners' site-text round 1). "A few of the robot[s] in hard hats and with equipment like diggers and jackhammers … putting a cable in the ground that will go to the Servers installed stage where we place an image of a server": the stage slide gains a strip of ground under the dates. The crew works across the first stage not yet done and the next — Strider with a jackhammer, Tread digging the trench, Pip lowering the cable from a reel in its beam — moving up step by step while the trench and the laid cable grow behind them. The cable runs laid through the done stages and as a dashed route to a server rack at "Servers installed" (or the last stage, if renamed), whose lights come on once every stage up to it is done. It scrolls with the stages.

  • The dig scene over the free give-away bar (owners' site-text round 1). "A smaller robot and a bigger robot 'fight' in trying to move the tracker": Tread stands on the green just past the red and digs into the bar, lobbing clods over his back onto a pile on the red; Max hops over him, picks a clod up, hops back and pats it into Tread's hole, and every third trip trips on the way. The scene overlaps the bar so they stand on it; scenes.js finds where the green begins from the meter itself, so no figure is passed to the script.

  • The coin desk scene under Slot deposits and Slots sold (owners' site-text round 1). Clerk keeps a desk with a stack of papers; Strider, Pip (its coin held in a new beam of light, the has-beam prop), Max and Tread queue with a coin each — three for Slots sold — pay at the desk, take a piece of paper (a bigger one for Slots sold), walk back behind the line to its end and find their coins again, in an unending loop. Directed by a new inline script, assets/scenes.js (hashed into the CSP with the others): it only sets classes and the --x/--t custom properties, moves are compositor transform transitions, it never starts with animations switched off or under prefers-reduced-motion, and a scene waits while it is off screen or the tab is hidden. Without it the scene is a still picture of its opening moment.

  • An animations switch in the signed-in header (owners, 2026-10-14). "An option to turn off the animations in the top bar in case some people find them annoying": a button beside the dark/light switch, on both the member dashboard and the operator console, that stops every animation in the signed-in shell (the robots, the launch summary's walker, the pulsing help marks) exactly as prefers-reduced-motion does, and starts them again. Saved per account with the colour scheme (epic_core::Animations, migration 0083 user_appearance_preferences.animations, CHECK-constrained to on/off, under the table's existing user-scoped FORCE-RLS), written into the body's data-animations attribute; no JavaScript, and paused on the pages that show a code or key only once, like the dark/light switch.

  • The dashboard's robot cast, and Strider walks the launch summary (owners' site-text round 1, 2026-10-14). Five robots the owners approved on 2026-10-14 — Pip, Tread, Clerk, Strider and Max — join the dashboard as inline SVG (epic-web::mascots), each drawn as a three-quarter side view and a front view, shaded by gradients that follow the member's colour scheme and rendered once per page. Every move and prop is a class on the robot (is-walking, is-front, has-hat, …), so a new scene is markup and CSS rather than drawing. Strider replaces the stick figure in the Founding launch summary, with the road's markings and the hills set to the pace of his stride. Nothing moves under prefers-reduced-motion, the scene keeps its pause control, and no inline style or new script is needed. The scenes for the other launch boxes follow in their own PRs.

  • The owners' accounts come back in every preview slot (2026-10-13, issue #194; ADR-049 addendum). epic-migrate ensure-accounts --manifest deploy/dev-accounts/preview.toml makes a preview slot's database agree with a checked-in list — each account by email (case-insensitive, its email marked verified), an organisation if it has none, its owner membership, its operator flag (the list decides both ways) and one complimentary slot on a named server (or the bootstrap seat for a new organisation) — and changes nothing that already agrees, so it can run after every migration. Passwords are Argon2id v19 PHC strings at no less than the platform's cost, read from FerraVault at epic-preview/dev-accounts/<key>/password-hash, held as secrets and written unchanged; a new one revokes the account's sessions and refresh tokens, as the web password change does. Every hash is checked before a row is written, and the changes and their audit records (dev-accounts-bootstrap, a new PgAuditSink::emit_in inside the caller's transaction) commit together or not at all. Only a preview slot: EPIC_DOMAIN_ENVIRONMENT from the environment alone, the list, and the database's name read on the server; dev is refused until a dev database carries a marker. An account that is only a member of someone else's organisation, or whose organisation is not active, is refused by name. The complimentary-slot grant moves to a module seed-org and ensure-accounts share. Tests against a migrated Postgres as a NOSUPERUSER NOBYPASSRLS role, including an unwritable audit log leaving nothing behind (checked to fail with the audit after the commit). Runbook: docs/operations/dev-accounts.md; running it in every slot waits on an image re-pin (#266).

  • Members write their own promo profile for the podium, may leave the affiliation programme, and a code is never given out twice (2026-10-13, Sprint AP.7, the owners' answers on #257 D1/D7 and B9; ADR-077 §8). A Your promo profile card on the Account tab takes a few words about the member, their channel's main page and one page on each of eleven social networks; every link is checked to be https on that network's own hosts (in the browser by each field's pattern, again on the server, and by migration 0079's CHECKs), and nothing is saved while one field fails. A podium place by name opens ("About …") on the text and the links, drawn as each network's logo (Simple Icons, CC0), never on an anonymous line; the console can remove a profile but never writes one, and its audit records the slots, not the words. Leave the affiliation programme (closed, with the warning and a required "I understand I will lose all my points") keeps every credit to the member spent and naming no one, retires their codes and removes their profile; agreeing to the terms again starts over. A code that stops naming its account — left, removed by an operator, or the account erased — is kept as a SHA-256 in retired_affiliate_codes and never given out again. The marker role is renamed epic_portal_calls, and the three new tables are revoked from the edge role. Postgres tests for leaving, the promo profile and the console; a browser step opens the podium's About.

  • The source gates read a page past its test-module declaration (2026-10-13, Sprint AP.7, found by ci.sh --full). inline_payloads_are_declared (the CSP gate) and ci/routes/verify.py both stopped reading a file at its first #[cfg(test)], assuming test code is always a file's tail. An out-of-line #[cfg(test)] mod tests; breaks that: in pages/app/me/affiliate/mod.rs it sat at line 40, hiding the rest of the page from both gates, and the tests.rs it names was read as if it rendered. Both now cut only at an inline #[cfg(test)] mod … { (clippy::items_after_test_module keeps that the last item) and skip a file declared out of line under #[cfg(test)]; one new test each, each failing under the old rule.

  • Smaller fixes after the stack review (2026-10-12, Sprint AP.7, issue #257 items 34, 36, 37, 39, 40, 41). Closing the Dashboard welcome from the keyboard hands the focus to the current tab (browser test, which fails without the fix); the tab-bar fades are measured again once fonts and images have loaded; the stylesheet no longer carries a global .transform and .ease-out read from words in the scripts (@source not inline, nothing else changed); every browser-test step that sizes the window puts it back, which showed one step had depended on another's size, now set by itself; a Postgres test drives moves and drags past a paused box through the store's locked transaction; the in-memory paused list breaks ties as Postgres does; page_layout and the in-memory affiliate store are split under the 500-line guideline.

  • Slot prices after the stack review (2026-10-12, Sprint AP.7, issue #257 items 17, 18, 33; ADR-077 §5). The prices include GST (owners), said on the console page, in the migration and in the code (SlotPriceAud::cents_for gives a purchase's amount in cents). "Already these prices" is decided under the row's lock, so two operators saving the same prices at once record one change; a stored price below zero is reported as damaged rather than as zero; a doc no longer promises a "who" it does not carry. The work to make checkout charge these prices is filed under launch tracker L1, and L2 now says what remains of the price decision.

  • The Q&A after the stack review, and "Email this member" (2026-10-12, Sprint AP.7, issue #257 items 12–16, 29–32, 47; ADR-077 §4). Removing a spam question no longer gives the spammer their allowance back: removed and withdrawn questions keep their row for the quota, their words erased (F-251-1). An answer saved but not published shows to its asker as still waiting (F-251-2). Members can withdraw a question or stop showing their name on it, and a closing account is named nowhere (F-251-3). The console opens on waiting questions, fifty to a page, with filters (F-251-4); publish and unpublish are audited under their own names; "answered" became "published" on the box. A test sends two questions at once and expects one through (F-251-5); it fails without the trigger's lock. The published list is no longer callable by the edge role. New: Email this member on each console question — the operator's message goes from the site's sender to the asker's sign-in address with their question quoted, audited by the question's id only.

  • Four broken interactions fixed after the stack review (2026-10-12, Sprint AP.7, issue #257 items 7–10 and 38). The page content is its own stacking context, so a box's controls no longer paint over the account menu and the legal notice and take their clicks (F-246-1). On a phone, where a row's boxes stand one above the other, dragging one moves it instead of flickering in place: the drag reads from the layout whether to compare across or up and down (F-247-1), and finishes a running glide before measuring (no jump on two quick swaps). A drag keeps a paused box where it was, so it comes back in place and the member's order stays default (F-249-1). The header search no longer offers a paused box (F-249-2). Each fix has a browser or unit test, and the two browser steps were checked to fail without their fix.

  • The affiliation programme after the stack review: one code per account, the terms first, members in transit (2026-10-12, Sprint AP.7, issue #257 items 1–6, 19–28, 35, 46; ADR-077 §8). There is one kind of account, with one set of rules: migration 0074 drops AP.6's influencer register and carries nothing over (owners, 2026-10-13: only the operators had accounts, and an influencer is a member like any other), and the console's Influencers page becomes Affiliates (the top 50 by points, find by code or email, remove an abusive personal code or promo profile). A code is made only after the member reads the owners' affiliation terms, ticks "I agree to these terms" and presses "Generate my affiliation code and link"; the version and date are kept and exported. A member may generate one personal code of up to 30 characters after a "you can only generate a personal affiliation code once" question; either code credits the same account, and Billing and the data export show the 8-character code, never a name. Leaving the named board puts a member in transit for 21 days before their anonymous line appears (option B), and so does waiting to come back; a rounded line under the leaderboard counts points, users, members in transit and members under 5 points. Points drop in proportion to a partial refund; equal points go to whoever reached them first. Profile names are unique and "Anonymous" is taken ("That name is already in use"). The portal's SECURITY DEFINER functions are no longer executable by the edge role, read the database's own clock, and the leaderboard is read in one transaction; an account already credited cannot probe which codes exist.

  • The leaderboard switch saves itself; anonymous members stand on the leaderboard (2026-10-11, Sprint AP.7 PR 11, ADR-077 §7). The Account tab's switch is green when public and red when anonymous and saves as it is flipped (no Save button, but for one in a <noscript>). Turning it off asks "Are you sure you want to become anonymous?" in a modal dialog when going public again would take three weeks; Cancel or Escape leaves it on. The first time a member goes public is immediate. Members not shown by name stand on the Dashboard leaderboard from 5 points as their own "Anonymous" line, never summed, through affiliate_anonymous_standings, which returns nothing that names them; the bonus note says it goes to the top three public members. The owners' new rules text. Migration 0078.

  • Dashboard polish from the owners' notes of 11 October (2026-10-11, Sprint AP.7 PR 10). Models, "Get started!" and Q&A get the boxes' title format and their own icons (a chip with its pins, a rocket, a question mark). The launch summary loses "The stages below show what comes next." and gains a little walker on a mountain road, as wide as the row of figures: the layers slide by compositor-only transform animations, only his 48×58 drawing repaints, it stops under reduced motion and has a pause control (WCAG 2.2.2), and its edges fade so a phone shows the middle. The marketing pool's explanation folds after its first paragraph behind "Read more" / "Read less" (READ_MORE_JS; without script the whole text shows), and its free slot tracker counts down — "400 of 400 FREE give-away slots left", the bar emptying as slots are given, labelled "400 in total" — as does the summary's figure. The interest form reads "Who would your order be for?" and the owners' new privacy line. The share pictures carry the landing page's full logo, and the Profile tab's custom colour shows a painter's palette in the swatches' colours instead of a pipette. The dev server sells nothing and gives two operators 20 complimentary slots each, as production will have them.

  • The leaderboard ranks by points, from 5 points (2026-10-11, Sprint AP.7 PR 9, ADR-077 §6). The Dashboard's leaderboard ranks influencers and public members by points — one per slot bought through their code — then sign-ups, and shows both ("12 points · 4 sign-ups"). No one is listed under 5 points, so a figure beside a name never stands for one or two small purchases; a public member below it is told how many points they have. Migration 0077 (affiliate_public_standings returns points and applies the floor); the influencers' standings apply it in their query. Replaces AP.6's "members see sign-ups only".

  • Slot prices the operators set: single, duo and multi (2026-10-10, Sprint AP.7 PR 8, ADR-077 §5). A new console section, Slot prices (/app/admin/slot-prices), holds the price per slot for a purchase of 1, 2, or 3 or more slots — AUD 4,950 / 4,850 / 4,800 as the owners set them — in one form, audited with the prices before and after. Whole dollars, a bigger purchase never dearer per slot (also a CHECK). The Dashboard's marketing-pool example ("23 slots, worth AUD 110,400") now follows the multi price, and leaves the amount out if the prices cannot be read. Migration 0076 (slot_prices, one row, operator-global). Not yet what checkout charges: no payment provider is connected.

  • A Q&A box: members ask, the team answers and publishes (2026-10-09, Sprint AP.7 PR 7, ADR-077 §4). The Dashboard's new last box shows the team's published answers and a form to ask — anonymously or under one's profile name — with a confirmation, the member's own questions and where they stand, and the owners' quota message past one question in 24 hours or three in 7 days (counted by a database trigger). Operators answer, publish, unpublish and remove at /app/admin/questions (audited by id, never the words). Migration 0075 (FORCE-RLS, hidden from the edge); the GDPR export carries a member's questions.

  • Every member has an affiliation code, with an opt-in leaderboard (2026-10-09, Sprint AP.7 PR 6, ADR-077 §3). A new Affiliation code box on the Account tab: the member's own code (made the first time it is shown) and link with copy buttons, the device's share sheet and plain share links (X, Facebook, LinkedIn, WhatsApp, Telegram, Reddit, email), three 1080×1080 pictures with the code on them drawn in the browser and downloadable as PNG, their own figures (accounts linked, points, free slots earned), and a switch to be shown on the leaderboard — anonymous by default, public at once or three weeks after the first account links. Member codes credit through the same path as influencers' (set once, never one's own, one namespace), and close once every slot is sold. The Dashboard's pool box carries the owners' explanation, a free-slot bar, a three-bar tracker (linked → deposit → bought) and a leaderboard of influencers and members who opted in. Migration 0074; the code table is FORCE-RLS and hidden from the edge; GDPR export carries the code.

  • Operators can pause any box for every member; Servers starts paused (2026-10-09, Sprint AP.7 PR 5, ADR-077 §2). A new console section, Paused boxes (/app/admin/paused-boxes), lists every box of every member tab with Pause… (through the confirm page) or Restore; an operator also sees a pause control on each box. A paused box is hidden for everyone, members keep their own place for it, and moving a box steps over a paused one so what the member sees and what is saved agree. Migration 0073 (paused_boxes, operator-global), seeded with the Dashboard's Servers box. Audited. The in-box text editor the owners floated is not built (it would fork the copy from the code); a draft issue explains why.

  • The Dashboard in the owners' words, and a team may put up to 500 slots on the interest tracker (2026-10-09, Sprint AP.7 PR 4, ADR-077 §1). The summary box ("Founding launch summary — a quick overview of the numbers") and every tracker carry the owners' copy and titles in the four formats; the summary's five figures — interest, deposits, sold, give-away slots, stages — are links that jump to their tracker. The interest form offers "A team" beside "Myself", up to 500 slots for a team (migration 0072 widens the CHECK and keeps an individual and a carried-over entry at 100, in the database too; InterestWrite::new checks the same). "Slot deposits received" and "Slots sold" carry the owners' notes, the latter's goal read from the servers. The Usage box gets a speedometer, the owners' copy, accent stat titles, and a fair-share standing with a "?" that opens the full explanation on hover or click (browser-tested: hover, leave, pin, close, Enter, Escape). DESIGN.md's last "4,000 per server" corrected; the dev server and the pixel-gate fixture seed 9,000 slots per server.

  • A dragged box follows the pointer, and drags down as well as up (2026-10-09, Sprint AP.7 PR 3). The box lifts (shadow, above the others) and stays under the pointer at the spot it was picked up by; the other boxes glide out of its way and it settles into its slot on release. A box now swaps once its own leading edge passes the middle of its neighbour — measured from the pointer, as before, dragging a box down meant dragging past most of the box below it, so it seemed only to move up. Layout positions come from offsets, so a block still gliding cannot undo its own swap. Browser-tested (drag down, follow mid-drag, saved, settled); the test fails against the old script.

  • A quieter portal header: the tab bar rises level with the logo and icons, and the Dashboard welcome folds away (2026-10-09, Sprint AP.7 PR 2). On a tablet or wider the member tab bar is pulled up until its raised tab lines up with the wordmark and the header icons; tabs scrolled under the icons fade behind them and cannot be clicked there by accident, and the gaps between tabs now scroll the bar instead of the page. The EPIC logo goes home (the member's opening tab), or to the console for an operator. The welcome reads "Welcome, " (or just "Welcome"), centred, folds away after 20 seconds once per browser tab, and has a close button; the "signed in as a member of …" line and the repeated page titles under the tab bar are gone (still announced to screen readers). Long words wrap rather than widening a phone's page; the interest form's likeliness scale wraps to two rows on a phone (the Dashboard was 37px too wide at 390px).

  • The portal's house style: one box design and four text formats (2026-10-09, Sprint AP.7 PR 1). Every box on every member tab is drawn by its wrapper with the launch tracker's shaded surface and an accent edge along the top (a little thicker on the launch box), so every title and control sits inside its box — the API-key box's contents no longer hang below it — and boxes side by side share one height. Four text formats (.fmt-eyebrow, .fmt-heading, .fmt-lede, .fmt-box-title) for the owners' copy. Portal only; the public site is untouched.

  • The visual-regression gate: the main drift explained, the engine pinned, and provenance recorded with every baseline set (2026-10-09, issue #175). Rendering phase-9-final and f28a11c under one engine reproduces #175's table to three decimals, so the drift was intended product change. The public pages are 0.00000 from f28a11c to today. The gate had never run its pinned Firefox: geckodriver launched the auto-refreshing snap instead. The binary is now passed explicitly and the tarball's updater is disabled. PROVENANCE.json (browser, binary, fonts, viewport, commit) makes the next drift say whether the environment changed. Non-200 pages are refused and retired pages' baselines removed. Re-recorded at d059c85; the workstation's /data set needs one ci/visual/run.sh --record.

  • The AP.5/AP.6 stack review, answered (2026-10-08, Sprint AP.6 §7, PR 7/7). Security: an email change is confirmed only from the signed-in account that asked, and a taken address is not revealed; the edge database role can no longer read members' password hashes, TOTP secrets or contact details; the interest list's clear is epic_admin-only and audited before it runs; hidden formatting characters and private-use country codes are refused. Correctness: menu, layout and colour-scheme saves are applied atomically and sent one at a time (browser-tested); account details save in one compare-and-set write; the cache is cleared after writes too; referral standings count only purchases made after the referral. Owners' decisions: members see influencer sign-ups only, and the interest breakdown from five entries. Migrations 0069–0071. Issues #239 (TOTP replay, pre-existing) and #240 filed.

  • The Profile tab: the avatar always in sight, and a mini tab bar for menu preferences (2026-10-08, Sprint AP.6). The Profile card's view mode is the account menu's header enlarged — avatar, profile name, email — and "Edit profile" opens the same table as before. Menu preferences gains a small copy of the top bar: drag a tab, or select one and use the arrow keys or the buttons, and "Open on this tab" moves a house icon to the tab signing in opens on; every change saves at once, and the plain form remains for a browser without a script.

  • Every box on every member tab collapses and moves, saved per account (2026-10-08, Sprint AP.6, ADR-076 §5). Dashboard, Group, Account and Profile: each box has a collapse arrow (a real form, so it works without a script; instant with one) and a grip — drag it, or focus it and use the arrow keys, or press it for arrow buttons. Side-by-side boxes swap left and right and move up and down as one row. Stored per page (migration 0068, FORCE-RLS); the server applies each change to the stored arrangement. Links to a collapsed box open it. Browser gates green: keyboard, contrast on every scheme, overflow.

  • Influencers, referral credit, and the promotional launch tracker (2026-10-08, Sprint AP.6, ADR-076 §4). Operators register influencers at /app/admin/influencers (name, referral code, optional https links, required consent; edit, pause, remove — audited). Members are credited once, never to themselves, by the influencer's link (/r/{code} → sign-up, a hidden field, no cookie) or by entering the code on Billing (migration 0067; FORCE-RLS on who referred whom; standings are counts of verified sign-ups and paid slots). The Dashboard's launch tracker now has the owners' five trackers in their order — interest, Admin/marketing (complimentary slots given against the budget; top three with avatars and links, next five, the rest collapsed), deposits (not open yet, at zero), slots sold, stages — under a hero banner with live figures, an icon per tracker and glowing meters. The GDPR export carries the referral.

  • A Paid slots interest tracker replaces the interest list (2026-10-08, Sprint AP.6, ADR-076 §2–§3). One honest entry per account on the Dashboard (migration 0066, FORCE-RLS on app.user_id): who it is for, 1–100 slots, how likely the member is to order (1–10); the form shows the account's email (counted once, used for nothing else, deleted when the server is full). A meter runs to one server's 9 000 slots and, once passed, slides the 9 000 marker left with a second marker for the total; a red-to-green column chart shows slots per likeliness, with a screen-reader table. The figures cross accounts only as counts. The operator console clears the list (confirm + reason, audited). Old interest-list entries carry over marked not stated; the old table stays, unread. Billing and the sold-out checkout link to the tracker; the GDPR export carries the entry.

  • The launch tracker's project stages are the owners' to edit, and each server sells 9 000 paid and 400 admin/marketing slots (2026-10-08, Sprint AP.6, ADR-076 §1). The stages move from code to launch_stages (migration 0065), seeded with the owners' eleven (the β platform done, the rest coming up, no dates) and edited at /app/admin/launch — title, status, expected date, order, add and remove, every write audited. The Dashboard shows them as one long slide, dulled red / orange / green with the status also in words, above a date track. Concurrent edits keep positions gapless under an advisory lock (the test fails without it). New servers default to 9 000 paid and 400 complimentary slots.

  • The member dashboard is five tabs — Dashboard, Playground, Group, Account, Profile — as the owners laid it out (2026-10-08, Sprint AP.5, ADR-075). Wiki Site Layout & Tabs, table 2. Dashboard: a launch tracker (slots sold, live, as a count and a bar; project stages that follow the servers' status; a labelled placeholder for the influencer leaderboard, which has no data yet), usage, servers, models, and a way to an API key. Group: group details, members and the fleet on one tab. Account: "Account details" (the old Profile card renamed — full name, email, and new optional phone, postal address, company and ABN, the ABN checksum-validated; the user public ID is no longer shown), Security with a button to the audit log, API keys, Billing, Privacy. Profile (new, /app/profile): profile name and avatar, colour schemes, menu preferences (the member's own tab order, by move buttons, and the page signing in opens on — migration 0062, FORCE-RLS), email notifications. A header search for pages and settings (an icon that opens a box without any script; suggestions as you type; a server-rendered results page), every "X" button reads "Close", and on phones the account menu lists the sections. Former pages (/app/usage, /app/models, /app/fleet, /app/keys, /app/members, /app/billing, /app/security) 303-redirect to their sections, /app/security closing F24's error path. Saving one of the two cards that used to be one can no longer clear the other's fields, and a tab whose one section cannot load shows that section as unavailable rather than failing (usage included — a failed read no longer shows as zero activity). The Account tab's billing notices come from a fixed list of keys rather than as text in the address, so a crafted link cannot put words on the page. Browser gates (overflow, contrast on every scheme, keyboard) green; pixel baselines move by design and are re-recorded with #175.

  • FerraVault drift is checked continuously, under an identity that can change nothing (#193, ADR-072 §4 Amendment, 2026-10-06). FerraVault v1.15.0 shipped ferravault#8: policies and mTLS bindings are readable per object with read (no longer sudo on **), carry a BLAKE3 content digest, and binding changes reach Vault.Watch. New binary epic-vault-audit (epic-system, deploy/k8s/epic-vault-audit/) holds read + watch on sys/policy/** and sys/mtls/** only, and compares FerraVault's own digests of every stored policy and binding with the repository's policy files and identity registry. It re-checks within 2 s of every policy or binding change, whenever its Watch reopens, and every 5 minutes. It exports epic_vault_audit_{policy_drift{policy},binding_drift{identity},unexpected_policies,unregistered_bindings,last_success_timestamp_seconds,checks_total{outcome},watch_connected}, with four alerts: drift pages after 30 m, an unregistered binding pages at once, and a check that cannot run never reads as clean. The expected state ships as generated ConfigMaps (ci/ferravault/render_expected.py); Stage 1 fails when they are stale. verify.py gains consumer auditor (its watch list read from the binary's WATCH_PATTERNS) and refuses any other identity whose read rules reach sys/policy / sys/mtls, which turns v1.15.0's upgrade hazard into a permanent gate. ferravault-verify-live.sh now runs the same binary's check in the auditor's pod; ci/ferravault/live.py is deleted. FerraVault crates 1.14.0 → 1.15.0 (all sixteen in lockstep; no EPIC source change needed; client HTTP/2 + TCP keepalive on by default — ferravault#5, epic#183). Tests: 29 unit; deployed_policy (the deployed policy against the binary's requests, with FerraVault's evaluator); integration_live_audit 5/5 against an in-process FerraVault 1.15.0 over mTLS — every drift kind, refused writes, and Watch-driven re-check (mutation-checked: disabling the event re-check fails it); 4 new gate mutation tests; promtool test rules on all five alerts. Review fixes (#195): the auditor now picks up its renewed client certificate. It polls the mounted pair by content (ADR-071's reasoning), re-dials, and reopens its Watch under the new certificate. A check or Watch reopen that fails at FerraVault re-dials, so a process that dialled once can no longer go silently blind when its 90-day certificate expires. New epic_vault_audit_{identity_renewals,redials}_total. A live test starts the auditor on a 6-second certificate, swaps in the renewal and expires the first, with zero failed checks across the expiry (mutation-checked twice). Probes now race shutdown; PatternSet moved to watch/pattern_set.rs. Corrected: the Active state and its metric help (granted at open, not proof of delivery), the WatcherHandle drop semantics, and ADR-072's readmission claim. Found while fixing this and filed: #216 (the edge, control and secret-sync never pick up a renewed FerraVault certificate — rotate_identity has no production caller) and #217 (readmission does not replay; OUT_OF_HISTORY retries forever). Review round 2 (#195): (1) WatcherHandle::shutdown() could hang. Every shutdown arm used watch::Receiver::changed(), which marks the change seen, so when a re-probe's arm consumed it, consume's own arm never fired again while shutdown() held the sender awaiting the join. The loop kept streaming and re-probing, and the edge's exit waited for SIGKILL. All six arms (five in the session, one in the synthetic loop) now use one shutdown_requested helper over wait_for, which tests the flag. A new live test stalls a TCP relay in front of an in-process FerraVault until a re-probe is provably in flight, then requires shutdown() to return; with changed() restored it hangs 3/3. (2) ferravault-unseal.md §9 was wrong about rollback. A pre-1.15 daemon on a fresh audit_orphan.db re-runs the one-time baseline and grandfathers every record past the chain head with no tamper signal (v1.14.0 reconcile_at_boot). The runbook now records ferra audit verify --output json on v1.15 first, stops on any unaccounted record, says what the baseline log line means, and drops the ferra audit acknowledge step v1.14 cannot run. (3) The auditor swaps in a renewal only once FerraVault accepts it: under TLS 1.3 a rejected certificate still dials, so the new client makes one authenticated PolicyList first (Health bypasses authentication and would pass an unbound certificate). It re-dials only on transport failures, not on vault decisions such as a missing grant. New epic_vault_audit_{watch_opens,identity_renewals_refused}_total and an EpicVaultAuditRenewalRefused ticket (promtool-tested). New live tests: an untrusted-issuer renewal and an unbound renewal are refused while checks continue, then a good renewal is picked up; a missing grant raises errors but no re-dials; the renewal test now requires the Watch to reopen before the old certificate expires. Each was mutation-checked, including the deletion of close_watch that Melvyn showed the previous test missed.

  • The avatar upload and Change Password, hardened through two review rounds (2026-10-06, PR #164 round 2 B1–B5 and PR #197 items 1–7, ADR-074). Avatar: decode limits checked from the header and against the decoder's own size before allocation (8192 px per axis, 80 MiB — a 24 MP phone photo fits, a decompression bomb does not); decoding on the blocking pool behind a two-permit semaphore whose owned permit stays held when a request is cancelled mid-decode, so resetting uploads cannot run unbounded pipelines; a 15 MiB body limit with an HTML 413 page; EXIF orientation applied, every other EXIF field (GPS included) dropped; 8-bit re-encode and a 400, not a 500, for anything still over the 256 KiB the column allows; Cache-Control: private, no-cache with an ETag; its own §9.7 bands, per IP and per account; at most four uploads received at once per replica and one per account, a slot taken before the body is read and released only when the decode ends, so the bodies in memory are bounded as well as the decodes; a "busy" answer is not charged against the member's allowance, and removing an avatar is its own plain route (rounds 3–4). Change Password: a per-IP and a per-account rate limit ahead of Argon2; an MFA step-up (current code or a recovery code, checked after the password so a wrong password never burns a recovery code); every failed step audited as user.password_change_failed with its reason; a failure after the rotation says the password changed rather than that it did not; a success banner. Personal data: avatar and name changes audited (that they changed, never the values); /api/me/export and /app/me/export carry the structured name and the avatar image.

  • A horizontal tab bar replaces the dashboard sidebar, on both the member dashboard and the operator console (2026-10-06). The selected tab is centred, drawn in the member's palette colour at 1.7× and rises into the header; the bar slides, swipes on a phone, stops at the first and last tab centred with empty space beyond, and shows an arrow at an end that still has tabs out of view. "Home" becomes "Dashboard" and is where login lands; the "Dashboard" word beside the logo is gone, and the logo links to the console for operators only. Arrow keys move along the bar, with a visible focus ring on every tab. The browser gates caught a mid-transition contrast reading and an unnamed second <nav> landmark; review round 1 caught a drag that could leave the bar following a released mouse; all fixed. See Sprint AP.4.

  • The Account page's layout follows the member's own card order, "MFA" replaces "Two-factor" on the Account page, and a self-service "Change Password" section joins the Security card (2026-10-06, Sprint AP.3). The MFA setup and login pages still say "two-factor"; aligning them is #208. Card order is now Profile → Colour schemes → Security → Privacy → Email notifications (operator-specified). The Profile card's status row and both Security-card mentions read "MFA" / "Multi-Factor Authentication" rather than "Two-factor" / "two-factor authentication" — same capitalisation throughout. The Security card gains a second no-JavaScript edit-mode section, styled like the Profile card's own toggle (a visually hidden checkbox + <label for>, :checked ~ CSS rules, a native <button type="reset"> for "X"): current password, new password, confirm, posting to a new POST /app/me/password (epic-control::routes::web::password). That handler verifies the current password against the stored hash, hashes and persists the new one, then — because landing back on /app/me still signed in was the explicit ask, not a bounce to /login — revokes every session and refresh family (the same safety guarantee routes::auth::password_reset_confirm gives a stolen-password rotation) and immediately mints a fresh session for the browser that just changed it, so "change password" costs every other device its access while leaving the member where they were. Best-effort SecurityAlert email; a hard audit emit (user.password_changed) for the same reason password_reset_confirm treats its own audit row as load-bearing. POST /app/security/totp/setup's successful-verify redirect changed from a separate /app/security/totp/enabled landing page to /app/me for the same "return to where you started, still signed in" reason — that page stays live and directly reachable, just no longer the default landing.

  • Members can upload an avatar, and the standalone "Log out" button becomes a UniFi-style account menu (2026-10-06, migration 0061, ADR-074). The avatar lives in Postgres as BYTEA (users.avatar_image + avatar_content_type, three CHECK constraints: presence-iff-content-type, content-type restricted to image/png, body capped at 256 KiB) rather than a new object-storage dependency. POST /app/me/avatar (epic-control::routes::web::avatar) decodes the upload via image::ImageReader::with_guessed_format — decode failure is the format whitelist, closing the SVG-upload-as-XSS vector a naive extension check would leave open — resizes to fit within 256×256 (never upscaled), and always re-encodes to PNG regardless of source format, so what's stored is one well-understood shape rather than a pass-through of client-supplied bytes; GET /app/me/avatar serves it back. The control plane's global 64 KiB request-body cap gained a path-specific exception (AVATAR_MAX_BODY_BYTES, 15 MiB — see the hardening entry above) for this one route, on both the custom content_length_guard middleware and a per-route DefaultBodyLimit layer. The Profile card gains an "Avatar" row, visible only in edit mode as a whole row (not just an editable cell, unlike every sibling row) with a preview, a file picker (which opens a cropper: place the photo in the square, zoom, tilt, turn), and — once an avatar exists — a "Remove avatar" button that asks first ("Are you sure you want to delete this avatar? …", with "No, keep avatar" / "Yes, remove avatar", a native popover, no JavaScript); the picker's controls use the HTML form= attribute to submit to a separate, otherwise-empty sibling <form>, since a multipart/form-data upload cannot nest inside the surrounding profile-form. The header's "Log out" button is now a small avatar/silhouette trigger opening a dropdown (no JavaScript — the same checkbox-hack as the colour picker, plus a full-viewport backdrop <label> for click-outside-to-close): the member's name (falling back to "EPIC User") and email, Support, Sign out, and Terms & Conditions / Privacy Policy (each opening a temporary "In progress, come back later, email us for any questions." notice — no placeholder legal pages are published, PR #164 review D5) — modelled on a supplied UniFi reference screenshot, deliberately omitting its "Site Manager" version-number row. The dropdown's name and email come from a new epic_web::identity module — a tokio::task_local! slot mirroring epic_web::appearance's shape, set from the same two chokepoints (routes::web::app::authenticate, admin::require_operator_session) — rather than a parameter threaded through every page function.

  • The colour-scheme work's review rounds 2–3: the muted blue as the default, a contrast margin, and a dark/light switch that cannot break MFA setup (2026-10-06, Sprint AP.2). The muted dark+blue is the dashboard default by decision (ADR-073 §6); :root stays the public site's own. Every preset surface and every derived custom surface clears 4.6 for white text, not merely the 4.5 floor. The header switch reloads the page it was clicked on, and the MFA setup page mints a fresh secret on every GET, so switching mid-enrolment silently invalidated a scanned QR code and the written-down recovery codes; on that page, a new API key's and a new access code's, the icon now opens a short note instead, and the server refuses the setup page as a return target. The browser contrast gate now measures all nine presets — a hand-typed list had left orange, pink and slate unmeasured. Migration 0059's comments no longer claim the defaults equal :root, fixed before it is ever applied.

  • A missing-chokepoint bug — seven pages rendering the dashboard's default chrome and colours regardless of the member's choice, one of them with no auth check on POST at all — fixed across every instance an audit found (2026-10-06). routes/web/{totp_setup,keys,abuse_opt_out,diagnostic_capture}.rs and me.rs's delete_post/profile_post/notifications_post/appearance_post were calling require_authenticated_user_pub directly instead of the dashboard's authenticate() chokepoint — the same defect class ADR-073 fixed for a different six pages the day before. The worst instance was me.rs's delete_post: an unauthenticated POST /app/me/delete reached the full account-deletion confirmation logic before any session check ran at all. totp_setup.rs gained a GET /app/security/totp/enabled route of its own (previously unreachable from a stateless router with no way to call authenticate()); the appearance-reachability gate test now reads the same all_dashboard_routes() list every router is built from, so a page missing from that list is itself a finding, not just a missing assertion.

  • The Account page's two identity cards merge into one editable "Profile" card, plus a structured full name (2026-10-05, migration 0060). The read-only identity summary and the separate one-field "Profile" form (display name only) were two cards covering overlapping ground; they are now one, with a no-JavaScript edit-mode toggle — a visually-hidden checkbox (.scheme-custom-input's own trick: opacity: 0, never display: none, so it stays focusable via its <label for>) that CSS :checked ~ rules key every row's view/edit cell and the button row off. "Edit profile" is that checkbox's label; "Save profile" is a plain submit; "X" is a native <button type="reset">, which reverts every control in the form — including the checkbox — in one native behaviour, so discarding changes and leaving edit mode happen together rather than needing two mechanisms. New first_name / middle_name / last_name columns (users, migration 0060, each nullable and independently capped at 128 chars) back a new "Full name" row — one joined line in view mode, three inputs in edit mode — kept distinct from "Profile name" (the renamed display_name row, unchanged as a column). Email, Public id, and Two-factor keep no edit affordance — email needs its own re-verify flow, Public id is immutable, and two-factor setup lives on its own page. The combined save goes through one new Store::update_user_profile call (UserProfileUpdate) rather than through UserInsert, which stays untouched — UserInsert has roughly fifty construction sites across the workspace for fields every signup carries, and none of these three are collected at signup. Both the "Appearance saved." and "Profile updated." banners are gone: the colour picker's own selection ring and the edit toggle collapsing back to "Edit profile" already show the save took, so a banner on top of either was noise. A phone_number column and row shipped and were removed from this same migration and card before either merged (operator request) — safe to edit the migration in place rather than carry a column nothing writes, since the branch had not merged.

  • A free-form accent colour, three more curated presets, and the auth chokepoint that made the first colour-scheme feature only reach six of its thirteen pages (2026-10-05, ADR-073). /app/models, /app/fleet, /app/playground, /app/usage, /votes/*, /app/me/export and /app/me/delete were calling require_authenticated_user_pub directly instead of the dashboard's authenticate() chokepoint, so every one of them rendered the default scheme regardless of what a member had chosen — invisible to the Sprint AP.1 gate, which drove a hand-typed six-path array that happened to be exactly the six paths app.rs itself registers. The gate now reads all_dashboard_routes(), the same DashboardRoute data every affected router is built from, so there is one list rather than a router and a second, independently-maintained idea of what it serves. Three more presets (orange, pink, slate) join the original six, same measurement discipline, same ≥ 5.5 / ≥ 4.5 floors. A genuine free-form accent sits alongside the curated set: a native <input type="color"> (the browser's own wheel and eyedropper, no copied assets) submits a hex, and derive_custom_accent adjusts only its lightness — hue and saturation held fixed — to the value closest to the AA boundary, proven for every hue and saturation by a property test rather than measured for a fixed sample. (The property test only proves the contrast floor; it didn't catch the first version collapsing every dark-base input to flat white, because white clears that floor trivially — caught by looking at the rendered page, and now pinned by its own regression test.) The one dynamic <style> block this needs gets a per-response SHA-256 computed from the bytes actually served and appended to that response's own CSP style-src — the policy stays hash-based, not nonce-based, for every payload including this one; proven on a real response, not just on source text, by a new wire-level test that seeds a custom accent and checks the rendered body's inline block against its own header. A header quick-toggle flips only the base (dark/light), no JavaScript, and validates its hidden return_to as a lookup key into the server's own route list rather than ever redirecting to request-supplied bytes. See ADR-073.

  • Member-selectable colour schemes, behind the login only (2026-09-19, Sprint AP.1). Two composing axes — a base and an accent — give twelve schemes from eighteen declarations, chosen on /app/me and stored per user (migration 0059, FORCE-RLS on app.user_id, mirroring the notification preferences). Nothing before the login changes: the pre-login shell is a different element with its own token set, and a test pins that at the HTML rather than trusting the CSS scoping. Every value was measured against a contrast calculator validated on the four ratios input.css already documented; foregrounds carry >= 5.5 against an AA floor of 4.5, weakest of eighteen measurements 4.70. Light mode was the larger half: six colours written as light-on-dark literals measure 1.79 / 1.55 / 2.58 against white — unreadable, not merely off-brand — and are now tokens with per-base values. .btn.danger and .preview-banner keep their literals deliberately, carrying their own dark grounds. The scheme is carried as a per-request task-local rather than threaded through 87 call sites, which is a five-file diff instead of ~150 edits; the trade is a failure the compiler cannot see, so it is backed by a mutation-tested gate that drives the authenticated routes through the production stack and asserts a non-default scheme — seeding the default would pass against a plane that had lost the feature entirely. site.css grows 3.1 % and is inlined into every response, so pre-login pages carry bytes they do not use. See ADR-070 and Sprint AP.1.

  • Slot purchasing & access levels — Sprint SP.3: sign-off (2026-09-18). The phase's evidence sprint. 34 §2 items across SP.1, SP.2 and SP.4, all ✅ Delivered, zero four-field deferrals. Three audits ✅ PASS: quality (workspace line 88.33 % — above the 80 % floor, above Phase 9's 86.51 %, and the highest recorded here; 2 889 tests / 137 binaries / 0 failed), security (this phase's own attack surface — webhook intake, money-adjacent authorisation, the rights gates' refusal carrying no oracle, tenant isolation under a NOSUPERUSER NOBYPASSRLS role, access codes hash-only at rest), architecture 77/77. The eleven §4 exit criteria were run as gates rather than inferred — 110 tests across 11 suites, named per criterion, with both concurrency properties proven against real Postgres rather than asserted from trigger text. Stage 8 returned 0 regressed, 13 stable against local-3826c6c, confirming SP.4's bench re-freeze rather than re-deciding it.

    The gate found five real defects and all five were fixed in-sprint, and three of them share one shape: a check that had been returning the right answer for the wrong reason. The ADR-026 pipeline-invariant command in CLAUDE.md §3.7 names pipeline.rs, and rg on a file path does not recurse into the sibling pipeline/ directory — where Sprint 2.4 moved the very import it polices. A dual-stack listener test asserted "both sockets are bound" for a socket that had never bound. And epic-meter's coverage figure had been carried by two prior phase audits as a live-substrate measurement gap when the substrate suites were running all along — the uncovered lines were production wiring nothing called, not paths nothing could reach. The lesson generalises past the crate: a coverage figure carried forward with an explanation attached is not the same as one re-diagnosed.

  • Slot purchasing & access levels — Sprint SP.4: complimentary slots, slot holds, and one login per account (2026-09-16). Every slot is now a row of its own, with a kind, its four rights, a status and an end, and each right is decided per slot rather than per account. Each server has two pools: the paid places it sells, and a resizable complimentary budget operators grant from. Servers are created hidden and listed when needed. Operators grant, hold, lift and terminate individual slots from the account's page, suspend an account (which holds every active slot, and lifting releases only those holds) or close it for good. A terminated paid slot keeps its place off sale, and that place can be handed on once to another account — as a new slot with its own id and its own signed certificate, the previous holder's marked no longer valid from the day it stopped. A grant or hand-on issues a certificate; going online re-issues them dated. The Plan page lists the account's slots by server with their certificates; an operator's reasons and notes stay off it. A five-minute sweep ends temporary grants at their day and every slot at its server's term end, and expiry warnings are counted from slot rows, so a place's new holder is warned and the terminated one is not. epic-migrate seed-org --complimentary-on <server> gives the bootstrap account a real slot instead of a bare seat. A login belongs to exactly one account: an invitation creates a new login inside the inviting account, through the whole registration pipeline, at a /invitation page that replaces an emailed link no route had ever served. Migrations 0057 and 0058 — 0058 refuses to run while any login still has two accounts, and names them. ADR-069 Accepted; ADR-067 §1 and §3 superseded in part. Detail: sprint-SP.4.

  • Slot purchasing & access levels — Sprint SP.2: one-off slot purchases on per-server pools (2026-09-15). Billing sells one-off slots instead of subscriptions. Servers fill in order. The investment step reserves slots for 30 minutes on the server they go to, is limited to the slots left, and offers a free interest list when every server is full. PurchaseEvent replaces SubscriptionEvent in the trait and in the Paddle, Lemon Squeezy and Stripe adapters, and the customer portal is gone. The database maintains orgs.seat_count from paid purchases: a refund puts slots back on sale and clears the newest seats. /app/billing lists purchases with their signed certificates. Operators run servers from pre-order to online at /app/admin/servers; going online dates every slot on the server and re-issues its certificates. Expiry warnings go out per organisation per server. Migrations 0054–0056; the organisation's plan dates are dropped. No merchant is live yet (tracker L1). Detail: sprint-SP.2.

  • Slot purchasing & access levels — Sprint SP.1: access levels and playground access codes (2026-09-14). Only slot users reach the playground, API keys and voting. The edge refuses a non-ephemeral key whose organisation holds no slot (403 slot_required on native, OpenAI and Anthropic), so keys issued before the rule stop working. Operators issue one-time playground access codes from /app/admin/access-codes (shown once, emailed best-effort, revocable). Members redeem them on /app/playground. Detail: sprint-SP.1.

  • Slot purchasing & access levels — planned (2026-09-14). ADR-067 Accepted: only slot users reach the playground, API keys and voting; operators issue one-time playground access codes; billing moves from subscriptions to one-off slot purchases on servers that fill in order, with each server's go-live starting one shared ~5-year term. Plan: sprints/slot-purchasing/ (SP.1 access levels and codes, SP.2 the billing reshape, SP.3 sign-off).

  • A member can ask for a new verification email (2026-09-14). Registration sends one verification mail, best-effort, and until now there was no way to get another: the signup check-email page pointed at a dashboard control that did not exist, and the verify-email error page at an account page that had none. The login page now offers "Didn't get your verification email?", leading to /verify-email/resend (and POST /api/auth/verify-email/resend). Every submission gets the same answer and the same page, and the answer is returned before any of the work starts — which runs on a detached task — so neither the response nor the time it takes says whether an address has an account (password-reset/request gives the same answer but does its work first, so its timing still differs). A new 24-hour link goes out only to an existing, unverified, undeleted account, using the same mail model registration uses, and is audited as user.verify_email_resent. Two bands: 5 / hour per IP in the middleware (AuthVerifyEmailResend), and 3 / hour per account after lookup (AuthVerifyEmailResendPerAccount, silent), because anyone can register with someone else's address and a per-IP band alone would let a caller spread across networks flood that inbox. The signup check-email page's last tip and the verify-email failure message now point at this flow. Mutation-tested: making the work synchronous, dropping the already-verified check, or dropping the per-account band each fails a test — the first via a mail server that takes 10 s to answer.

  • A rate-limited form post gets the page back, not the API's error envelope (2026-09-13). The waitlist band is 5 submissions / hour / IP by design (DESIGN.md §9.7) and the limiter is middleware shared with the JSON surface, so hitting the ceiling from the landing page rendered {"error_code":"rate_limited",…} as the whole document — the API's answer shown to a browser. The deny branch now bounces HTML form paths to their own Post/Redirect/Get target (/?joined=busy#join, a new Notice::TooMany reading "Too many submissions from this network. Please try again later."), and every /api/ path keeps the envelope unchanged. The target is a fixed per-path constant rather than a Referer or a form field, because a rate-limit denial is precisely the response an attacker can summon on demand and a bounce to request-supplied input would be an open redirect with extra steps. Two tests: the map, and a cross-check that every path the limiter guards is either an API path or has a target — so a new public form cannot quietly inherit the JSON answer.

  • The script's URL carries a fingerprint, and the popup centres itself (2026-09-13). asset.rs justified a year-long immutable cache on the grounds that "the URL names the content" — true of the fonts and images, whose filenames change when their bytes do, and false of brand.js, whose name never changed. So every deploy that altered the script left returning visitors holding the old one, and the damage was not merely "no new behaviour": a cached script paired with a fresh stylesheet is a pair that never existed anywhere, and this one put the join popup as a static block at the end of the document — off-screen on a phone, a scroll away on a desktop. The layout now emits /assets/js/brand.<fingerprint>.js (eight hex of FNV-1a over the bytes, deterministic so identical bytes give identical URLs); the route serves the fingerprinted and the bare form, and the CSP guard that requires every <script src> to be a readable literal was taught this one expression and mutation-tested to confirm it still rejects an external origin. The popup is also positioned by the message itself rather than only by its scrim, so the two halves of the feature can no longer disagree about where it goes. Verified at three scroll depths and two widths, and in the scrim-less shape an older script produces: centred to 0px, fully on screen, every time. The earlier verification of this had checked only scrollY = 0, where a box anchored to the document's top is indistinguishable from one fixed to the viewport.

  • An address typed once is not typed again, and the signup page offers the way out first (2026-09-13). /signup?email= already prefilled; /login?email= now does too, and the signup page shows an "Already have an account? Log in" route above the form whenever an address arrived with the request, carrying it across. It says if rather than reporting a fact, and that is the design: checking whether the address is registered and saying so would answer "does this email have an EPIC account" to anyone who crafts the URL — an enumeration oracle cheaper than the one POST /signup already is, needing no Turnstile, no password and no policy acceptance. Offering the route costs a returning member one click and tells a stranger nothing. The already-registered error card keeps its own login button, and the pre-form route is suppressed when it is showing, so there is never one above the other. prefill_email moves to routes::web for both handlers to share, and the percent-encoder that was private to the operator console becomes epic_web::url rather than getting a second copy.

  • Four fixes from reading the preview on a real phone (2026-09-13). (1) The first screen was taller than the screen. --brand-nav-height claimed 4rem while the bar rendered 77px — min-height is a floor and the floor stopped binding when a 2.75rem panel button arrived inside 2rem of block padding — so the hero reserved 13px more than it was given and its second button sat under the fold. The bar's block padding is 0.5rem now, which makes the floor bind and the token true. The hero also subtracts the measured height of the whole pinned stack (--brand-topbar-height, published by brand.js, falling back to the token), so the preview banner is accounted for: the banner used to be deliberately unsubtracted on the reasoning that it should cost space, which made the preview an unfaithful picture of the page it exists to preview. And the hero's vertical budget is trimmed below md, where its content is genuinely taller than a short phone screen — measured against the fold at four viewport sizes rather than adjusted by eye. (2) The join popup is a centred modal. It was a strip at the bottom of the page and was missed; it now sits in the middle of the viewport at 18px over a 60 %-black, 6px-blurred scrim that covers the page, which is also a visible target for the click-anywhere dismissal. (3) A region with no CF-IPCountry falls to the browser's locale before falling to USD: the header only exists behind the production proxy, so every preview showed USD to everyone, including a reader in Australia. The region subtag is read (en-AU → AU, zh-Hant-TW → TW), a region-less tag yields nothing, and a country header still outranks it. (4) The already-registered signup error carries a login button, because its message ends "Try logging in instead" and the form's own login link sits below everything the reader has just been told not to fill in.

  • An unrecognised region is priced in USD, not AUD (2026-09-13). Region detection already existed — CF-IPCountry picks the currency, and an explicit cookie choice still outranks it — but everything unmapped fell to AUD on the reasoning that EPIC is operated from Australia. Unmapped now means USD, which is the currency the underlying prices are quoted in and so the one figure on the page that is not a conversion. Australia is listed explicitly as part of the same change, along with its external territories and the three Pacific states on its dollar: AU had been relying on the very fallback that moved, so changing it alone would have priced Australian readers in US dollars — the one place it would have been obviously wrong. The no-country case is stated at the call site rather than deferring to Currency::default(), so the visitor-facing fallback and the type's default can move independently. Verified end to end against the running page: US/JP/GB/NZ/CA/XX → USD, AU/KI/NF → AUD, DE/FR → EUR, CN → CNY, no header → USD, and a USD cookie still wins over a DE header.

  • The bar's section links are centred on the page, not in the gap (2026-09-13). They had flex: 1 and centred themselves inside whatever space the two side groups left over, so the block sat half the difference between those groups away from the middle — measured at 14.7px right of centre, identically at every width, and it shifted every time either side changed: the currency control entering the bar moved it one way, leaving moved it back, and "Get Early Access" becoming "Login" moved it again. The side groups now grow equally from a zero basis, so the middle item is centred on the content box whatever sits beside it, and the content box is centred on the page because the bar's padding is symmetric. Measured 0px at 920–1900px, and still 0px with 200–300px of filler injected into either side group, which is the property that was actually asked for — centred at all times rather than centred for today's contents.

  • The currency control left the navigation bar for the comparison section (2026-09-13). It spent the day at both ends of the bar — beside the wordmark, then beside the action — and is now in the heading row of the only section where the choice changes a number, which is also where /currency has always redirected back to. A global control for a local effect is one people hunt for at the moment they want it. Three alignments, all of them properties of one flex row rather than measurements copied from the table, so none drifts when the table changes: space-between puts the control's right edge on .brand-table__wrap's right edge, the label keeps the content edge every other section's label sits on, and flex-start puts the control's top on the label's text top. Measured at 1280, 900 and 520: all three offsets are 0px. render_brand_document_for loses its currency parameter and brand_nav takes none, so every pre-login page renders the same bar.

  • "Join the Waitlist" is "Join the Community" (2026-09-13). Both places it appeared — the hero's primary button and the closing section's eyebrow. No visitor-facing "waitlist" wording is left on the landing page; the word survives only in the #join anchor, the form's /waitlist action and the operator console's own nav link, none of which a reader sees. It is now the section's heading ("Be part of the founding community.") and its lede ("our community portal") saying the same thing as the button above them.

  • The join form answers in a popup, and its two buttons branch (2026-09-13). The three outcomes the server already produced — recorded, already on the list, refused — are reworded to the operator's exact sentences and promoted to a dismissible popup that closes on any click or after five seconds. Rendered inline first and promoted by script, not the other way round: a fixed popup rendered by the server would have nothing to close it where scripting is off, and would sit over the page until the next navigation; inline, it is a banner that needs no dismissing. The three cases the server never sees are handled in the browser, and only there: nothing typed plus Notify Me does nothing at all, nothing typed plus CREATE ACCOUNT opens an empty signup page, and a malformed address stays put and says so. A valid address plus CREATE ACCOUNT carries it to /signup?email=…, which prefills the field. That value is a link anyone can send anyone, so it is trimmed, length-capped and dropped unless address-shaped — the renderer escapes, so the risk is not injection but what a hostile link can make the page say, and the answer stays "an address in a field you can see and edit". form.noValidate is set by the script rather than in the markup, so with no scripting the browser still refuses an empty or malformed address in its own words: the messages can only become ours where there is something to replace them with.

  • The join section leads with the account, not the waitlist (2026-09-13). The lede is two lines as written, the optional short-message box is gone with its counter and the maxlength constant that fed it — the server keeps rejecting an over-long message, which is the only one of the three checks an attacker could not skip anyway, and the field is still #[serde(default)] so the form posts fine without it. The emphasis is swapped: CREATE ACCOUNT is --primary at 44px and Notify Me is --ghost at 88% of that, which is what the copy now argues for. The pair's heights come from one token (--brand-form-action-height) with the submit written as calc(… * 0.88), so the ratio is the thing stated and a change to the primary height carries the proportion with it rather than silently altering it. Swapping it exposed a latent defect and fixed a live one. button, .btn — the pre-brand style the logged-in surface still uses — paints every bare <button> with --epic-accent-surface, which is blue; every .brand-btn--primary covered it with white, so the leak was invisible for exactly as long as every <button> on this surface was primary. The compare section's no-script "Recalculate" submit was already --ghost and already blue, and renders only with scripting off, so nobody had looked at it. .brand-btn now states background-color: transparent, which fixes the class rather than the instance.

  • A navigation panel, and one account action instead of two controls (2026-09-12). The pre-login bar carried five section links, a "Log in" text link and a "Get Early Access" button; below the design's md breakpoint the links wrapped into a second row, which made --brand-nav-height 7rem there. That row is why the logo sat under the bar on a phone — the hero's --brand-hero-lift pulls its content 4rem up toward the bar to close the dead space true centring leaves, .brand-hero both centres with align-items: center (which overflows in both directions once content exceeds the box) and clips with overflow: hidden, and on a short screen the lift had nowhere to go. The links now live in a <details> panel opened by a button to the left of the wordmark — the same disclosure the currency selector has always been, so it needs no script and has no focus trap — three bars coloured green-500 / green-300 / white, the hero headline's gradient stood on end. The bar is one row at every width down to 360px, and --brand-hero-lift is 0 below md where there is no dead space to reclaim. The two trailing controls became one that always reads "Login" and always goes to /login: a version that chose between "Login" and "Create an account" by reading the session cookie was built and then removed, because the cookie answers "has this browser signed in lately" rather than "do you have an account", and a button whose destination moves under the reader is worth less than one that always means the same thing. The panel is on every width, not only phones, and that is what makes the fixed label affordable — it is the only place "Create an account" appears on the pre-login surface, so hiding it at wide widths would leave signup reachable only by typing the URL, the same defect Sprint 9.8 found for /login. The inline section links now join the bar at 56rem rather than the design's md: measured at 768px they needed ~14px more than the bar had and took a second row, the tablet form of the same defect, and deferring them costs nothing when the panel is one tap away at every width. Above 56rem they are in both places, which is the accepted cost of one panel rather than two that drift. The waitlist is unaffected: "Join" remains a section link in both places and /#join is unchanged.

  • A local dev server for the authenticated surface (2026-09-09). cargo run -p epic-control --example dev-server serves the real router_with_middleware tower — the same CSP and response security headers production sends — on 127.0.0.1:8898 against the in-memory adapters, so /app/*, the playground and the operator console render with no Postgres, Ferrum, FerraVault or cluster. /dev/login plants a session cookie and redirects to /app, so a developer lands signed in rather than pasting a cookie into devtools. It closes the middle rung of the three loops collaborator-build-and-preview.md describes: ~29 s against ~1.4 s for the pre-login site and ~15 min for a preview, and it is the loop that was missing where the Phase 7 F23 defects lived. Three deliberate departures from the visual_regression.rs harness it is modelled on, each documented in place: the system clock rather than TestClock (a dev server reading "as of 8 months ago" on /app/fleet teaches the wrong thing), the is_epic_operator flag so /app/admin/* renders rather than 403s, and a session cookie written without Secure, which stops working the moment the example is bound to a LAN address to check a page on a phone.

  • A workstation access runbook, and the ignore rule it turned out to need (2026-09-09). docs/operations/collaborator-workstation-setup.md records what a second owner's machine needs to reach the git server, the image registry and the preview slots from outside the cluster's LAN, and the toolchain versions that make deploy/docker/build-image.sh work — the access layer under collaborator-build-and-preview.md's loops. Three facts it exists to stop the next person rediscovering: the registry validates on the name, so its tailnet address is deliberately absent from the leaf's SANs and connecting by IP fails; preview DNS is not published, but the slots serve a public Let's Encrypt wildcard, so curl --resolve substitutes for it with full verification; and registry access is network reach plus CA trust, which is push and delete on every repository there, mitigated for substitution by digest-pinned admission but not for deletion, since a digest-pinned pod still re-pulls on reschedule. Review caught that the document asserted a protection that did not exist: it described the token file as gitignored when git check-ignore said otherwise, and neither the pre-commit hook nor the secret-hygiene gate would have caught it — a Gitea token is bare 40-hex and matches no vendor prefix. .git-auth-config is now in .gitignore, and the gate gained a CREDENTIAL_FILENAMES check matching such files by name, since their contents carry nothing to recognise; tested in both directions, so the check cannot quietly stop working.

  • Raising a preview no longer needs cluster-admin (ADR-065). A preview namespace is now a slot — epic-preview-slot-<n>, committed in deploy/k8s/preview/slots.yaml, permanent — rather than a name derived from a branch. That is what makes the provisioner credential expressible: Kubernetes RBAC ignores resourceNames for create, so a namespace-per-branch provisioner needs namespaces: create cluster-wide and cluster-wide write on Deployments and Secrets to go with it, which is not a credential anyone can hand over. deploy/preview/mint-kubeconfig.sh issues a bound token for an identity that cannot create, delete or list namespaces, holds no pods/exec anywhere, reads exactly one production Secret (the preview wildcard), and holds no verb on RBAC objects. Three supporting changes: up moved off kubectl exec into the Postgres primary onto the one-shot Job the reaper already used — and sends a SCRAM-SHA-256 verifier rather than the password, so the plaintext never crosses the API server; teardown deletes a preview's objects by slug label instead of deleting the namespace, so the slot's RoleBindings survive; and the preview hostname is slot-derived (slot-<n>.preview.<base>, two permanent AAAA records published once by preview.py dns), which removes the Cloudflare zone-edit token from the raise path entirely. New Stage-1 gates in ci/preview/verify.py, all mutation-tested: slots.conf/slots.yaml/rbac.yaml must agree; teardown's kind list must cover every kind the base emits (and name none it does not); and the RBAC claims — no exec, no wildcards, no RBAC verbs, namespaces bounded by resourceNames, and the namespace-scoped ClusterRoles never bound by a ClusterRoleBinding — are asserted rather than written down.

  • Phase 9 — Portability, rebrand & collaboration ✅ COMPLETE 2026-09-01. Tag: phase-9-final (unsigned annotated — commit signing remains indefinitely deferred). Nine sprints, opened 2026-08-22 on phase-8-final. 90 §2 items across 9.1–9.8: 82 Delivered, 4 Partial, 4 Deferred at their sprint — and seven of the eight closed outright inside 9.9. The eighth — T-9.8.4, mail — had its whole chain proven end to end through the substitute path its own deferral named, and carries one verification step that no server-side check can make: reading a received message's Authentication-Results, which waits on Postmark account approval — a third-party review with no self-service path. Zero outstanding four-field deferrals, and one open verification, stated separately rather than folded into the first number. Three audits PASS: quality (workspace line 86.51 %, above the 80 % floor and above Phase 8's 85.16 %; 2 600 tests / 110 binaries / 0 failed), security, architecture 68/69 with one documented file-size deviation. Seven ADRs Accepted (058–064). Phase 6 (held open since 2026-06-20) and Phase 7 close at the same moment.

    The phase's thesis: three things that should have been configuration were facts of the source tree — the public domain, the host substrate, the address family. Its structural contribution is subtractive: epichub.ai went from 138 files to none the binaries run; five drifted client-address parsers became one resolution boundary; the manifest gained profiles instead of forks.

    What the gating sprint found is the part worth reading. Five defects, all fixed in-sprint, and three of them share one shape — individually invisible because the system was self-consistent around each:

    • Postmark default_from was read raw where ADR-059 derives it, so the client disabled itself at boot two log lines after the domain resolved successfully. A half-applied Sprint 9.2 migration whose sibling field had been converted and whose derivation had passing tests the entire time it was never called.
    • verify_email was sent under two different field names, and once that was unified, under one wrong URL path: every verification email ever sent linked to a 404. Unifying the model had made the two senders consistently wrong rather than inconsistently wrong. Invisible until a message reached a real inbox.
    • Three DR components built, documented, and never once executed — both backup CronJobs unapplied, both network policies unapplied, rclone absent from the workstation. No off-site backup had ever been attempted.
    • grep -q under set -o pipefail reported a loaded GPU driver as missing, because grep -q exits on match, the producer takes SIGPIPE, and pipefail fails the pipeline because the check succeeded. The sweep found the same shape producing a false negative in the Ferrum reseed drill, a Phase 5 gate.
    • cargo deny advisories failed on a yanked transitive crate (dev-dependency path).

    And the Ferron dial had never been authenticated. All three EPIC→Ferron Bearer keys in FerraVault were stale and identical, returning 401 on both /v1/admin/load and /v1/chat/completions. It presented as one narrow broken endpoint on an otherwise-healthy dial — the inverse of the truth, because /v1/capabilities is unauthenticated, so the single route that did exercise auth was the only one still complaining. That is the entire content of the T-9.5.8 deferral, "the dial is re-pointed but nothing has traversed it".

    Closed on the deployed plane, with live traffic: a real inference over the public domain on both address families (peer addresses recorded, 26 SSE frames streaming over IPv6); per-request kill-switch enforcement end to end (403 request_blocked → revoke → restored in ~6 s); the first off-site encrypted backup and restore (custody → 2-of-3 Shamir → decrypt → 569 restorable archive entries, ciphertext sha256 byte-identical to the upload); and the full-verify auth SLO measured for the first time at mean 0.977 ms, p99 ≤ 5 ms — roughly 50× the standing 0.100 ms figure, which had been measured with no valid key in existence and therefore described only the reject path. See Sprint 9.9 §7.

    Phase 7 closed at the same moment, and closing it meant fixing what it still held. Its register carried F23 — eleven authenticated /app/* pages scrolling sideways at a phone width, /app/admin/audit rendering 1009px into a 390px viewport — filed in 9.7 by the responsive gate the day that gate was written, and deferred there with its fix explicitly assigned to the walkthrough folded into 9.9. Tagging around it would have made Phase 7's own first exit criterion false. Fixed: .app-grid had no narrow-viewport case at all, and .app-table had no scroll container. Notable that the obvious half was not the whole — containing the tables took eleven pages to three, and the last three needed grid-template-columns: minmax(0, 1fr), because a bare 1fr track's minimum is its content's min-content width and one unbreakable model id widened the track and dragged the document with it. KNOWN_HORIZONTAL_OVERFLOW is now empty, so any page that overflows fails the build outright; the desktop pixel gate returned 0.00000 on every app_* page, proving the change is scoped to narrow viewports. The same run went red on six pre-login pages, and locating the diff before re-recording showed why: the differing pixels sit in one 14px band, the nav bar — the "Log in" link added the same day to a front door that had no way in. F24 (/app/security in the sidebar 404s — never built, its functionality having landed on /app/me) is operator-deferred to L19 and filed with the detail that matters: the AuthApiError::Conflict redirect lands an already-enrolled member on the missing page, so it is an error path and invisible until someone hits it. Three Phase 7 sprint rows (7.4, 7.5, 7.6) had also claimed IN PROGRESS for two months after shipping, corrected against their own checklists.

  • Sprint 9.8 — mail worked in no sense at all, and each of five breaks looked exactly like the others (2026-09-01, Phase 9). Reported as an operator unable to receive a login email — which was true about the mail and false about the login: nothing gates sign-in on email_verified, so the "check your email" screen after signup reads as a barrier and is not one. Behind it, five independent failures, every one of which alone produces "no mail" and none of which reports anything: the Postmark server token was never seeded; [postmark] default_from was read raw where ADR-059 derives it, so the client disabled itself two log lines after public domain resolved — a half-applied Sprint 9.2 migration whose sibling field had been converted and whose derivation had passing tests the whole time it was never called; epic-control had no network path to Postmark at all, the Turnstile policy saying so in its own header ("Turnstile only, NOT Postmark") while the reserved epic-egress namespace it defers to does not exist; the sender signature was domain-bound to the retired zone; and verify_email was sent as verify_url from registration and verify_link from change-email, which Postmark renders as empty rather than as an error, so one of the two paths had always delivered a verification mail with nothing to click and logged a successful send. That last one surfaced only because the merge fields had to be written down for the operator to author templates against — they are an external contract and had never been stated in one place. Fixed by construction (templates::models::verify_email, one builder, both call sites) rather than by aligning a string. Egress is a CIDR group, not toFQDNs: Sprint 9.5 measured that any L7 DNS rule redirects every query from the selected endpoints through Cilium's DNS proxy at ~20 s per name, and because the redirect is per-endpoint that would trade a blocked mail path for a broken control plane — so the file states its own width plainly (82 AWS us-east-2 prefixes, egress-only, one port, one workload) and names the narrower epic-egress answer all three CIDR groups should collapse into together. DKIM and the pm-bounces Return-Path published and verified, with no Postmark SPF include — correcting an instruction given earlier in the same sprint, since Postmark aligns SPF through the Return-Path and a second v=spf1 TXT is a permerror that fails every message rather than only the new ones. The ADR-059 gate then caught two hardcoded domain literals in this work's own new code. See Sprint 9.8 §7.

  • Sprint 9.7 — the pre-login brand surface, the waitlist, and three defects the design audit found (2026-08-26, Phase 9). The supplied epic-ai-solutions design ported to Leptos SSR: eight sections, self-hosted fonts, server-side currency selection from CF-IPCountry (the design's browser-side ipapi.co call — every visitor's IP to a third party with no processor agreement, on the page that also links the privacy policy — is gone), and a re-encoded backdrop at 171 KB against the source's 2.7 MB, compressed far past what a photograph normally takes because it sits under a 90 % black wash: measured against the source washed identically, the worst channel error anywhere in the frame is 10/255. An operator design audit found three things and all three were real. The "weird side panel" in Built on trust was .brand-grid keeping user-agent list styling — this stylesheet declines Tailwind's preflight, and unlike its two sibling grids that rule never reset it, so a 40 px inset painted as a stray panel because the grid's background is what draws its gridlines. The hardware cards' editable per-kWh rate was dropped in the port; rebuilt so it works with scripting off (a GET form the server renders from, so the result is a link you can send) with live recalculation added on top — and a browser probe caught the two paths disagreeing, the script rendering 350 where the server rendered $350. The colour drift was four separate slips, including the two comparison columns rendering the same kind of value in two different colours and the primary button dimming on hover where the design brightens. A second report — a horizontal scroll on a phone — produced a gate. .brand-orb--centre is a 37.5 rem decorative blur and only the hero clipped its own, so a 390 px page could be dragged 105 px sideways to look at it. The pixel baselines are recorded at 1280 px and could never have caught it, so responsive_pages_do_not_scroll_horizontally checks a property instead: every page at 390/430/600 CSS px, naming the widest offending elements, asserting on the width actually reached rather than the one requested (headless Firefox will not open a window under 500 CSS px, and a test that believed the answer would be quietly weaker on one engine). It was red on its first run, on eleven pages nobody was looking at — every /app/* and /app/admin/* page scrolls sideways on a phone, because .app-grid is a fixed 276 px sidebar at every width; filed as F23 and enumerated in a list the gate asserts exactly, so the debt is visible and can only shrink. The waitlist (migration 0051): operator-global and not RLS, because a pre-signup visitor belongs to no organisation; erasure is a real DELETE and the append-only audit log records a BLAKE3 hash, never the address, since an address written there could not be erased in turn. Its own §9.7 rate-limit band rather than sharing the abuse-report bucket, a role-address Postmark notification (the design mailed a personal Gmail through Resend), and an operator console whose CSV export defangs formulas typed into the public message field. The Postgres suite found a bug the in-memory adapter could not: erase_by_email bound the address as text, and citext = text resolves by casting the column down — an erasure request in a different case than the person originally typed would have matched nothing and reported success. And submitting against the live preview found that all three public forms drew a Turnstile challenge and then skipped verification when the verifier was unwired, reporting a protection that was not in force; closed with one accessor all three now read. See Sprint 9.7 §7.

  • Sprint 9.7 — one site instead of a landing page and everything else, and two accessibility gates that found real dead ends (2026-08-26, Phase 9). ADR-062 Accepted. Fifteen pre-login pages moved into the brand chrome, keeping their markup: a .brand-prose block re-points the shared content vocabulary at the brand tokens inside that column, so the dashboard re-skin later gets the same treatment instead of a second migration. Page metadata became one value where canonical_path: None causes the noindex directive rather than sitting beside it as a second switch, which makes both wrong states unrepresentable — an indexable page with no canonical is how duplicate URLs get indexed separately, and a noindex page carrying a canonical asks a crawler to both drop and consolidate the same URL. New /sitemap.xml and a Sitemap: line in robots.txt, derived from [domain]; a preview serves a valid but empty urlset, because listing production's URLs from a branch deployment invites a crawler to attribute one to the other. The SEO gate checks both directions with an independent page list (deriving it from the sitemap would make the check circular), and a third test requiring every public path to be in exactly one of the two lists immediately caught /reset-password and /verify-email in neither. Two contrast/landmark gates run in a real browser on all twenty pages, and building them correctly was most of the work: colours are canvas-resolved because Firefox returns oklch() verbatim and an rgb() regex on it produces confident nonsense; backgrounds are resolved by walking ancestors because measuring against document.body marks every dark-on-white button as failing at 1:1; and the first working version reported every element at 1.00:1 against itself, because fillRect with a transparent colour does not overwrite the pixel without a clearRect. The ramp was then re-measured against the background the page actually paints — .brand-backdrop puts the visible ground at #0d0d0d, not the #000000 token — which is lower contrast, and flattered exactly the two values closest to the line: ink-subtle was documented "large text only" and used on four pieces of 12–14 px copy at a real 4.02:1, and --color-brand-ink-faint was a token for a value this project's own comment says "FAILS AA outright, not given a token". Every /app/* page failed AA on its primary button and current nav item (white on blue-500 = 3.68:1); no single value fixes it, because the same token is a foreground on the dark ground where blue-600 drops to 3.76:1 — so the accent is two tokens now. Result: 20 pages, 669 text elements, zero below AA, with an empty known-failures list rather than a debt register. The landmark gate found a dead end: /app/usage and /app/fleet had zero landmarks and zero focusable elements, rendering a hand-written HTML string with no stylesheet, navigation, <main> or skip link — seven handlers did this, and for a keyboard or screen-reader user it is a page with no way back to the site, reached whenever a data source blips. Skip links (WCAG 2.4.1) on all four document shells, asserted to be the first focusable element; the playground's prompt textarea gained the accessible name a placeholder never was. Also: /transparency's failure is 503 + Retry-After in chrome rather than a bare 500 that a crawler reads as a reason to deindex; a 1200×630 social card with a test that parses the JPEG's own frame header, because a wrong aspect ratio is silently cropped rather than rejected; and ci/domain/verify.py now sees untracked files after reporting green on a tree holding thirteen literals that had not been git added yet — a gate that only sees committed work objects one commit after the literal is already in the history. See Sprint 9.7 §7.

  • Sprint 9.6 — preview environments: a branch running live, isolated by the systems that own the data (2026-08-26, Phase 9). ADR-063 Accepted. deploy/preview/preview.py up --branch <b> deploys a branch to <slug>.preview.<base> — its own namespace, its own LoadBalancer VIP, its own Postgres database seeded fresh, a 72 h TTL and an hourly reaper. Two measurements decided the architecture and neither matched the plan's assumption: the worker has 939m of CPU free and a Ferrum node under the single-node profile requests 500m, so a Ferrum per preview does not fit even once. Previews therefore share the substrate and are isolated by the system that owns each piece of data — a separate Postgres database with CONNECT on epic revoked; a separate Ferrum keyspace; a separate FerraVault mTLS identity and JWT signing key; and no internet egress at all, so a preview cannot mail from the production sender. That is the stronger arrangement (a NetworkPolicy protects a port; a Postgres grant protects a database) and it is written into the manifests rather than left to be found in a packet capture. The cap is a table of addresses, not a number: a slot is a pair of addresses a human configured on the router, so running out is a refusal naming which branch holds each one, and raising the ceiling means going to the router. The signing key is the sharpest edge — internal ids are sequential BIGINTs, so a preview signing with production's key would mint tokens that authenticate as real users; three independent things prevent it and Stage 1 asserts all three, twice. epic_store::KeyspaceScope lives on FerrumClient rather than on the adapters, because the first cut put that decision at fourteen call sites whose failure mode is silent. Two latent defects closed on the way: epic:diag:settings is a fixed Ferrum key, so any second EPIC deployment would have overwritten production's diagnostic settings with no collision involved; and datacl on the epic database was NULL, so PUBLIC held CONNECT and every login role in the cluster could open a session on it. Previews are noindex three ways (robots.txt, the meta tag, and X-Robots-Tag near the outside of the tower so it covers a 429, a timeout, a panic-to-500 and every asset with no <head>), plus a banner on every page — production byte-for-byte unchanged, the robots tag absent rather than index, follow. Proven live, not asserted: a preview raised end to end on the cluster serving real Let's Encrypt TLS with chain validation passing, its own keyspace and database confirmed in the boot log and pg_stat_activity; and the reaper drill run against a real expired preview, a real database and a decoy namespace — it reaped exactly the one preview, leaving the decoy, every production namespace, the epic database and all seven cluster roles untouched. Ten mutations on the guard and slug and ten on the gate are all caught, and the process found four defects in this sprint's own work, each an assertion true for the wrong reason — a guard claiming three independent conditions where one was implied, a slug test that passed with case-folding removed, a banner assertion matching the inlined stylesheet, and a gate check asserting at least one key-scope application rather than one per dial site. Six more came only from execution: python3-minimal is the interpreter without the stdlib; epic-state is default-deny so the teardown Job hung rather than failed; psql -c "A; B" runs both in one transaction and DROP DATABASE cannot; namespace-first teardown leaves an invisible orphan database; kubectl_json read every non-zero exit as "does not exist"; and a preview's derived app_origin was production's apex, so its links pointed at the live site. A post-close self-review found nine more, one critical: preview.py used PEP 701 f-string syntax and therefore did not parse on the Python 3.11 the reaper image runs — the deployed reaper worked only because its image predated the line, and the next rebuild would have produced a reaper that died at startup every hour until the host filled. The image now compiles its scripts with the interpreter that runs them, so that class is a failed build. Also: one failing teardown aborted the entire reap run, so a single stuck preview blocked every other one forever; the teardown diagnostic could be empty (a pod that never starts has no logs) and now leads with Job conditions and container states; the branch name went raw into generated YAML, which git check-ref-format proves is reachable; epic_store::KeyScope was renamed to KeyspaceScope because epic_core::KeyScope already existed and the collision had produced a doc-link ambiguity that was patched rather than removed; a new gate checks the boundary keys actually cross rather than which files mention scoping; certificate-expiry parsing was locale-sensitive; copy_tls_secret would apply a well-formed empty TLS Secret; and slot addresses were never validated as addresses. The in-cluster reaper had never executed its destroy path — its one scheduled run was a dry run with zero previews — so it was drilled properly, from the CronJob under its own ServiceAccount, against a real expired preview and two decoys including one with a mismatched slug. The rename also produced a guard worth keeping: because a preview default-pins to production's image, an env override the running binary does not understand is silently ignored, so up now reads the keyspace posture back out of the pod's own log and refuses rather than reporting a preview isolated without evidence. One live production defect was caused and fixed in the same session, and it is the most instructive finding here: re-pinning epic-control bumped the web Deployment and not the meter aggregator, which shares the web Service's selector on the explicit basis that it is the identical binary. The result was two pods behind one public address serving different builds — /robots.txt returned 200 404 200 404 200 404 404 404 200 200, an exact 50/50 split that reads as a flaky route rather than a version skew, while every pre-existing page stayed 200 because it exists in both builds. It was visible only because this sprint had just added a route that one build lacked; a behavioural difference would have presented as an intermittent bug with no pattern. check_image_parity now requires every epic-control digest pin under deploy/k8s/ to be identical, because an invariant living in three files cannot be held by a comment in one of them. Two of the ten mutations on the new gate checks survived the first pass, both real: one check was satisfied by its own explanatory comment, and another's vacuity floor sat below the real count. See Sprint 9.6 §7.

  • Sprint 9.3 — IPv6 readiness, code plane: the address family becomes a configured value (2026-08-23, Phase 9). ADR-060 Accepted. New epic_core::net (pure, no I/O) — BindSet (validated non-empty bind list; parses a string, a comma-string or a TOML array; rejects duplicates and wildcard/specific conflicts at config load rather than at bind()), the single v4-mapped normalisation point, the forwarding-header element parser, the trusted-hop ClientIpPolicy, and Ipv6PrefixLen + rate_limit_identifier — plus a new epic-net crate that binds one socket per family with IPV6_V6ONLY set explicitly, so behaviour never depends on the host's net.ipv6.bindv6only sysctl. Every listener on both planes (edge h3 + h2, control h2 + h3, /metrics on both) now takes a bind list; the h3 listener binds its own UdpSocket and hands it to quinn::Endpoint::new rather than letting Endpoint::server inherit the sysctl. EPIC_*_BIND accepts the comma form, since env is the one layer without TOML arrays. ADR-026 invariants re-measured under v6, not inferred: 9 allocs/token over IPv6 (vs 11 over IPv4) and cancellation p50 8.5 ms / p99 10.7 ms with the peer family alternating across the sustained-load drill; the dual-stack listener proved on real sockets, both families on the same port, each completing a TLS request through the real router. T-9.3.10 verified against sibling source (record): Ferrum and FerraVault both accept bracketed IPv6 and already carry an SNI override; Ferron was the gap and it was EPIC's — tonic derives its SNI name from Uri::host(), which for a bracketed IPv6 authority includes the brackets and is not a valid rustls server name, so https://127.0.0.1:50051 configures and https://[::1]:50051 does not. Fixed with [ferron] tls_server_name, matching the field both other siblings already take; not hypothetical, since the deployed edge dials Ferron at an IP literal (ADR-039). Also recorded for Sprint 9.4: Ferrum's client returns v4.or(v6), so a dual-stack Ferrum Service is dialled over IPv4 — deliberate upstream behaviour, not a defect. Deployed ConfigMaps stay IPv4-only deliberately (the cluster is SingleStack; binding [::] on a pod with IPv6 disabled crash-loops at socket()), with the list syntax documented so Sprint 9.4 is a one-line flip. See Sprint 9.3 §7.

  • Sprint 9.2 — Domain indirection: the public domain becomes a configured value (2026-08-23, Phase 9). ADR-059 Accepted. epichub.ai appeared in 138 tracked files, 484 times; it now appears in five, none of them code the binaries run — the parity proof, one comment in an applied (checksum-immutable) migration, the generated epic-domain ConfigMap plus the gate that reads its source, and FerraCert's own config. No Rust source file, no Kubernetes workload manifest and no configuration either binary reads contains it. New epic_core::domain derives every host, origin, cookie domain, role mailbox and ACME SAN from one [domain] base, resolved once at boot into an immutable DomainSnapshot (newtyped DomainBase/Host/Origin; Host::is_within is label-aware, so the origin allowlist refuses a look-alike registration that ends_with admits). Deployed values arrive as EPIC_DOMAIN_* from the generated epic-domain ConfigMap via envFrom — no templating step, so ADR-016 §3's vanilla kubectl apply posture is unchanged; deploy/domains/check.py verifies the deploy-layer half (FerraCert SANs, brand identifiers, Cargo homepage) that cannot be substituted. K8s label namespace epic.epichub.ai/ → epic.internal/ across 35 files (ICANN-reserved, so it can never track a rebrand again). No behaviour change, proven by crates/epic-config/tests/domain_parity.rs, which pins every derived value against the literal it replaced. New Stage 1 gate ci/domain/verify.py (22 tests incl. the end-to-end negative) makes the next hardcoded literal fail the build, and already polices epic-ai.solutions ahead of the Sprint 9.8 cut-over. Four latent defects found and fixed: a Kyverno cosign policy naming a registry host that has never existed (would have admitted every image unverified while reporting healthy); a chaos runbook left selecting the old labels; Cargo.toml/OCI image.source pointing at a non-existent GitHub repo; and epic-config's diff_boot silently not covering the two Sprint-3.3 signing sections. A post-close self-review found five more, all fixed rather than recorded: the visual-regression gate was failing on six pages (the SSR identity was never installed in the harness — now it resolves production's, and all 20 pages return pixel-delta 0.00000 against pre-sprint baselines, which is the real no-behaviour-change proof); AppState::wire fell back to example.test with a warning and now fails closed, because a control plane that boots without a domain comes up healthy and serves a wrong site that no readiness probe catches; allows_browser_origin allocated on the per-request path while four documents said it did not (rules unified into one allocation-free check_host, pinned at exactly zero by a counting-allocator harness); Origin serialised to a string but deserialised from a struct, so it did not round-trip; and deploy/domains/check.py --env dev|preview reported 19 failures for manifests that were never theirs (a check that goes red when used as documented teaches people to ignore it). Because both planes now fail closed, check.py gained a seventh check — every plane workload must receive EPIC_DOMAIN_BASE and its namespace must hold the ConfigMap — with a vacuity guard, and every check was mutation-tested to confirm it is not inert. See Sprint 9.2 §7.

  • Sprint 9.1 — collaboration & git-server enablement (2026-08-22, Phase 9). EPIC becomes a two-owner project: ADR-058 (review authority, why the sprint documents stay authoritative over the issue tracker, the break-glass override, the fast-loop/ceremonial-gate split) and ADR-064 (a phase takes its number when it opens and never gives it up; unopened phases are named — ending three renumbers in ten weeks). Ships CODEOWNERS with a ci.sh lint against inert rules, PR + issue templates, .gitea/workflows/README.md, a CONTRIBUTING.md second-developer path executed literally on a clean container, the gate on push-to-main with deduplicated issue-on-failure, .env/credential sweeps in Stage 1, and tools/sprint-sync — a one-way sprint-document → Gitea bridge (9 milestones, 102 issues) with a drift check proving the board cannot diverge from the contract. A self-audit pass then found and fixed eight defects in that work, three of them live — most seriously that Gitea caps an issue page at 50, so the failure reporter's dedup was blind to any report older than the 50 newest and would have filed a duplicate on every run (the nightly carried the same bug); and that the secret sweeps gave different answers under GNU grep and ugrep, so they now run on one engine with unit tests. See Sprint 9.1 §7.

  • Sprint 8.2 — shim tool surfaces + compat hardening (2026-07-03, Phase 8). The OpenAI + Anthropic shims carry tool-calling end-to-end and land the generic guided/structured-output surface (ADR-055 Accepted). Native guided substrate on InferRequest (illegal states unrepresentable; threaded to Ferron's guided(18) field). OpenAI: tools/tool_choice/response_format; message.tool_calls (content null on tool-only turns); streaming delta.tool_calls[] fragment forwarding (never buffered); assistant tool_calls + role:"tool" history round-trip. Anthropic: tools/tool_choice/output_format; tool_use/tool_result content blocks (Phase-3.2 reject arm removed); six-event SSE machine extended with lazy dynamic block indexing (content_block_start(tool_use) → input_json_delta → content_block_stop); POST /v1/messages/count_tokens (no Ferron dial) + lenient anthropic-version. Conformance gates extended (tool-calling + structured output; SDK re-pin openai 2.44.0 / anthropic 0.116.0 — 15/15 + 11/11 green); real-SDK agentic-loop drill recorded (ci/drills/phase-8-harness/). ADR-026 re-asserted on both shim tool streams (zero per-token alloc, cancel < 100 ms, ~256 ns/~120 ns per-fragment reframe). See Sprint 8.2.

  • Sprint 8.1 — native tool-calling substrate (2026-07-03, Phase 8). EPIC's native API surface gains tool-calling end-to-end against Ferron's grammar-constrained engine (ADR-054, Path A): re-vendored Ferron proto (sha256 provenance guard); native tools/tool_choice/tool_calls + StreamEvent::ToolCall{Start,Delta,End}; edge renders tools into the model chat template; translate.rs maps the structured tool frames (Malformed stub removed); usage.prompt_tokens sourced from Finish.prompt_tokens. Live drill PASSED against the real Qwen2.5-3B-Instruct-FP8 tool-calling build. Roadmap renumber: new Phase 8 (tool-calling) inserted, Launch → Phase 9, hardware → Phase 10+. See Sprint 8.1.

Changed

  • A due OFAC re-screen no longer fails every open PR at once (2026-10-25). seed_re_screen_is_not_overdue fails at 30 days after the list's # published: date by default, and at 20 in the nightly sweep (EPIC_OFAC_RESCREEN_DUE_DAYS), whose failure issue is now the prompt. From day 20 ci.sh Stage 1 also warns in every gate's log without failing. The guard still refuses every signup at 35 days. On 2026-10-24 the old 20-day test failed main and every open PR, with 15 days still to go before production was affected.

  • /pricing and /transparency say account and pool, not group (2026-10-23, #400; Glossary #381). Both pages carry the owners' wording (Melvyn, 2026-10-23; wiki Glossary §8) with typos and light polish fixed. /pricing is four sections — Pricing, Slots (Term, Capacity, Sharing), Models, What is not in scope — and still names no price: the configured slot prices are seeded, in AUD, and not read by this stateless page, so it says a guideline price will be given, in the landing page's words, until the owners decide to publish one. /transparency counts and suppresses accounts, keeps its live threshold, day counts and figures, and says the page fills in once the first server comes online. Tests pin the new text and that no price figure or "group" appears.

  • orgs.seat_count is slots_held, and a ballot's seat_count is vote_weight (2026-10-23, #381). The owners' names for what the two columns hold since ADR-069: an account's slots that are active or on hold, of any kind and with any rights; and a ballot's weight when it was cast (Org::vote_weight). Migration 0094 renames both, their three constraints and PG18's NOT NULL constraint names, and recreates enforce_seat_limit() and slots_recount_orgs(), whose bodies named the column. Org::seat_count, BallotInsert/BallotRecord/CastRequest.seat_count and the test builders follow. For one release each old name stays as a read-only generated column, with the edge's column grant copied to it in the same transaction, so the planes can roll one at a time and roll back. The migration refuses to run while a vote is open, since a control plane on the release before casts into the old ballot column. The next release drops both (docs/operations/migration-runner.md §5). Wire names stay: the Postmark slots_remaining key and the seat_count key in model_vote.cast audit payloads. ADR-014 and ADR-069 gain amendments.

  • The avatar upload limit is 10 MiB, down from 15 MiB (2026-10-22, owners' decision on the #372 review; ADR-074 §2, amended). The large platforms cap a profile photo at about 5–10 MB (Google about 5, LinkedIn about 8, Discord 8–10), and a stored avatar is at most 256 × 256, so the extra 5 MiB a body could hold bought nothing a member sees. ADR-074 chose 15 MiB so that every default phone JPEG (5–12 MiB) fit; most still do, but a phone photo over 10 MB now gets the "too large" page. That page now names the limit, read from AVATAR_MAX_BODY_BYTES so it cannot drift, and says how to fit under it: make a smaller copy of the photo, or choose a different picture. With JavaScript the avatar cropper uploads a 512 × 512 square, not the photo, so only a browser without JavaScript, or a file the browser cannot open, meets the limit. Memory: at most 4 bodies in flight per replica is now 40 MiB, not 60, and the replica's worst case ≈ 560 MiB, not 580. body_limit::the_avatar_limit_is_ten_mib pins it: 10 MiB passes, one byte more and 15 MiB get the page.

  • One word for the account on screen, and the stale fair-share formula fixed (2026-10-22, #381). The member portal and the operator console called the thing that holds slots a "group"; they now say "account": the console's Accounts tab, Suspend account and Unsuspend account, "account not found", the checkout's Account name (optional), and the Fleet, Security, keys and audit-log lines. The signed-in Group tab is now Pools (Melvyn, 2026-10-23), the tab ADR-082's PL.4 turns into the pools tab: its label in the tab bar and in Profile's opening-tab list, its page title and hidden heading, its search section, its name in the console's paused-boxes list, and the Members box's line, "Everyone in your pools, and their role." Its stored ids (MemberTab::Group, page key group) and its route /app/org are unchanged. Its "Group details" card is Account summary, because the Account tab already has an "Account details" card. The pre-login invitation page keeps "An EPIC group", and the public pages are unchanged. The console's per-server "Pools" card is now Paid and complimentary places, and the test-key option says "sandbox models" (owners, "for now"), leaving "pool" to ADR-082's pools. Stale definitions fixed: fair_weight = seat_count in DESIGN.md (§6.3, §8.3, §9.5, §10.4, §12.3, Appendix A) and in code comments, and ADR-045 and ADR-048 marked superseded in part by ADR-067 §1 and ADR-069 §4 with their text kept; the weight is max(slots_capacity, 1) (Org::fair_weight). Also Appendix A's slot entry ("2,500 will be sold") and ADR-066's SubscriptionEvent (now PurchaseEvent). On-screen text and docs only: no stored key, route, error code or column changes.

  • Changelog entries are one file per PR in changelog.d/, so PRs no longer conflict on CHANGELOG.md (2026-10-22, #376). Every PR used to add its entry at the top of the same section, so each merge to main left the other open PRs conflicting on the changelog alone: six of the eight conflicts on 2026-10-22 were nothing else, each costing a merge-in and a fresh PR Gate run. A PR now adds changelog.d/<PR number>-<words>.<section>.md. ci/changelog/compile.py folds the fragments into CHANGELOG.md and deletes them, only on main, as the first step of building a production image: deploy/docker/build-image.sh refuses a production build while any fragment is committed or staged, because the public /changelog page embeds CHANGELOG.md at build time and a build cannot commit; a preview build passes --preview (tag -preview, label epic.internal.image.preview=true). Each entry is placed by the date in its title's closing parentheses among the entries already in its section, newest first, and same-day entries by PR number, highest first; it ends with a hidden <!-- changelog.d/<name> --> marker, so a fragment that comes back after compiling is refused even if the entry was edited, and a later compile knows the entry's PR number. The gate refuses a missing or future title date, two dates, a title that does not close on its date, a title already in CHANGELOG.md or in another fragment; only files git tracks are compiled, and trailing spaces are kept. ci.sh Stage 1 checks every fragment (ci/changelog/verify.py, 89 unit tests); the domain gate treats a fragment like CHANGELOG.md. CLAUDE.md, sprints/CLAUDE.md, CONTRIBUTING.md, the PR template and the preview and migration runbooks say so.

  • The dashboard modules' test routers become one test-only helper, and the h3 body-limit test drives the mounted tree (2026-10-22, #358 review). The 21 dashboard modules' router()s, called only by tests since #358, give way to one #[cfg(test)] routes::web::test_router(routes) that folds a module's routes() as web_router folds all_dashboard_routes(), so production cannot mount a dashboard module apart from the list (referral::public_router() and email_change::public_router() stay). integration_h3_body_limit serves router_with_middleware instead of avatar::router(), and a new test stores a real image over 64 KiB through the mounted avatar route over h3 and refuses one over 15 MiB; removing the route's DefaultBodyLimit fails it. No route changes: ci/routes/verify.py reads 56 controls against 221 routes, its dashboard fixture now in the tree's new shape.

  • A newly published RustSec advisory no longer fails every open PR (2026-10-24). cargo audit and cargo deny check advisories run on the PR Gate only when Cargo.lock, a Cargo.toml, deny.toml or ci.sh (the two ignore lists) changed since the base (ci/advisories/should_run.py, EPIC_ADVISORIES=changed); the log says why either way. cargo deny's bans, licenses and sources still run on every gate. The nightly sweep and a local --full run the advisories every time, and the nightly's failure issue is where a new advisory is now reported. Prompted by RUSTSEC-2026-0387, which failed main and every open PR on 2026-10-24 within an hour of its publication, for a crate only FerraVault's test server pulls in.

  • The PR Gate does less avoidable work (2026-10-22, #380). It had grown from ~65 to ~145 minutes. Four changes: (1) cargo geiger, ~45–70 minutes and informational, runs on the gate only when a dependency manifest changed or unsafe was added since the base (ci/geiger/should_run.py, EPIC_GEIGER=changed); the log says why either way, and the nightly sweep and a local --full still run it every time. (2) The test profile carries line tables instead of full debuginfo, which makes a test binary ~60% smaller (an epic-control suite went from ~1.6 GB to ~0.6 GB). (3) Each crate's integration suites build as one binary, tests/integration/main.rs, with each suite a module under its old name: 130 test binaries become 38. The 16 that keep their own process say why in their header (a #[global_allocator], a latency measurement, a WebDriver, or process-wide state), and ci/test-layout/verify.py (Stage 1) fails on an unexplained second binary or a suite main.rs never declares. Before and after list the same 788 tests by name. Run one suite with cargo test -p <crate> --test integration <suite>::. (4) The runner goes from 4 CPUs / 16 GiB to 6 / 48 (EPIC_CI_JOBS 6, EPIC_DOC_JOBS 4); docs/operations/host-resource-split.md records the memory trade.

  • The pre-login pages in the owner's logo pack (owners' second landing round, 2026-10-21). The hero shows the owner's website badge, served as his own Badge.svg (26.8 KB over the wire, sharp at any zoom); the pre-login bar and the comparison table's EPIC column show the EPIC letters alone, as a vector (epic-header-mark.svg, extracted from his text-only logo by tools/make-header-mark.py). The pre-login pages take the logo's palette, defined once on .brand-page: the hero headline word by word in the strapline's colours, section headings in a blue–green–orange sweep, a green accent, blue links, orange figures, and a call to action in the badge frame's band. Every pair clears WCAG AA (tools/palette-contrast.py), and ink-subtle moves to oklch(66%) so small print stays above 4.5:1 on the new ground. The photograph behind the pages is replaced by a star field like the owner's social banners (tools/make-starfield.py: two depths of seeded stars over the banners' teal glow, 6.5 KB where the photograph was 340 KB), whose stars brighten around a hovering pointer (starfield.js; nothing on touch screens, no fade under reduced motion). Two extras, on the landing page only and behind one switch (pages::landing::fun::FUN_EXTRAS): an occasional shooting star, and the badge turning to face the pointer; neither runs under reduced motion. The dashboard is unchanged.

  • The Dashboard's Models box: the owners' intro, and which access applies to you (2026-10-21, #367; issue #276). The intro is Maran's wording as the owners accepted it, and a line under it says whether the viewer's slots give unlimited access, an access code gives access until a stated time, or neither yet, with a link to the playground's code form and, for an owner or admin, to the slot options (the new access::can_buy_slots, which /app/billing now uses too, so the box offers a purchase only to those checkout lets buy). The API keys section and the playground page follow the same rule (owners, 2026-10-23): their "See slot options" is shown only to an owner or admin, and anyone else is told slots are bought by an owner or admin of the account. The box says that too, and the header search offers "Buy slots" only to an owner or admin, in its suggestions and its results (filtered per request, like paused boxes). It is decided by the new epic_control::access::model_access (the API right, else the playground's own playground_access), and the box and the playground page show the access code's end through the same formatter, so the two cannot disagree. Suspension wins over a running access code (owners, 2026-10-22): a member of a suspended or closed account gets no access from one on the playground page, the key mint or the box, and a code redeemed for such an account is left unused with a notice saying why (ADR-067 and ADR-069 amended, DESIGN.md updated). The app_home visual baseline needs re-recording.

  • The member dashboard is mounted from the list the appearance sweep reads (2026-10-21, #320 item 9; issue #199 C14). web_router merged each dashboard module's router() by hand, while all_dashboard_routes() listed the same modules for the appearance sweep, so the two could disagree. web_router now folds all_dashboard_routes() itself: a dashboard route is served exactly when it is listed. The two modules that also serve a public page get a public_router() for it (referral's /r/{code}, email_change's confirm and cancel pages), mounted beside the list; each module's own router() stays, for its tests. ci/routes/verify.py names all_dashboard_routes a collector, as it does the console's all_admin_routes: what it lists is mounted, and a module it leaves out serves nothing and fails the gate (mutation-checked on the real tree with keys dropped; two new fixture tests). No route changes: the same 221 routes are served.

  • Tests: the cancel-budget test bounds its median against the runner's own baseline (2026-10-21; issue #357). cancellation_budget_holds_at_100_rps_sustained failed a PR Gate run with no code change: its median, measured from the batch start, is mostly how fast the runner moves 100 requests to their first chunk, and on the 4-CPU runner that sat at 21–25 ms against a fixed 21 ms bound. Seconds now alternate (ABBA) between cancel windows and control windows running the same batch, where the control samples the same streams' first chunk and drops them after the window. A cancel sample is time-to-first-chunk plus the cancel path and a control sample time-to-first-chunk alone, so the test bounds the difference of the medians, cancel p50 − control p50 ≤ 10 ms, which isolates the cancel path and does not widen on a slow runner (a ratio would have allowed ~21 ms there). It keeps the 200 ms per-iteration bound and an absolute 50 ms backstop on the median, looser than the old 21 ms because time-to-first-chunk alone reaches 21–25 ms on the runner. Both windows now settle before the next starts, and every window must observe all of its cancels (or first chunks) inside the sampling window, so a late or lost cancel fails instead of dropping out of the median. Every run prints both medians, the difference and the bounds past libtest's capture, so passing PR Gate runs record the runner's numbers. On the laptop the difference read −2.8 to +1.4 ms on unchanged code; a 12 ms delay in the cancel arm fails it, a 7 ms delay does not, and 1 ms of blocking per cancel sits at the bound (failed 3 of 6). Each peer family now carries half the cancels (they all went over IPv4). The default run is 20 s, keeping ~500 cancel samples. Test-only.

  • One electricity rate for the six "real cost of going local" cards (owners, 2026-10-20). The per-card rate fields are replaced by one field at the end of the introduction (?rate=, still a GET form that works without JavaScript); each card now reads "Annual electricity @ " in the currency's own unit ($/kWh, €/kWh or ¥/kWh). The rate is bounded at 10 per kWh (was 100) and a single decimal comma is accepted. The hint names what the default is an average of, per currency: a US, Australian or Chinese national average, and a eurozone average for EUR (it said "a national average" for all four). The field takes the 2px accent focus ring the other brand controls have (it had a 1px border colour and no outline). A rate it cannot use is said in words under the field, and the field is aria-invalid; the red border follows aria-invalid, so colour is never the only signal (WCAG 1.4.1, 3.3.1). A repeated rate or joined parameter keeps its first usable value, where it had become a plain-text 400. The landing page's wording is updated where the owners rewrote it (Why EPIC heading and hardware footnote, How it works step 3, two principles, the join lede), and the join lede's two lines no longer run together as text. GB300 is no longer considered (owners, 2026-10-22): /about names the hardware the landing page does, the NVIDIA Vera Rubin NVL72 and the AMD Helios; the model-proposal form says "the fleet's hardware envelope"; and a test holds the landing, about, pricing and security pages and the proposal form to never naming GB300.

  • Tests: RLS cases for the four user-scoped tables that had none (2026-10-16, #307; issue #200). The tables from migrations 0059, 0062, 0066 and 0068 each gain a positive and a negative case under the NOSUPERUSER NOINHERIT NOBYPASSRLS test role; each fails with the policy dropped, RLS disabled, or USING (true). Test-only.

  • Tests and docs: the appearance review follow-ups (2026-10-16, #309; issues #199, #201, #203, #207, #210). The AA contrast proof uses ThemeBase::grounds() itself; new tests hold grounds() and appearance.js's live-hint grounds to the stylesheet's --epic-bg values; the account menu's identity and the console scheme gain assertions; references left stale by the ADR-073 split are corrected. The logout test's doc comment, which had fused onto the legal-items test, moves back onto its own test: the last open part of #210, whose file sizes and other fused doc comment were already fixed on main. No behaviour change.

  • Tests: the avatar pipeline on WebP, a real downscale, and uploads over 64 KiB through the production tower (2026-10-16, #317; issue #204). Three tests drive the real process_avatar_upload (format sniffing, decode limits, Lanczos3 resize, PNG re-encode). Test-only.

  • Tests: the signed-out sweep covers every dashboard route and method (2026-10-16, #318; issue #199 C17, C18). The sweep derives its routes from all_dashboard_routes() and sends GET and POST to each until the handler itself answers, and the scheme gate's floor counts what it renders. Test-only.

  • Production Ferrum rolled sprint-9.16 → 0.3.0-fix-121: idle CPU 1.27 → 0.15 cores (2026-10-20, #293, ferrum#121). An idle ferrum-0 spent ~1.27 cores in its two ferrum-raft threads, woken ~63,600×/s, because Ferrum's slot-migration controller waited on a fresh clone of openraft's metrics watch each time and so never slept. Ferrum fixed it in chart 0.5.0 (tag ferrum-chart-0.5.0, 4d17d639). Measured on the live pod the way #293 specified (process CPU over 30 s, per-thread voluntary_ctxt_switches over 5 s), ten minutes after the roll: 0.150 cores process total, 0.004 of it Raft, Raft wakes ~70/s. What remains is the two shards and the wasm-epoch thread on their ~920/s tick, unchanged by this release. Image 4d17d6392f5a-0.3.0-fix-121, pinned to Ferrum's own manifest digest sha256:6633aadc…. From 0.5.0 Ferrum publishes to the private Gitea registry, and the chart's default needs a pull secret. EPIC instead mirrored the image into registry.epic.internal:5000 with crane copy, which keeps a single-arch manifest byte-identical, so the cluster holds no Gitea credential and the pin is the upstream digest. No template, config, wire, client-crate or on-disk change: the rendered release differed from live only in the image digest, and the server-side dry run passed. helm upgrade --rollback-on-failure (release revision 8). Ferrum was down ~25 s (07:14:11–07:14:36 UTC). Both edge pods, control and the aggregator re-seeded in-process within 20 s (ADR-126), with no pod restarts. The CI test-server pin (ci/ferrum/test-server.pin) moves with it. The six Ferrum-backed suites ran strict against the new binary: 20/22 pass locally, and the two integration_adapter_stack cases that need a Postgres testcontainer could not start one because the workstation's Docker is down. The release's other fix also landed: shards now pin only within the process's CPU mask, so those suites no longer abort on this workstation's user.slice (sched_setaffinity failed for core 0). Rollback: helm rollback ferrum 7. Record: deploy/k8s/epic-state/ferrum-values.yaml.

  • The second sign-in factor has one name: Multi-Factor Authentication, MFA for short (2026-10-16, #320 item 11; issue #208). #197 renamed it on the Account page, but the setup page, the page after enrolling, the login step and their error messages still said "two-factor", so "Set up Multi-Factor Authentication" led to "Set up two-factor authentication". They now use the Account page's wording, the login page included. A new test, the_mfa_setting_has_one_name, fails on "two-factor" in any string literal of epic-web or the control plane's web routes. The search index keeps "two-factor" and "2fa" as synonyms. None of these pages has a visual baseline.

  • Transactional mail verified on a received message (2026-10-14, #91). Postmark approved the account; a real password reset reached an outside (gmail.com) address with SPF, DKIM (d=epic-ai.solutions) and DMARC all passing. Closes Phase 9's open verification (T-9.8.4 / matrix row 13); T-9.8.4's role-address routing remains unverified — Sprint 9.8 §7, Sprint 9.9 §7.

  • The member's avatar has a versioned URL, cached for good (2026-10-16, #320 item 10; issue #209). Every authenticated page showed <img src="/app/me/avatar">, revalidated on each view (private, no-cache and an ETag), so each page view ran a second authentication and an image read. Migration 0085 adds users.avatar_hash, a generated column holding the first 128 bits of the image's SHA-256. Pages now name /app/me/avatar/<hash>, read with one primary-key lookup (Store::user_avatar_version). That URL is served private, max-age=31536000, immutable, so the browser fetches it once. A new upload is a new URL, and an old version or the bare URL redirects to the current one. The leaderboard's avatars keep their fixed URL and ETag. ADR-074 §3 is amended.

    • Signing out clears the browser's cache (#346, the owners' decision). Under immutable the browser no longer asks for the avatar, so a shared computer would keep showing it after sign-out. Every response that signs the browser out now sends Clear-Site-Data: "cache" with the clearing cookie: POST /logout (on the 303, which browsers honour), POST /api/auth/logout, the account deletion (POST /app/me/delete and POST /api/me/delete, which now also clears the cookie) and the email-change confirmation. The header names only the cache. "cookies" would clear every subdomain's cookies, and nothing in browser storage is the member's (the launch figures are the same for everyone). ADR-074 §3 and DESIGN.md §9.7a record it.
  • The HTTP/3 comments match ADR-026 (2026-10-14, review notes on #267). The workspace Cargo.toml block and epic-edge's module docs still described an io_uring-driven, quinn-proto-only edge listener; since Sprint 2.3 both planes serve HTTP/3 through quinn + h3-quinn + h3. The #265 entry under Security now names every path quinn-proto arrived through (ferrum-client/ferrum-protocol as well as quinn). Comments and changelog only.

  • The rest of the "quinn-proto-only edge" wording follows ADR-026 (2026-10-14, Maran's review of #271). epic-edge's listener module docs, its package description and its tokio comment (which listed listener/h3.rs among the runtime-free pipeline modules), epic-control's Cargo.toml and main.rs, and the Talos worker patch still described a hand-rolled quinn-proto edge listener. ADR-011 now carries a "superseded in part by ADR-026" status note, and ADR-024's listener bullet a dated annotation; both records are otherwise unchanged (append-only). The workspace note names all five ADR-026 invariants, io_uring portability included, and says the pipeline modules import no runtime crate beyond tokio::sync. Comments and records only.

  • No more cfg(loom) claim (2026-10-15, #302 item 1, Maran's review of #298). epic-edge's tokio comment said the pipeline modules compile under a cfg(loom) swap-out for crossbeam channels; neither exists in epic-edge. It now points at the check that does run, the CLAUDE.md §3.7 rg invariant. ADR-026's table, where the claim came from, gains a dated note: row 5's swap-out was never built, and row 4's rg returns the one sanctioned tokio::sync::mpsc import, not 0. Comments and records only.

  • Production Ferrum rolled sprint-9.14 → sprint-9.16, client crates 0.2.0 → 0.3.0 (2026-10-06, #212). Image b2be29c4c0d2-sprint-9.16 (sha256:7a44b43a…, Ferrum's own manifest digest, mirrored digest-preserving), rendered from Ferrum chart 0.4.0 (63b9994c). 9.15–9.16 are Ferrum's Kafka exactly-once rebuild, and EPIC uses none of it (kafkaTxn.enabled stays false). The roll keeps the server inside the pairing Ferrum supports for the 0.3.0 client ("server ≥ sprint-9.16"). The client change is additive for EPIC, and no EPIC source changed. Before production moved, the six Ferrum-backed suites passed strict against the 9.16 binary (22/22), and the meter benches were A/B'd against 9.14 in one session because 9.16 rewrote CONSUMER.SUB under the meter consumer: no regression, frozen as baseline tag ferrum-9.16. A server-side dry run caught the chart's version bump being rejected, because volumeClaimTemplates carried versioned labels and are immutable. A plain helm upgrade would also have left the new [kafka_txn] ConfigMap beside a 9.14 pod that rejects it. Ferrum fixed it in chart 0.4.0 (ferrum#119, after the missing image and chart bump in ferrum#117). Done as: clean shutdown snapshot, volume tarred to /data/backups/ferrum-9.16-roll/, one-time delete statefulset --cascade=orphan, helm upgrade --rollback-on-failure (release revision 7). No data step: the AOF and snapshot formats are unchanged. Ferrum was down about 2 minutes, 13:06–13:08 UTC. Both edge pods, control and the aggregator reconnected in-process (ADR-126 reseed) with no pod restarts. Rollback is helm rollback to revision 6, chart and image together, because 9.14 rejects [kafka_txn]. Record: deploy/k8s/epic-state/ferrum-values.yaml.

  • Dashboard copy and tracker fixes from the owners' site-text round 1 (2026-10-14, #278). The first round from the site editor: 14 texts reworded as written (the marketing slot pool, Usage, Models, Q&A); the free give-away bar shows what is left in green from the left and what is gone in red at the right-hand end, so it empties from the right as slots are given, with the subtitle "Anyone can win these slots!"; no tracker draws a marker line at its end; the Fair-share "?" pulses until opened, the standard for every help mark; the Project stages bar is a wave with the stages on its hilltops and in its dips. The round's robot mascots and stages drag editor follow in their own PRs; the Models box question is #276.

  • Production Ferrum rolled sprint-9.11 → sprint-9.14 (2026-10-04). Ferrum main 5f305866, image sha256:79cc8c7f…, via helm upgrade with the chart from the same commit — not an image-only roll: 9.14 rejects the old chart's config (unknown field audit) and refuses a pre-ADR-134 AOF, so the old AOF files were removed after a clean shutdown, with the volume tarred to /data/backups/ferrum-9.14-roll/ first. Brings the fix for #156 (ferrum#2) plus the production fixes that mattered more: counters no longer double-applied on restart, EX TTLs no longer early, arena memory returned. Encryption at rest is live for the first time — the 9.11 chart rendered a key the server never read — so a Ferrum snapshot restore now needs master.key (disaster-recovery.md Part D). Both planes and the aggregator reseeded in-process, no pod restarts. Record: deploy/k8s/epic-state/ferrum-values.yaml.

  • FerraCert pinned to 1.0.0, its first tagged release (2026-10-05, #137). The pin moves from the August 0.1.0 build (c8f306f) to sha256:b937625a… (tag v1.0.0, mirrored digest-preserving from the lab registry). It brings ferracert deliver and the serve redelivery sweep (FerraCert ADR-029), so a certificate that issued against a stale delivery-key pin is recovered without spending a Let's Encrypt duplicate-certificate slot; stack-relocation.md now says so. Config is additive only and migration 009 applies at store open; FerraCert's kustomize base is unchanged since the prior pin.

  • FerraVault crates 1.12.1 → 1.14.0, and the bearer-key rotation chain is now tested over an mTLS-only Watch (2026-10-05, #182). v1.14.0 changes no proto and no config; the client side needed no code change. FerraVault's published manifests require their sibling crates only at ^1.12.1, so all sixteen in the lockfile are moved together — updating just -client/-proto/-test-utils resolves a 1.12.1 ferravault-server against a 1.14.0 ferravault-proto, which does not compile. The new epic-ferron-client suite integration_bearer_rotation_mtls_watch drives the edge's chain end to end against v1.14.0's mTLS test listener: a certificate-only VaultClient bound to the deployed epic-edge.toml policy, spawn_watcher, spawn_rotation_watcher, two successive rotations observed in the BearerAuth holder with no restart; a negative control proves a policy without watch does not rotate. The production fix is the FerraVault server roll: ferravault-values.yaml and the backup CronJob are re-pinned to 1.14.0 (sha256:b2736fb1…) and 1.14.0-debug (sha256:8047efc5…), mirrored digest-preserving from the lab registry; a render against the live release differs only in those two images.

  • Ferrum client crates 0.1.0 → 0.2.0 (2026-10-05, #178). The registry's 0.1.0 (afdb8d9d, 2026-08-18) predated every Sprint 9.14 client fix: its smart router chose a shard by slot % shard_count while the server places keys by xxh3 % shard_count, so about half of EPIC's commands took an intra-pod forward hop under threads: 2, and three reply-desync fixes were unpublished. Ferrum published the 0.2.0 closure from sprint-9.15-final (ferrum#113). No EPIC code change; the six Ferrum-backed suites pass strict (21/21) against a ≥ 9.14 server. Client 0.2.x needs server ≥ sprint-9.14, which production runs.

  • Signup asks for credentials and consent only; the investment step asks for the organisation's profile (2026-09-14). ADR-066. The signup form loses its country, organisation-name and separate age fields; the Acceptable Use Policy box now reads "I am 18 or older and accept the EPIC Acceptable Use Policy", so one tick is both. Signup's sanctions check now screens the country Cloudflare resolved for the connection (CF-IPCountry, and CF-Region-Code so the list's occupied-region entries can match at all), believed only from a [client_ip] trusted proxy; a connection with no trusted country is refused under the new [boot.sanctions] unknown_country = "block" (declared in production) and accepted-but-recorded under "allow" (preview slots, via EPIC_SANCTIONS_UNKNOWN_COUNTRY). What was screened goes into the user.registered audit row. /app/billing/checkout gains a required country, an optional organisation name and a required 18+ box, enforced in create_checkout for the form and POST /api/billing/checkout alike: the declared country is validated and OFAC-screened, and the organisation updated and billing.checkout.profile_confirmed audited, before any payment provider is called. orgs.name is optional (migration 0052, orgs_name_not_blank): Org::label() falls back to the billing email where an organisation must be identified (admin console, audit summaries, billing email, the slot certificate), profile pages show "(not set)", and the JSON org APIs return null. One normaliser (normalise_org_display_name) now governs names at registration, the investment step and rename. Wire: RegisterRequest.country_code removed (ignored if sent); CheckoutRequest requires country_code + age_affirmed. Production signup now depends on the zone's IP Geolocation setting — cloudflare-zone.md §5b.

Security

  • The production edge no longer links the test-fixture crate (2026-10-25, #414). epic-test-utils, and through it a whole in-process FerraVault server, is now optional behind a non-default dev-stack feature. Only that feature compiles in the in-memory adapter stack that dev.allow_in_memory_stack = true boots. Without it, the edge refuses that config with a message saying how to rebuild. The production edge's dependency graph drops from 526 to 438 crates. Advisories in test fixtures no longer read as production advisories: RUSTSEC-2026-0387 reached the edge only this way.

  • tokio 1.52.3 → 1.54.1 for RUSTSEC-2026-0386 (2026-10-23, #408). The advisory, published 2026-10-23, found that tokio::io::unix::AsyncFd's safe constructors accept any AsRawFd. Safe code can therefore register a closed or reused file descriptor, which violates I/O safety and can lead to a use-after-free in tokio's driver. EPIC's code never uses AsyncFd, but the advisory failed cargo deny on every PR Gate. Lockfile only.

  • ctutils 0.4.2 → 0.4.5 for RUSTSEC-2026-0385 (2026-10-23, #406). The advisory, published 2026-10-23, found that ctutils::Choice's mask generation and its u128 comparisons could compile to branches on secret values, so they were not reliably constant-time. ctutils reaches the graph through digest 0.11 → hmac 0.13, including FerraVault's client crates. The patch bump is lockfile-only, and it turned every PR Gate's Stage 2 green again.

  • Blocked email addresses and the restricted abuse record (2026-10-23, #365, #378 comment 6374; owners' decisions of the day before). An address EPIC blocks for a policy or terms breach stays blocked until an operator lifts it:

    • What it blocks: sign-up (register, /signup, the invitation sign-up), being invited (403 email_unavailable) and an email change, each checked where the request comes in. Sign-up answers a blocked address exactly as a taken one (409 conflict, one fixed message, which also stops the duplicate case passing Postgres's own message through), and an email change reads as one to a taken address, so neither reveals a block. Phase 10's federated sign-up calls the same EmailBlockService::is_blocked.
    • Storage (migration 0093, blocked_emails): a BLAKE3 keyed hash of the normalised address under a FerraVault key (epic/email-block-key-v1; ops step in the runbook), and the readable address for one year; an hourly sweep then clears it unless held, and the hash keeps blocking. Operator-only (FORCE-RLS deny-all).
    • The restricted abuse record (abuse_records): kept when EPIC blocks or deletes an account for abuse: ids, email, keys, the abuse reports and kill-switch decisions about it and their audit row ids. One year from the decision unless an operator sets a legal hold; the account's erasure leaves it alone; operators only, every read audited. GDPR Art. 17(3)(b)/(e).
    • Console: Abuse triage → Blocked emails (block, find, lift, legal hold) and Abuse records (list, record, legal hold); "Block this email" and "keep an abuse record" on the member page's suspend and delete. Every change audited; EpicAbuseRetentionFailing / EpicAbuseRetentionStale alerts.
    • A member's own deletion blocks nothing: the address signs up again once the account is erased.

    DESIGN §10.6, §13.4 and §14.1 updated.

  • An erasure no longer fails on an audit row, and "Keep my account" always wins (2026-10-23, #403). Two ways an account erasure could go wrong:

    • An audit row naming the account or the person stopped the erasure, every hour, for good. The append-only trigger refused the org_id → NULL the account's deletion sets, and the erasure's actor_user_id → NULL. A member who reached the investment step while the pool was sold out, then deleted their account, was never erased; an untouched own account carrying such a row could not be replaced by an invitation. Migration 0091 now lets audit_log_immutable allow exactly an UPDATE that sets those columns to NULL, every other column unchanged. Neither is in the chain hash, so every row_hash, the chain and every signature still verify. Everything else is still refused.
    • "Keep my account" between the sweep's listing and the erasure could still end in an erasure. The sweep now erases through Store::erase_user_if_due, which re-checks under the person's row lock that disabled_at is set and past the cutoff, and leaves the account as it is when it is not (counted as kept in the pass's report).

    Tests against Postgres 18, as the application role: an erasure with audit rows naming the account and the person, the chain and signatures verifying after; an invitation replacing an own account named in the audit log; the trigger refusing every other change; a cancel between the listing and the erasure. DESIGN §10.5 and §14.1 and the erasure runbook updated.

  • An admin key works only for an EPIC operator (2026-10-23, #402, #384). Any slot user could mint an admin-scoped key and use every /api/admin/* route, because the routes checked only the scope. With #374's suspend-user that meant a lasting lockout of every operator.

    • POST /api/keys refuses the admin scope (403 forbidden) to anyone but an operator (users.is_epic_operator, ADR-049), and the web form offers it to operators only.
    • Every /api/admin/* request reads the key's issuer from the database and refuses it (403) unless they are an operator whose account may sign in. Keys minted before, and an operator who lost the flag or was suspended, are refused on the next request.
    • The routes are listed once and mounted behind one route layer that runs the check before any extractor. A test sweeps every listed route by every method through the plane's router; a source scan fails on an /api/admin path registered outside the list.
    • epic-migrate seed-org flags the user it gives the bootstrap admin key as an operator, as ADR-049 already said it did. A deployment seeded before runs grant-operator for that user once.
    • The diagnostic scope gates no route today, so it is unchanged.
  • Account erasure runs, covers everything, and knows who asked (2026-10-22, #365; owners' decisions the same day). Members were told their account was deleted after 30 days, and it never was:

    • The sweep runs: ErasureSweep is spawned hourly in epic-control, audits each erasure (user.erased) and is counted (epic_control_erasure_sweep_*), with EpicErasureSweepFailing / EpicErasureSweepStale promtool-tested (runbook).
    • It no longer skips sole Owners, and nothing blocks it: one transaction per account (epic_store::erasure) handles every foreign key to users and orgs, listed with its disposal and pinned by a test against pg_constraint; erased as the application role, not the superuser. An Owner's account goes with them; its other members keep their sign-in, each getting a new account of their own with no slots if it was their only one (owners' decision on #378, same day).
    • One exception, for the owners to confirm: purchases and slot certificates stay, with the name, email and id replaced; the account row stays closed and nameless as their anchor.
    • Who asked matters (migration 0091): a member who deleted their own account can sign in for 30 days, only to "Keep my account" (keys stay disabled); an account EPIC deleted (new console action and POST /api/admin/delete-user) cannot sign in. /app/me/delete no longer promises a restore on written request.
    • Losing one's account never disables or erases the member: removal from one's only account now gives the member a new account of their own with no slots instead of disabling them (which the now-running sweep would have turned into an erasure). Migration 0091 re-enables the logins the old removal disabled and gives every login without an account one of its own; login does the same as a net. 403 no_group is gone.
    • An untouched account may accept an invitation (owners' decision of 2026-10-22, #378 comment 6374): a login whose own account is exactly as it was made (no slots, purchases, keys, usage, other members, invitations, name or settings) joins the inviting account, and the empty one is deleted in the same transaction (InviteStore::accept_invite). That covers a new sign-up as much as a member who lost their account. An account that holds anything still refuses the login with 409 account_exists (ADR-069 §1), and a refused or mismatched acceptance no longer uses up the invitation.
    • EPIC's deletion sends its own email (same comment): account_closed_by_epic, with the day it was closed, the day it is erased and where to appeal (the [domain] abuse and support mailboxes), body in deploy/postmark/. A member's own deletion still sends gdpr_erase_scheduled. It has to be pasted into the Postmark console (#294).

    DESIGN §9.2, §10.1 and §14.1 updated.

  • Sign-in state that should end now ends (2026-10-22, #366; owners' decision the same day). Three ways it didn't:

    • Sessions end on the server: 30 days after sign-in (the cookie's Max-Age, now derived from the same value) or after 7 days unused, whichever comes first ([control] session_max_age_secs / session_idle_timeout_secs). Before, a copied cookie worked until someone revoked it. An hourly sweep deletes revoked and ended sessions, their refresh tokens and expired refresh tokens a day later (auth_session_purge, migration 0090; epic_control_session_sweep_* metrics).
    • A disabled account can't sign in: the login refuses disabled_at with 403 account_disabled, and a session it still holds is revoked on its next request.
    • The user kill switch is a durable suspension: firing it records users.kill_switched_at + kill_switched_reason (migration 0090) before revoking keys and sessions; the login refuses it with 403 sign_in_suspended ("sign-in", not "account": ADR-069 uses suspended account for an account-level hold that still lets the member sign in; owners' approval on #381), the session lookup signs it out, and is_fired reads the column once the 24 h tombstone is gone. An operator lifts it from the member's console page (/app/admin/orgs/{org}/members/{user}) or POST /api/admin/unsuspend-user; both surfaces can now fire it too (suspend-user), which no route could before.

    Both refusals come only after the password and the second factor verify, so a wrong password still answers like an unknown address. DESIGN §9.2, §10.1, §10.3 and §13.4 updated.

  • Every signed-in form and script post carries a token bound to the session (2026-10-21, #356 item 1; ADR-081). A state-changing request that carries the epic_session cookie must carry BLAKE3-derive_key(context, cookie), as a hidden _form_token field or the X-Epic-Form-Token header, or epic-control refuses it with 403 form_expired (logged, counted in epic_control_form_token_refusals_total). A tab rendered for one account can no longer post as the account that signed in after it, with or without JavaScript, and a same-site subdomain can no longer forge a request on the cookie. One layer in router_with_middleware, before routing, stateless, compared in constant time. Exempt by exact path: the pre-login forms and their JSON equivalents (/api/auth/{register,login,password-reset/*,verify-email,verify-email/resend}, /api/abuse/report), which render no token; and /logout, so "Sign out" works from any tab, however old (owners' decision on the #372 review; SameSite=Lax keeps a cross-site sign-out cookie-less), which is why no sign-out form carries a token, #354's on /login and /signup included. The avatar upload's body is never read by the layer, which runs before authentication: the handler checks the token in it after authenticating the member and taking an upload slot and the account's claim, with the layer's own check, so the #197 bound on upload memory holds (proved through the production tower and over HTTP/3). The refusal page says what happened in plain words: the page was opened under an earlier sign-in (the browser signed in again, or the password was changed, since), nothing was saved, reload it; and a reader who sent nothing may have had another site try to. Every signed-in <form method="post"> and the three scripts that post (layout.js, menu_editor.js, the playground's key mint) carry it. Router sweeps cover 64 dashboard routes, 84 console routes, 24 other cookie routes, a path no route serves and every exempt path; a source gate, quote- and expression-aware, covers every POST form and refuses a token rendered only under a condition. The gated playground_browser_e2e now runs through the production tower, with its stale #pg-output selector fixed.

  • The edge role's check sees memberships, and the grants apply in one transaction (2026-10-17, #340 items 1, 2 and 4; follow-ups to #330). Because epic-edge-ro is NOINHERIT, the privilege checks behind edge-grants.allow ignored role memberships: a GRANT epic_admin TO "epic-edge-ro", after which SET LOCAL ROLE epic_admin reads api_keys, or a pg_read_all_data membership, still matched the list.

    • The check now also reads the role's attributes, every role it is a member of (directly or not), and everything pg_shdepend records against it beyond the grants already listed: objects it owns, large objects included, policies naming it, default privileges, and grants in other schemas or databases. A new test makes twelve such changes and checks each one is reported.
    • In production CNPG sets the attributes and memberships from postgres-cluster.yaml, which that test never reads. A new Stage 1 gate (ci/postgres-roles/) fails if the role's entry there declares inRoles, any attribute but login, or leaves out inherit: false (CNPG defaults it to true).
    • The migrate Job ran postgres-roles.sql statement by statement, so a failure after the edge section's REVOKE ALL left the edge with no grants: an edge-wide auth outage. It now runs psql with --single-transaction, and the file opens with SET LOCAL lock_timeout = '10s'. preview.py wraps its cut of the file in its own transaction, since a COMMIT in the file would be cut off.
    • The gate also requires --single-transaction and ON_ERROR_STOP=1 on the Job, and keeps transaction control and psql meta-commands out of the file. The file's header now says so; it had claimed the DO blocks only grant (item 4).
    • Rolling back means re-applying the previous file the same way: docs/operations/migration-runner.md §3.
    • #350 review. The check now reports a privilege the role may pass on (… WITH GRANT OPTION), MAINTAIN (PG17+; it lets a role LOCK TABLE … ACCESS EXCLUSIVE, which would stall auth on both planes) and every ALTER ROLE … SET for it (setting lines). A dependency row is now hidden only when every privilege in it is one the lines report, judged from aclexplode, so a privilege type the query does not know surfaces as a depends line rather than vanishing. Five new drift cases cover them. The query moved into deploy/k8s/epic-state/edge-effective-privileges.sql, which the test loads with include_str! and the runbook (§2) runs on the cluster with psql's \bind, so both check the same text. The gate also checks the role as each substrate profile renders it (kubectl kustomize deploy/k8s/profiles/*), and refuses a role, depends or setting line in edge-grants.allow, which would otherwise let one pass beside the grant that causes it. Its SQL scanner moved into ci/postgres-roles/sql_scan.py.
  • A new account confirms its email with a code before it can log in, and a browser holds one account at a time (PR #354; Melvyn's request of 2026-10-20; the multi-tab report in #356). Sign-up already sent a 24-hour confirmation link, but nothing asked for it: an account logged in whether or not its owner held the address.

    • Confirmation by code (owners' decision 2026-10-22, #354 review). The mail carries a six-digit code, not a link: a link confirmed whoever opened it, so someone who registered another person's address with their own password could have that person confirm the account for them, and ADR-068 §6.3 auto-links federated sign-ins on a verified email. The code (CSPRNG, stored only as a keyed BLAKE3 hash bound to the account, compared in constant time) works for 24 hours, and only together with a pending-confirmation token that EPIC gives the browser that just set the password (sign-up) or proved it (a login, with any second factor, refused only for the unconfirmed email): 32 random bytes, stored hashed, one per account, one hour, in a __Host- HttpOnly SameSite=Lax cookie on the site and as pending_confirmation_token in the JSON API. Each try is counted before it is compared, atomically, and the fifth wrong one voids the code; "Send a new code" (/verify-email/resend, which now needs the token) replaces it. An account is mailed at most three codes an hour and ten a day, sign-up's included (owners' decision 2026-10-23); past either, the answer is 429 with when the next can be sent. With five tries a code that holds someone who has the password but not the mailbox to 50 guesses a day, about 0.15 % over a month, while the address's owner receives every code. Code entry also has its own per-IP band. A password change ends the token. Migration 0092 adds email_verification_codes and email_verification_pending and drops 0024's link table. Sign-up and an unconfirmed login go straight to /verify-email, the code page; the ?token= link, the address-taking resend form and both check-email pages are gone. The Postmark verify_email template in deploy/postmark/ renders {{code}} and {{valid_hours}} and must replace the one in Postmark when this deploys.
    • Confirmation before login. POST /api/auth/login refuses an unconfirmed account with 403 email_not_verified, only after the password and any second factor, so a wrong password still gets the same unauthorized as an unknown address. Only new logins are gated: an existing session keeps working. A completed password reset does not count as confirmation. There is no grandfathering and no migration of the flag: an existing account that never confirmed logs in, gets the code page and confirms like any other, so email_verified keeps meaning that the owner of the address proved it from the browser that holds the password. The owners confirm their own accounts the same way; preview accounts are already confirmed by epic-migrate ensure-accounts.
    • One account per browser. A browser holds one session cookie for all its tabs, so a second login replaced it under the tabs still showing the first account, and their forms acted for the new one (Melvyn saw the interest totals mix across tabs, #356). login and register now read the request's own session first and refuse 409 already_signed_in before anything about the account asked for is looked up; a session lookup that fails is 503, never "signed out". /login and /signup show which account the browser is signed in to, with a sign-out button, and logging in again as that same account goes on to the dashboard with no second session. Every dashboard and console page carries its account (data-account, only when there is one) and a small guard script (account_guard.js, CSP-hashed) that records it in localStorage: a tab whose account another tab signs out of or replaces is blocked behind a modal that offers only "Reload". The login and sign-up forms, shown only when the server found no session, remove that record (account_signed_out.js, CSP-hashed), so a session that ended without a sign-out (expiry, a deleted account) leaves none on a shared machine; a failed session lookup still shows the form but leaves the record, so an outage never blocks the other tabs.
    • No cross-site pre-login posts (#354 review). Login, sign-up, the code and the resend (web forms and JSON API alike) refuse a request whose Sec-Fetch-Site is not same-origin or none, or whose Origin is not the site's own, with 403 cross_site_request, before anything is looked up; a request with neither header (the CLI) passes. Without it a page elsewhere could post the attacker's own credentials and sign a visitor's browser in to the attacker's account (login CSRF): that post carries no session cookie, so the one-account rule saw a signed-out browser. "No session is issued" now holds cross-site too. The site's origin is [domain]'s app origin, or [control] public_origin where a deployment overrides it (a preview, a local run).
    • Tests: the code service (issue, confirm, five wrong tries void it, the code's day and the token's hour, a resend replaces the code with fresh tries, no token, one account's token with another's code, a password change ends the token); the handlers and pages, both ways in, through the whole router with Postmark mocked so the code entered is the one mailed; anti-enumeration unchanged; every cross-site case on every pre-login POST, with nothing created, issued or mailed; a real session-row and account-row lookup failure answering 503 on login and sign-up with a cookie present; the daily cap reached and its window rolling over, with sign-up's code counted and logins counting nothing; Postgres under the NOSUPERUSER NOBYPASSRLS role (the lifecycle, 24 concurrent tries getting exactly five, the sweep, an email change, the edge role seeing neither table); the h2 end-to-end suite confirms with the mailed code; and the earlier one-account tests, including the two-tab Firefox run in dashboard_browser_e2e. The visual set swaps signup_check_email for verify_email (the code page's "log in to continue" state), which needs a baseline recorded.
  • rustls-webpki 0.103.16, webpki-roots and webpki-root-certs 1.0.10: no trust in the distrusted emSign intermediates (2026-10-21; RUSTSEC-2026-0381/0382/0383, CVE-2026-111943). The private keys of several emSign intermediates are now publicly known. rustls-webpki 0.103.15 could accept a certificate path through them (0382), and webpki-roots 1.0.7 / webpki-root-certs 1.0.9 bundled emSign ECC Root CA - G3, which reaches them (0381, 0383). Cargo.lock moves those three crates and nothing else. All three are transitive, through rustls, hyper-rustls, reqwest and sqlx, so this is lockfile-only like cryptoki in #263, and cargo deny / cargo audit keep any lockfile from resolving back. Found when cargo deny failed ci.sh Stage 2 on #350's gate; it fails on main and every open PR from the advisories' publication.

  • The checkout hands merchants absolute return URLs only (2026-10-16, #320 item 12; #315 review). A return path such as the form's /app/me#billing is now resolved against the public origin, so a merchant only ever receives an absolute URL on this site and can never resolve a relative one against its own domain. With no public origin configured the checkout stops with a 503 before any slot is held. An explicit [control] public_origin or edge_public_origin loses a trailing / when it is resolved: kept, it refused this site's own return URLs and made email links such as {origin}/verify-email start with //. New cases pin an uppercase host, a trailing dot, an explicit default port and an IPv6 origin with a port as refused, as they already were.

  • The edge's database role holds an explicit allow-list, checked in CI (2026-10-16, #320 item 8; #242 item 7). epic-edge-ro had SELECT on every table, default privileges on every future one, every sequence and two INSERTs, so a stolen edge credential could read abuse_reports, waitlist_signups, playground_access_codes and every other member's data. Through grants that migrations make to PUBLIC, it could also run every SECURITY DEFINER wrapper, including auth_session_insert: a session for any member. postgres-roles.sql now grants the edge only what its code runs. That is SELECT on the columns of the five tables it reads and the API-key lookup function; it writes nothing. The file also gives each other callable definer function to the control role and epic_admin and revokes it from PUBLIC, and revokes PUBLIC's TEMPORARY. edge-grants.allow pins the role's complete effective privileges, and two tests in integration_postgres_roles fail if they drift or if any query the edge makes no longer works under the role.

  • A recovery code is spent only with the change it authorises (2026-10-16, #320 item 5; issue #205). The password change, the email change and the two-factor disable each removed the code first and made their change after, so a failure in between (hashing the new password, a database error) spent the code for nothing. AuthFlowStore::spend_recovery_code now removes the code with a compare-and-set and makes the change in the same transaction: a change that fails rolls the spend back, and of two requests presenting one code only one makes its change (8 concurrent spends against Postgres: exactly one wins). An email change whose mail cannot be sent is withdrawn together with giving its code back, but only if the withdrawal removed that very request, and only while MFA is still on with the same secret, so a re-enrolment never revives an old code; this retires the residual ADR-075 accepted. The withdrawal (AuthFlowStore::withdraw_email_change) is keyed on the request's confirm-token hash, with or without a recovery code: a send can report failure after the mail was accepted, so the link may already have been used, or another tab may have replaced the request, and then nothing is deleted, nothing is given back, and the member is told to check the Account tab rather than that nothing changed (review of #322). Each withdrawal is audited as user.email_change_withdrawn with recovery_code_restored, and user.email_change_requested is written after the spend, so a request that lost its code to another never reads as requested (#337). RecoveryCodeUse::RequestEmailChange carries no user of its own, so a change can only be recorded for the user spending the code. consume_recovery_code is gone, so no caller can spend a code apart from its change.

  • Passwords: 15 to 256 characters, and never a known breached or common one (2026-10-16, #320 item 1). NIST SP 800-63B-4 requires at least 15 characters where a password is the only factor, and a check against known compromised and common passwords. epic_core::password_policy now holds both: the length in Unicode code points after NFC, an embedded blocklist of 255,874 entries of 15 or more characters (SecLists' xato, Pwdb and NCSC lists, folded and sorted, searched in place with no index), and NIST's context check against the account's email and the site's domain. Register, reset and the Account tab's change call it, and each refusal carries its own stable code and a reason. Sign-in never does, so existing passwords keep working, and the login form drops its minlength. A reset now reads its token without spending it (AuthFlowStore::find_password_reset_token), so a refused password keeps the link working. The sign-up form's hard-coded 12–128 now reads the policy, like every other form.

  • The operator console's routes are gated and tested from one list (2026-10-16, #320 item 4; issue #147). Every /app/admin/* handler called require_operator_session itself, after its form and query extractors, so a signed-out POST reached a form extractor before any check. The tests that proved each route gated read hand-kept lists, which had drifted twice: the waitlist routes, then the questions email route. Each console section now lists its routes as AdminRoutes, and admin::router mounts exactly all_admin_routes() behind one route layer that runs the gate before any handler or extractor. The authorisation tests sweep the same list by every method, through the plane's whole router, signed out (303 to sign in) and as a non-operator (403), so a new route is gated and tested by construction, and a handler another router registers on a listed path fails the sweep. The routes gate (ci/routes/verify.py) holds the console to that shape (#342 item 1). all_admin_routes is a declared collector: its calls mount the sections only once it is itself reached from the root router, so a section left out of it fails as unmounted, and every other listing still mounts nothing. A path under /app/admin registered any other way than as an AdminRoute in a section under routes/web/admin/ fails, and so do a .nest(…) whose prefix covers the console, a section's routes mounted other than through the collector, and anything merged, nested or routed after the gate's .route_layer(…), which would be served ungated.

  • The checkout's return URLs must be on this site (2026-10-16, issue #149). POST /api/billing/checkout passed success_url and cancel_url to the merchant unchecked, so an owner or admin could make the merchant redirect to any address after payment. Each must now be a path on this site or an absolute URL on its public origin; anything else is a 400 before any slot is held. The path rule is the currency form's (is_safe_return_path: no //, /\, control characters or over-long values). Low severity: the caller must already own or administer the organisation, and no merchant is live yet (tracker L1).

  • A plane that cannot reach Postgres at boot no longer prints the database password (2026-10-14, issue #275, found investigating #91). Both planes put the full DSN into their connect error's context, so the epic-control crash of 2026-10-12 wrote the live password into its pod log. One epic_config::sections::redact_dsn now masks the userinfo password and a libpq ?password= parameter on both planes, and withholds an unparseable DSN whole. The exposed password still needs an operator rotation — see #275. A test on each plane drives the real connect path with a password in the DSN and asserts the printed error chain does not contain it.

  • rustls 0.23.46: four advisories published 2026-10-14 (2026-10-15, #300; RUSTSEC-2026-0346/0347/0348/0349). Debug on client session values printed TLS secrets (0346); buffered application data went to the outer server after an ECH rejection (0347); Brotli certificate decompression could allocate up to 1 GiB before authentication (0348); an unauthenticated ServerHello could downgrade later connections to classical key exchange (0349). Cargo.lock moves rustls 0.23.45 → 0.23.46 and nothing else; the workspace entry rises from 0.23 to 0.23.46. Unlike quinn-proto before #265, this floor binds: seven member crates inherit rustls = { workspace = true }, so no lockfile can resolve below it. cargo audit had failed ci.sh Stage 2 on main and every open PR since the advisories were published.

  • The quinn-proto 0.11.19 floor is now enforced (2026-10-13, issue #265). The workspace entry #258 added bound nothing: no member crate inherited it, and quinn-proto arrived only transitively — through quinn 0.11.9 (which accepts 0.11.12+) and ferrum-client/ferrum-protocol 0.3.0 (^0.11) — so only the lockfile and cargo deny held the line against RUSTSEC-2026-0338…0342. epic-edge and epic-control, which run the HTTP/3 listeners, now declare quinn-proto = { workspace = true } for the floor alone, and a cargo update cannot resolve below it. Also from Maran's review: the two copies of the kube read timeout cross-reference each other, and the unsafe-libyaml note says what RUSTSEC-2024-0449 records.

  • A TOTP code is accepted once (2026-10-13, issue #239; RFC 6238 §5.2). A code valid within its ±1-step window could be used again for up to 90 seconds — by anyone who saw it over a shoulder, relayed it from a phishing page or read it from a log — at sign-in and at every step-up a code guards (change password, change email, disable MFA, owner transfer). TotpService::verify now says which step a code matched (still constant-time over the three candidates), and the new TotpService::verify_once spends that step through AuthFlowStore::accept_totp_step: one compare-and-set on the new users.totp_last_step (migration 0080) that records it only if it is later than the last step accepted, so of two concurrent submissions of one code exactly one passes. A replay is AuthError::TotpReplayed, answered on the wire exactly as a wrong code. A new secret or disabling MFA clears the history. All five verification sites (login, enrolment confirm, disable, the shared web step-up, owner transfer) use it. Tests: verify_once unit tests (replay, steps only forward, a wrong code spends nothing, new secret, backend failure), a Postgres suite under the app role (spent once, forward only, resets, 20 rounds of a concurrent double submit), the edge's hidden user columns, and login and step-up replay tests.

  • quinn-proto 0.11.15 → 0.11.19: five remote denial-of-service flaws in both planes' HTTP/3 listeners (2026-10-13; RUSTSEC-2026-0338/0339/0340/0341/0342). A peer could grow the receiver's memory without bound (empty DATAGRAM frames past the receive-buffer limit; gapped STREAM frames past the reassembly cap; repeated NEW_CONNECTION_ID frames past the retirement limit) or panic it (zero-length STREAM frames in unordered reads; a min/max ACK delay inversion). Found by the PR Gate's newer advisory database; the workspace floor is now 0.11.19 so no lockfile resolves below it. quinn stays at 0.11.9 — the FerraVault test server pins it exactly — and takes the new quinn-proto unchanged.

  • kube 3.1 → 4.2 and k8s-openapi 0.27 → 0.28: no YAML parser written in unsafe C-translated Rust (2026-10-13). cargo deny began failing Stage 2 on every branch, main included, with RUSTSEC-2024-0449: unsafe-libyaml (a c2rust translation of libyaml, archived, no fixes) came in through kube-client 3 → serde_yaml (deprecated), which reads a kubeconfig. kube 4 reads it with serde-saphyr, which forbids unsafe, and the two crates leave the lockfile. k8s-openapi takes v1_36, the lab API server's own version, in place of v1_35 within the skew. One behaviour change in kube 4 is undone on purpose: it dropped the client's 295 s read timeout (for exec and port-forward sessions, which EPIC never opens), so epic-cluster and epic-secret-sync set it themselves — epic_cluster::kube_client and epic_secret_sync::sink::with_read_timeout — and a stalled API server still fails a call instead of hanging the reconciler; a test in each pins it. No code beyond that changed: both crates compiled unchanged against kube 4.

  • The sign-in email changes only through a verified flow; PATCH /api/me no longer changes it (2026-10-08, ADR-075). PATCH /api/me set a new email at once on nothing but a session, so a stolen session cookie could move an account to its thief's address and then reset the password from it. It now refuses an email change with email_change_requires_confirmation before writing any field. The change takes the current password and the MFA step-up (shared with Change Password, including one rate allowance for both forms), a single-use 24-hour link to the new address (only its hash stored, migration 0064), a confirm button on the page the link opens (a mail scanner opening the link changes nothing), and a notice to the old address; confirming signs the member out everywhere. An address another account holds gets the same answer and no mail. The notice to the old address carries a cancel link that needs no password, the Account tab shows a pending change with a Cancel button, and any password or MFA change cancels a pending change — otherwise "change your password" would not have stopped an attacker who already held the link (found by an independent review before merge). Confirming also kills every reset and verification link mailed for the old address, and a disabled account cannot confirm. Two Postmark templates (deploy/postmark/email_change_*) must be added to the console before the form is used in production.

  • The control plane's HTTP/3 listener holds a bounded amount of unread request data (2026-10-07, PR #225). quinn's defaults allowed 1.25 MB per stream and no limit per connection — about 125 MB of unread data on one connection with 100 streams open. The QUIC receive windows are now 1 MiB per stream, hyper's HTTP/2 default, and 4 MiB per connection — larger than HTTP/2's 1 MiB because QUIC flow-controls a request's headers, so with the two equal a single upload the app is not reading would stop every other request on the connection from starting. Ordinary requests never meet either window, and a large upload is paced by the app's own reading as over HTTP/2. Three tests over real QUIC pin the stream window, the connection window and the room left for the next request; each window moved in either direction, the setters swapped, or quinn's defaults restored fails at least one.

  • The control plane's HTTP/3 bridge streams the request body into the app instead of collecting it first (#198, 2026-10-07). It used to read the whole body into memory before dispatching to axum — so over HTTP/3, on any path and before authentication, a request body was bounded by nothing but the sender's patience. The body now streams in as QUIC delivers it, capped at the path's limit (64 KiB, 15 MiB for the avatar upload), exactly as on HTTP/2: authentication, the rate bands and the avatar upload slots all run before any of it is held, and the peer is told to stop sending once the answer is out. Tested against the production serve_h3 over real QUIC, including an unauthenticated upload and one with no free slot, each answered while the body is still arriving.

  • cryptoki patched for RUSTSEC-2026-0286 (2026-09-17). An out-of-bounds read: cryptoki 0.12.0 treated the CKA_ALLOWED_MECHANISMS ulValueLen byte count as an element count, so a valid nonempty attribute read back through the safe Session::get_attributes could build an out-of-bounds slice. Caught by the SP.3 gate sweep, which was failing Stage 2 on every branch including main — the reason PRs #129 and #134 were red with no defect of their own. Lockfile only, 0.12.0 → 0.12.1, two lines, no other crate moved. It reaches EPIC transitively and only through test fixtures (cryptoki → ferravault-server → ferravault-test-utils → epic-test-utils → epic-edge), and nothing in EPIC opens a PKCS#11 session, so the vulnerable path is linked but never called. The last edge of that chain is a [dependencies] edge, not [dev-dependencies] — so the fixture crate compiles into the production edge binary. That is pre-existing (Sprint 2.3) and is recorded with a concrete remedy as Q77 of the architecture review rather than reshaped inside a sign-off sprint.

  • The website's login, signup and password-reset forms are rate limited (2026-09-14). The DESIGN.md §9.7 bands were matched on /api/auth/* only, but /login, /signup, /forgot-password and /reset-password post to server-side handlers that call those API handlers in process — so the middleware saw only the form path, matched nothing, and the forms the site actually offers members had no anti-abuse band at all. Password guessing through /login was bounded by the Argon2id cost and Cloudflare's one-a-second volumetric rule, i.e. ~3,600 attempts an hour per address against a band of 40. Each form now shares the band of the endpoint it calls, so alternating surfaces does not double the allowance. Two related defects closed with it: the check ran for every method, so opening the public /abuse form five times in an hour locked a reader out of filing a report and a CORS preflight cost a login attempt (only submissions count now — GET/HEAD/OPTIONS pass through, matching Cloudflare's own rule); and a refused website form, or one refused because the limiter's backend was down (still fail-closed), gets a page with the wait and a fixed link back — or, for the landing page's join form, a new ?joined=unavailable popup — instead of the JSON envelope. Mutation-tested: un-listing /login or removing the method check fails seven tests.

  • The reaper's namespaces: get, list, delete cluster-wide grant is gone (ADR-065). It was the most dangerous grant in Sprint 9.6 and could not be narrowed with resourceNames, because a preview's namespace name was derived from a branch and unknown ahead of time — so lib.guard_namespace was carrying the whole weight of bounding a process that deletes namespaces on a timer. The reaper now empties a slot rather than deleting a namespace, and every verb it holds is bound by name; its ClusterRole is an empty rule set, kept rather than deleted so applying it to a cluster carrying the old binding replaces the rules instead of leaving them in force. The guard stays — the operator runs the same tool with cluster-admin, where RBAC refuses nothing — and its first condition is now membership of the slot set rather than a name prefix, which also refuses epic-preview-notes. Writing the tests for it found two real defects: a hand-edited preview-slot: "0" raised ValueError rather than NotAPreview and so escaped every caller's except, and int() accepted " 1 ", "+1" and "1_0" as slot 1, meaning three label values nobody wrote satisfied "the label reproduces the name".

  • OFAC monthly re-screen — 2026-09-10, no change to the six codes (out-of-band). The embedded seed was attested 2026-08-18; seed_re_screen_is_not_overdue tripped at 20 days on 2026-09-07 and main had been red since, with MAX_LIST_AGE due to be crossed on 2026-09-22 — at which point OfacGuard::check fails closed on every country including US and blocks every signup. The tripwire did exactly what ADR-051 built it to do, and the failing gate is the deliverable rather than an inconvenience. Re-screened 2026-08-18 → 2026-09-10 against OFAC's Recent Actions feed and Sanctions Programs index: ten actions, every one an SDN designation/delisting or a licensing change, none altering the scope of a country- or territory-wide embargo — which is the only thing the file encodes. CU stands (2026-09-03 added six SDN entries and issued GL 4A; a general licence authorises conduct within an embargo and the CACR, 31 CFR 515, is untouched — the action tightens). IR stands (2026-09-08 designated 43 aviation entities, suspended GL J-1, issued wind-down GL DD; net tightening inside the ITSR, 31 CFR 560). KP stands, no action in window. The two judgement calls the file asks to be re-made each cycle were re-made: Kherson and Zaporizhzhia stay off — E.O.s 13660/14024/14065 authorise blocking persons who materially support the occupation, which is SDN screening rather than a territorial embargo, and neither oblast has been determined a Covered Region under E.O. 14065 — and SY stays off, its case strengthened by OFAC removing Syria's State Sponsor of Terrorism designation on 2026-08-24. Research assistant-performed against primary sources and countersigned by the operator, matching the provenance the 2026-08-18 cycle recorded. Merging this does not unbrick production: the list is include_str!'d into the binary, so the running pods carry their image's seed until epic-control is rebuilt and rolled. Automating the cycle remains tracker item L8 (OfacGuard::spawn_refresh).

  • Sprint 9.7 — response security headers, on both planes (2026-08-26, Phase 9). The public surface served none: no Content-Security-Policy, no Strict-Transport-Security, no X-Content-Type-Options, no Referrer-Policy and nothing preventing the site being framed — found while checking a claim in this sprint's own T-9.7.2 rationale that a CSP already existed. Both planes now stamp them from a layer mounted outside the panic catcher, which is the position that makes "no response leaves without them" a property rather than a description of the happy path: a panic converted to a 500, a timed-out request, a body over the cap and a rate-limit 429 are all responses no handler produced, and they are the ones easiest for an attacker to provoke. Pinned by a test that panics a route, and mutation-tested by moving the layer one position inward — it fails only that test, because every other response shape would be decorated either way. The document policy carries no 'unsafe-inline' and no 'unsafe-eval' and a test fails the build if it ever does: the inlined stylesheet and playground client are admitted by the SHA-256 of their content, chosen over a per-response nonce because every payload is an include_str! of a repository file, so its digest is a property of the build rather than of the request. That failure is silent — a hash that does not match is not a build error and not a server error, the browser drops the element and the page returns 200 with all its text and no styling — so it gets three gates: a source scan requiring every inline element in epic-web to interpolate a declared constant rather than a literal, a wire test checking one real response's inline blocks against that same response's own header, and visual_regression now rendering through the production middleware tower, which makes the twenty existing pixel baselines a CSP gate at no extra cost. style-src-attr is 'none', which cost 42 inline style= attributes (five declarations repeated across sixteen files) — style attributes are governed separately from style elements and there is no per-attribute hash, so one anywhere forces the exception for the whole document; proven visually inert, with 17 of 20 baselines at pixel-delta 0.00000 and the three that differ reproducing identically on stashed-clean HEAD. Turnstile was verified rather than trusted: Cloudflare documents only script-src and frame-src, and the failure mode if that is wrong is signup, so an env-gated browser test loads the real challenges.cloudflare.com script against Cloudflare's test sitekey under the real policy and asserts the widget produces a token — written as a controlled comparison (re-run with the policy removed) so "the policy blocked it" and "this machine cannot reach Cloudflare" are different outcomes. That design earned itself immediately: the first probe looked for an iframe, found none, and would have read as a CSP failure — the real cause was that Turnstile builds itself inside a shadow root, and a less careful test would have weakened the policy for nothing. epic-edge was added to the task's scope (it was written control-only): it renders no documents so it takes the transport-level half, asserted on health probes, an auth failure, a 404, a CORS preflight and the streaming path, whose headers are written before the first token; values are shared in epic_core::security_headers, layers stay per-plane, ADR-026 re-checked at six header inserts per response with all six zero-alloc suites unchanged. HSTS carries no preload — an operational decision, not a side effect of a constant. DESIGN.md gains §9.7a. See Sprint 9.7 §7.

  • Sprint 9.3 — three client-IP trust defects, found by the IPv6 survey and closed (2026-08-23, Phase 9). None of the three is an IPv6 problem; the survey walked into them because making the address family a configured value meant reading every site that touches a client address. Recorded in full in ADR-060 §Context.

    1. epic-edge honoured X-Forwarded-For from any caller. The edge is a DNS-only grey-cloud record (ADR-015 §2) — QUIC cannot transit an HTTP reverse proxy, so nothing sits in front of the data plane and the transport peer is the client — yet ip_from_proxy_headers preferred the header and never consulted the peer. A single X-Forwarded-For: <address> therefore defeated both the per-API-key IP allow-list and the IP/CIDR kill-switch. The pipeline had documented the intended design all along ("IP allow-list runs in the route handler because it needs ConnectInfo") and both listeners had been installing ConnectInfo since Sprint 2.3 — nothing consumed it.
    2. epic-control keyed every DESIGN.md §9.7 anti-abuse band on a caller-supplied header. extract_client_ip preferred ConnectInfo and fell back to the first X-Forwarded-For hop — but none of the control plane's three serve paths installed ConnectInfo (not axum::serve, not axum_server::bind_rustls, not the h3 bridge), so the first branch could not fire in production and the unvalidated fallback ran on every request. Rotating a fabricated header bought a fresh bucket per request: effectively no rate limit on login brute-force, registration spam, password-reset flooding, TOTP-verify brute force or abuse-report flooding. The same unvalidated value also reached Turnstile's siteverify, the abuse_reports.reporter_ip evidence column, the hash-chained audit log, and the admin API key's IP allow-list check — whose 127.0.0.1 fallback additionally meant an admin key allow-listing loopback was usable from anywhere.
    3. Login wrote a literal 127.0.0.1 into every session row. A Sprint 1.1 placeholder ("the proper extraction lands in T-1.1.1 when the listener is wired") that survived eight phases, so sessions.ip carried no information at all. Nothing renders that column today — there is no session-listing surface — so this was a security record that recorded nothing rather than a broken member-facing feature: the column an incident review reaches for to answer "where did this session come from" answered 127.0.0.1 for every session ever issued, and any future device-management surface would have been built on a constant.

    Closed by one model on both planes (ADR-060 §4): the client address is resolved once, at the ingress boundary, under a configured [client_ip] trusted-hop policy — the transport peer is the client unless the peer is inside a configured trusted-proxy CIDR, in which case the first parseable address from the first present configured forwarding header wins. epic-edge trusts nothing (empty set, the safe default); epic-control trusts the Cloudflare ranges and reads CF-Connecting-IP. All three control serve paths now install ConnectInfo — two in the sprint, the h3 bridge in the post-close self-review below; v4-mapped addresses are normalised at that single point (using to_ipv4_mapped, not to_ipv4, which would let ::203.0.113.5 impersonate 203.0.113.5); and five hand-rolled X-Forwarded-For parsers — which had already drifted apart — collapse into one typed extractor. Both planes fail the boot on a [client_ip] that does not compile, matching Sprint 9.2's [domain] posture, because a half-applied trust list is not a degraded service any probe would catch.

    No test failed when the edge's old behaviour was removed. That is the finding rather than an aside: it is the sprint's own premise — that fixtures which only ever exercise 127.0.0.1 are what let this class hide — confirmed against the tree. The new gates were mutation-tested against the behaviour they replace (restoring the old resolver fails exactly the four bypass tests; removing set_only_v6 fails the two listener tests), and that pass caught an inert UDP coexistence check and replaced it with a direct socket-option read.

  • Sprint 9.3 post-close self-review — six further defects, four of them in the client-IP work itself (2026-08-23, Phase 9). Recorded rather than folded in, because the pattern is the finding: every one is a place where configuration could mean something other than what it said, which is the class ADR-060 §4 was written to eliminate. Closing the code path is not the same as closing the class. Full write-up in ADR-060 §4c–§4f.

    1. The control plane's h3 listener never installed the transport peer (§4c). The sprint fixed two of the three serve paths and missed the QUIC one, which computed incoming.remote_address() for a log line and discarded it. The effect on HTTP/3 was worse than the defect being fixed: with no peer, resolution trusts no header and returns Unresolved, so every h3 request fell to the "unknown" rate-limit identifier and shared one anti-abuse bucket across the whole transport, while no session row, Turnstile call or abuse report recorded an address at all. Fixed by threading the peer through, with a build_axum_request seam extracted so the property is unit-testable without a live QUIC connection (5 tests; 4 fail under mutation).
    2. The deployed control plane declared no [client_ip] at all (§4d). The policy was built, wired, tested — and pointed at nothing. epic-control is orange-cloud proxied behind an externalTrafficPolicy: Local LB, so its peer is always a Cloudflare edge address and the direct() default keys every §9.7 band on the proxy: all members behind one Cloudflare node share a login bucket, and the proxy's address is what reaches session rows, Turnstile remoteip, and admin keys' IP allow-lists. Pre-9.3 that plane read the first X-Forwarded-For hop with no trust check — spoofable, which is defect (2) above, but it did yield a distinct value per honest client. So the sprint closed a security hole and opened an availability one. Closed three ways: the section is now declared explicitly on both planes (empty-and-written on the edge — omitted means nobody asked, empty means someone answered); both planes log their posture at boot; and the new ci/client-ip/verify.py Stage 1 gate fails the build when a plane's declaration stops matching its ADR-015 exposure. Cloudflare's ranges (15 v4 + 7 v6) retrieved from the published lists, with the refresh procedure in docs/operations/cloudflare-zone.md §5a.
    3. ClientIpPolicy::new rejected trust-with-no-header but accepted its mirror (§4e). Headers-with-no-trust is equally inert and more dangerous to leave un-rejected, because it fails safe — the headers are never read, which at run time is indistinguishable from a correct direct posture. Now symmetric.
    4. RFC 7239's own Forwarded header could be configured but could never parse (§4f). Its elements are for=192.0.2.43;proto=https, not bare addresses, so forwarded_headers = ["forwarded"] validated, deployed and then silently fell back to the peer forever. Now parsed, reading for= and ignoring by= / host= / proto= — by= being an address that is emphatically not the client's.
    5. The Ferron IPv6 fix replaced a transport bug with a deployment trap (§4h). T-9.3.10 correctly found that tonic derives SNI from Uri::host(), which for a bracketed IPv6 authority includes the brackets — so https://127.0.0.1:8443 configures TLS and https://[::1]:8443 does not — and fixed it by adding a [ferron] tls_server_name knob, required for any IPv6 endpoint, on the reasoning that "there is no name to derive from an address literal". The same test's IPv4 case disproves that: rustls accepts an IP literal as a server name and matches it against the certificate's IP: SAN. The deployed edge dials Ferron at an IP literal (ADR-039) and Sprint 9.4 is the sprint that repoints it at a v6 address, with nothing in the manifest prompting the knob and an opaque transport error at boot as the symptom. EPIC now derives the name for every family (epic_core::net::tls_server_name_for_uri_host); tls_server_name survives as an override and is normalised identically.
    6. The control plane held two client-IP trust policies, and one of them made a test lie (§4g). RateLimitMiddleware carried its own Arc<ClientIpPolicy> and re-resolved the address separately from the ingress layer — initially as an optional builder step defaulting to direct(). Making it a required parameter was not enough, and mutation testing is what showed it: a new tower-level test asserting two members behind one Cloudflare node get independent buckets still passed when the tower's policy was swapped to direct(), because the limiter consulted its own copy. A test that appeared to exercise the composed stack exercised a private field. The duplicate is now removed rather than parameterised — rate_limit_anti_abuse reads the ResolvedClientIp the ingress layer already resolved, so the plane has one trust policy and one resolution per request, and the limiter keeps only the IPv6 bucket prefix. The same mutation now fails, naming the symptom.

Fixed

  • Two rustdoc links that turned main's doc gate red after #371 (2026-10-23, #406). cargo doc --workspace -D warnings refused a module-doc link to the private drops_event (epic-meter) and a link to FerrumStream::ensure_stream_configured, which exists only on #359 (epic-store); the first is now plain code, and the second names ConsumerSettings::dead_letters and FerrumStream::ensure_consumer_with, which apply those settings on main. No code change.

  • Logging in, signing up and confirming an email work from www, admin and api too (2026-10-23, #401). #354's same-origin rule on the pre-login POSTs compared Origin with the apex alone, but the control plane serves the same pages on the www, admin and api hosts, with no redirect, and the session cookie's domain is the base, so a sign-in there is a real one. A member on www was refused every login, sign-up, confirmation code and new code with 403 cross_site_request. The rule now takes the site's own origins: the one the plane serves itself from and, when that is [domain]'s app origin, the www, admin and api origins, all derived from [domain] (AppState::site_origins). Each is compared whole, so a look-alike (https://www.<base>.evil.example, https://evil<base>), another scheme or port, and null are still refused; Sec-Fetch-Site is checked as before. A preview or a local run, which sets [control] public_origin, has only that origin. Tested on every pre-login POST, JSON and form, through the bare router and the production tower.

  • A vote's quorum is measured against the weight that can be cast (2026-10-22, #385). Quorum compared turnout with Σ orgs.seat_count, every slot active or on hold with any rights, while each ballot weighs only the account's active slots carrying the voting right (Org::vote_weight, ADR-069 decision 9). Slots on hold and complimentary slots without the vote inflated the denominator, so a vote every eligible account joined could still fail quorum. VoteStore::sum_eligible_seats is now sum_eligible_vote_weight and sums slots_voting over the active founding-member accounts. ADR-014 §4 is amended. A Postgres test run as the application role pins the denominator to Σ Org::vote_weight, and an epic-control test closes a fully attended vote at a 100 % quorum. The fleet test that pins the shown fair-share weight to the edge's annotation now uses slots on hold and slots without the capacity right, so it fails if the page shows the slot count or the raw capacity count.

  • The interest total updates live in every tab, and "since your last visit" is per account (2026-10-21, issue #356 items 3 and 4). The Interest box's "N slots of interest from M entries", its bar and the note past one server's slots were drawn once, so each open tab kept the total it had loaded. They now redraw from the GET /app/launch/figures answer the scenes already ask for every 20 s (assets/scenes/counts.js, which now also holds the bar drawing the coin desks used). The likeliness chart, split and average stay as drawn until the next page load: the figures carry no breakdown. The figures' last-visit baseline was one localStorage entry per browser; it is now kept under the account's public id (data-account on the Dashboard's <body>, from a typed Identity.account: Option<UserPublicId>, left off when there is none), and the old browser-wide entries are removed unread. Because those entries say which accounts used the browser, they are kept only while it is signed in (#368 review, the owners' decision): a sign-out from a dashboard or console page removes every one as the form is sent (assets/figures_on_sign_out.js), and the login and sign-up forms when the server finds no live session, and the pages an email change and an account deletion end on, remove them at once (assets/figures_signed_out.js), so an expired session leaves nothing either from the next such page on. The cost: after a sign-out the next visit's "since your last visit" starts fresh. ADR-074 §3 (amended), DESIGN.md §9.7a and epic_control::cookie list what the browser keeps and when it goes; epic_web's browser_storage holds both scripts. What the robots say (item 2) is unchanged, left for the robot-scene branch.

  • Diagnostic captures and review samples are stored, travelling from the edge to the control plane over Ferrum (2026-10-21, issue #329; ADR-079). The production edge never set its capture or review-sample sink, so both fell back to no-op sinks: every capture an org consented to and every review sample an abuse opt-out required was dropped, while ADR-032 and ADR-037 said the Postgres stores were wired. Since #330 the edge cannot write those tables, so the edge now seals each record (Vault.Wrap + AES-256-GCM, the ADR-008 envelope unchanged) and publishes only the ciphertext to epic.diagnostic.captures / epic.review.samples (epic_protocol::sealed, versioned protobuf with no plaintext field, fuzzed). The control plane's consumer re-checks consent (or the opt-out) under a row lock, drops a record withdrawn in flight, inserts the rest ON CONFLICT (public_id) DO NOTHING, then ACKs; a Postgres outage leaves records in the stream for redelivery.

    • The sinks are required: AppStateBuilder::new takes ContentSinks, so a build without them does not compile; tests and the dev stack say ContentSinks::discarding(), and a discarded record is counted. Every record is counted on epic_edge_content_records_total{kind,outcome} and a dropped one logged with public ids, never content.
    • The consumer (epic-control-content-ingest, [control] run_content_ingest, on the singleton aggregator workload) has one stable name, max_deliver 1024, and resumes a paused Ferrum consumer at boot, after a failed pass and on every idle pass, as #325 does for the meter; FerrumStream keeps #325's one consumer API, with ensure_consumer_with for these settings. Metrics: epic_control_content_ingest_{records,failures,consumer_resumes}_total. Runbook: docs/operations/content-ingest.md.
    • Tests: an end-to-end suite through the edge over real Postgres (deployed roles; the edge still holds no INSERT), Ferrum and an mTLS FerraVault with wrap-only and unwrap-only identities; the per-case real-service suite; the zero-allocation streaming suites with both tees firing (3 allocations per token, as without).
    • #359 review: the tees' work is bounded per pod and kind ([content_tees], 16 in flight, 10 s; over it dropped_backpressure, past it dropped_timeout); the stored expiry is never past capture + 7 d / sample + 90 d; a record whose key or user is not its org's is refused (org_mismatch); renewing consent while capture is on keeps the window, so captures from just before a renewal are no longer dropped; the edge holds a wrap-only KeyWrapper and the control plane an unwrap-only KeyUnwrapper; records Ferrum gives up on go to {<stream>}.dlq, are replayed each minute, counted (epic_control_content_ingest_dead_*) and alerted on (content-ingest-rules.yaml).
    • Maran's 2026-10-22 review of #359: the consent audit's renewal flag is the store's answer (EnabledWindow), so a first enable is no longer called a renewal because Postgres keeps microseconds and the clock nanoseconds (F1); a record claiming a time more than five minutes ahead of the control plane's clock, or whose clamped expiry has passed, is refused, acknowledged and counted as rejected (IngestOutcome::{DatedAhead, ExpiredOnArrival}), and the streams keep six days plus one in their dead-letter streams (SEALED_DEAD_LETTER_SETTINGS), so ciphertext leaves Ferrum within a capture's seven days (F3); Ferrum counts no retention eviction, so epic_control_content_ingest_backlog and EpicContentIngestBacklog stand in, and steady edge drops (dropped_ferrum now documented to include Ferrum refusing a publish past its flow-control stop) alert as EpicEdgeContentRecordsDropped (F2); dead_letters_waiting reads Ferrum's contiguous acknowledged position, pinned by a test that fails on the "highest acknowledged" reading (F4); a record of a newer version waits unacknowledged instead of being dropped as malformed, and the control plane rolls first (F5); org_mismatch, malformed and rejected alert (EpicContentIngestRecordsRefused) and the consumer logs the record's org as claimed_org (F6). ADR-079 carries a dated amendment.
    • Production stays fail-closed: FerraVault 1.15 lets only sudo on ** create an org's KEK, and the policies are not changed, so turning capture on and approving an opt-out still fail before anything is written, until ferravault#11 (the request). Deploy note: roll epic-control (the aggregator Deployment now sets EPIC_CONTROL_RUN_CONTENT_INGEST=true) and epic-edge, in either order: a record published before the consumer runs waits in its stream. No migration, no policy change.
  • The landing page's logo was cut off under the bar at high zoom (owners, 2026-10-20). From 768px wide, any viewport too short for the hero — a laptop at 100 %, a desktop zoomed to 125–200 % — lost the top 32px of the logo: the hero lifted its content with a negative margin, which let the content out of the top of the box it was centred in, and the hero's overflow: hidden cut it off. The lift is now padding moved from the hero's top to its bottom: the same picture where the screen has room, and never clipped where it has not. A browser gate, the_hero_logo_is_never_cut_off_or_covered, checks it at twenty zoom-level viewports.

  • A help "?" stops pulsing only once it is opened on purpose (2026-10-17, #320 item 13; #306). #306 marked a "?" as seen whenever it opened, so passing the pointer over it, which opens it for a look and closes it again when the pointer leaves, stopped the pulse for good in that browser. It is now marked from the click on the "?": a click on a closed "?", the click that pins a look, or Enter or Space on it, which the browser delivers as a click. A look that ends when the pointer leaves, or with Escape or a click in the box, leaves it pulsing. The toggle event no longer marks anything: it says that the box opened, not why, and it arrives from a queued task, by which time a look may have been pinned or ended. dashboard_browser_e2e covers hover then leave, hover then click, a click with no hover before it, and Enter and Space, each across a reload. With the old toggle marking, the hover-then-leave case fails.

  • Every control-plane write goes through the cached store (2026-10-17, #320 item 3; follows #209 and #314). #314 moved the profile and avatar saves onto AppState::cache_aware_store(), but many more writes to cached rows still went through the bare PgStore, so the LayeredStore never dropped its copy and a page reading the row through the cache showed the old one for up to the cache TTL (60 s by default). Not a security problem: the edge checks keys and orgs uncached. Moved onto the cached store: the /api/me name change and erasure (and the keys erasure disables); the /api/orgs rename, member removal (and the account it soft-deletes) and the other membership writes; the admin JSON kill-key, suspend and unsuspend; the operator console's suspend, unsuspend, close, kill-switch key and kill-switch org; ending a member's playground access; the checkout profile; every /api/keys write, the playground mint and the admin-key use stamp (the ApiKeyService was built over the bare store); the kill switch's cascade (NCMEC quarantine and abuse auto-disable), which was built over a PgStore of its own, so the DESIGN.md §13.4 claim that a firing invalidates the cache was not true until now; and the ephemeral-key reaper. Store::purge_expired_ephemeral_keys now returns the deleted keys' public ids, so LayeredStore drops their cached copies. The rule is now kept by the compiler: the bare store is a private field of AppState, reachable only through AppState::uncached_reads(), a new UncachedReads view that has the Store reads and none of the writes. Reads that were uncached on purpose stay uncached: membership and role checks, an org's standing before an operator acts on it, and org rows, whose seat and right counts the database keeps from slot rows that no cache invalidation sees. Guards: compile_fail doctests (a write through UncachedReads, and the field from outside the crate) and a test that fails if anything in epic-control/src outside state.rs reaches the field or builds its own PgStore. Tests: each kind of write caches the row through a real LayeredStore, runs the handler and reads it back through the cache; LayeredStore gains one test per cached row type covering every write to it.

  • Avatar and profile saves clear the cached account; two org-admin pages read the session once (2026-10-16, issue #209). The avatar upload and removal and the profile save wrote through the bare PgStore, so the LayeredStore invalidations written for them never ran in production, and a cached account kept its old name and has_avatar until the cache TTL (60 s by default). They now write through AppState::cache_aware_store(). The diagnostic-capture and abuse opt-out pages used authenticate() and then read the session and the primary org again in their admin check (the diagnostic POST three times); the check now uses the session authenticate() resolved. MemSessionLookup::find_count lets tests pin one session read per request.

  • The meter aggregator counts each event once, even after a crash (2026-10-16, #320 item 2; issue #185). A crash after the rollup committed and before the Ferrum ACK redelivered the batch, and the additive UPSERT counted it twice. Migration 0086 adds meter_applied_events, a ledger keyed on MeterEvent.request_id. MeterAggregateStore::apply_meter_events (replacing upsert_meter_aggregates) records each event's id and adds only the events it had not recorded, in the same transaction. It is keyed on the event's own id rather than the stream offset, because Ferrum redelivers on a timer, after later offsets may have committed, and a stream reset renumbers offsets. An event with no usable request_id is dropped and counted as malformed. events_already_applied on the aggregator health route counts the redeliveries absorbed.

    • #325 review: no replay counts twice. The aggregator's Ferrum consumer was named after its pod, so every roll created a new consumer, which reads the stream from its start: each roll re-counted the whole history (the production rollups are over-counted by every past roll) and left a dead consumer that Ferrum keeps for good and whose position its publish flow control waits on. The consumer is now one configured name, [boot.meter_aggregator] consumer_name (default epic-meter-aggregator), with event_horizon_days (default 30: an older event, judged by its UUIDv7 request_id, is dropped and counted on events_dropped_beyond_horizon) and initial_deliver (all, new or by_sequence:<offset>, used only when the consumer is created), each with an EPIC_METER_AGGREGATOR_* override. The ledger keeps ids for the horizon plus a day and is pruned by request_id range, 10 000 rows a tick; observers see only newly applied events; ledger and rollup rows are locked in one fixed order.
    • Ferrum pauses, and leftover consumers. Ferrum pauses a consumer whose pending records go un-ACKed past the stream's stuck threshold (150 s by default; a Postgres outage that long), and only CONSUMER.RESUME clears it; the aggregator now resumes it on an idle tick (consumer_resumes) and counts the other consumers on the stream hourly (stream_other_consumers). GET /api/admin/meter/consumers and POST /api/admin/meter/consumers/{name}/delete (admin key, reason, audited; the aggregator's own consumer refused) are how an operator sees and removes them, since the Ferrum image has no CLI.
    • Auto-disable counts each reject at the time it happened, and ignores rejects older than its hour, so a backlog or a rebuild cannot disable a key.
    • #325 review, questions 3 and 5: a failed lookup retries, a backlog drains, and a paused consumer reads on. A failed lookup of an event's organisation or user (a database error, not a missing row) fails the tick instead of dropping the event and ACKing it, so Ferrum redelivers the batch and the ledger counts it once; a missing row, or ids that do not parse, still drop the event. After a full batch the loop pulls again at once instead of waiting 60 s, so a backlog or a §4b rebuild drains at about 1 000 events a second (measured locally) instead of 1 000 a minute. And after resuming a consumer Ferrum had paused, it pulls at once too: Ferrum pauses it again within 500 ms unless the paused records are read and ACKed first, so with the 60 s wait the rollup never moved again after any outage over 150 s. The drop counters now count each event once, when its batch is ACKed. Maran's 2026-10-22 review: a failed lookup holds back only its own event (events_held_lookup_failed; the tick fails with LookupsFailed), and the rest of the batch is counted, so one organisation row this image cannot read no longer stops everyone's metering. Ferrum gives an event up after max_deliver deliveries, about one a minute under such a failure; owners' decision (Melvyn): the consumer is created with max_deliver 10 (Ferrum's default 5 dropped the batch after five minutes, the first proposal's 1 024 held the stall for 17 hours, past Ferrum's 16 384-record publish stop) and a dead-letter stream on the source's shard, {epic.meter.events}.dlq (METER_CONSUMER_SETTINGS, METER_DEAD_LETTER_SETTINGS: 30 days, 256 MiB), which keeps such an event rather than dropping it. epic-store gains #359's unified ConsumerSettings (dead_letters: Option<StreamSettings>) and creates the dead-letter stream before the consumer. Ferrum fixes both at creation, so they ship before #325's switch-over creates epic-meter-aggregator; the replay of dead letters, the console's dlq_total, alerts and a check of an existing consumer's settings follow the deploy. The ledger prune and the consumer audit now run on a failed tick too (a persistent failure skipped them), and a resume in the middle of a drain no longer logs aggregator caught up. See docs/operations/meter-aggregator.md §2a.
    • Deploy note: the first deploy needs the one-off switch-over in docs/operations/meter-aggregator.md §4; with the default initial_deliver = all the new consumer would re-count the last 30 days. The aggregator therefore refuses to create its consumer at all while any counting inside the horizon was done without the ledger, saying what it found and the fix in the health route's last_error and retrying every tick, unless [boot.meter_aggregator] allow_full_replay (EPIC_METER_AGGREGATOR_ALLOW_FULL_REPLAY) is set. It reads that from the stream (other consumers over an empty ledger) and, since Maran's 2026-10-22 review, from Postgres: rollup hours inside the horizon that end before the ledger's oldest event, or another consumer's cursor row that counted inside the horizon before the ledger's first apply (MeterAggregateStore::{oldest_applied_meter_event, meter_rollup_hours, meter_cursors_ticked}, replacing has_applied_meter_events). So deleting the old consumers before the switch-over, or re-creating epic-meter-aggregator at all within 30 days after it, is refused too; §4c re-seeds it, and the guard lets all through once the ledger spans the horizon.
  • An API key's "Last used" is real (2026-10-16, #320 item 6; issue #148). api_keys.last_used_at had no writer for inference keys, so the Account tab said "never" for every key, including keys in use. The meter aggregator now takes each key's latest event in a tick (MeterEvent.api_key_public_id, max(emitted_at)) and moves last_used_at forward with one forward-only UPDATE as epic_admin (MeterAggregateStore::record_api_key_last_use). The edge pays nothing, a redelivered batch never moves a key back, and an event stamped in the future counts as the tick's own time. A key used before this deploy but not since still shows "never" until its next use.

  • The menu mini bar can be dragged more than one place on Chromium (2026-10-16, #320 item 8; #242 item 8). The drag captured the pointer on the dragged tab, and moving that tab within the strip could release the capture on Chromium, which ended the drag after one step. menu_editor.js no longer captures; it listens for the move and the release on the document, as the dashboard boxes' drag (layout.js) does. A new browser step drags a tab left past two others and checks it lands two places earlier and is saved. On Chrome 155 it fails against the old code (one place) and passes now, and it passes on Firefox as part of the full suite. EPIC_CHROME_BINARY lets the suite launch a Chrome for Testing build.

    • A drag no longer swallows the next click (#326 review). The click that follows a release was marked by a flag that only that click, on reaching a tab, could clear. Without capture, a release off the strip sends its click to what the press and the release have in common, outside the strip, and Chromium sends none once the pressed tab has moved, so the flag stayed set and ate the member's next click on a tab. It now lasts for that one release (cleared by a 0 ms timer, which runs after the release's click and not before it, as a microtask would), and it is checked before looking for a tab.
    • A drag that is not dropped puts the tabs back and saves nothing (#344 items 1 and 2). A cancel kept the half-done order and saved it. And with no capture to lose, nothing ended a drag whose release went elsewhere (a context menu opened mid-drag, a switch to another window): the tabs followed a pointer with no button held, and the next click anywhere saved where they had got to. A move with no button held, a fresh press of the same pointer, or the window losing focus now abandons the drag the way a cancel does. Restoring rather than keeping is deliberate: a release that never arrived is not a drop, and nobody chose the order the tabs were in when it went missing.
    • The dashboard boxes had both defects, and get both fixes (layout.js). There the stuck flag also ate a keyboard press of a grip, since the grip is a button and Enter and Space are clicks; a lost release could also leave the page scrolling under the box near an edge. A box drag that is cancelled already put the boxes back.
    • New browser steps: a click on a tab after the two-place drag, and after a drag released below the strip; a press of a grip after a box is dragged up (which moves the pressed grip itself); and each of the four ways a drag can end without a drop, for the tabs and for the boxes, checking the order is put back, nothing is left lifted and nothing is sent. Run on Firefox (the whole suite) and on Chrome for Testing (the drag steps).
  • A details save over another tab's is refused, not written over it (2026-10-16, #320 item 8; #242 item 6). The Account details and Profile forms compared against values read in the same request, so a save from a page opened before another tab saved silently overwrote it, despite the 409 ADR-075 promised. Both forms now carry the row's version as rendered (UserRowVersion, users.updated_at to the microsecond) in a hidden seen field, and Store::update_user_details(id, seen, next) writes only while the row is still at it. Otherwise, or with no version, the member gets the 409 page saying to reload and make the change again, with the posted form's own tab (Account or Profile) selected. Store::user_row_version reads the version uncached, and each page reads it before the values it shows rather than after them (#341 point 3): read last, a save landing between the reads paired the old values with the new version, and saving the page then wrote them back over that save. Any write to the account row moves the version, so an avatar or password change in another tab also asks for a reload; the owners chose that over a narrower details-only version (#327), and the in-memory store's version now includes the avatar so it moves in the same cases. The Profile card takes its avatar from the same read as its version. A page rendered before this deploy carries no version, so its first save after the roll asks for a reload.

  • Follow-ups from the #241 re-review: names, caches and the browser checks (2026-10-16, issue #242 items 4, 5 and 9, plus nits). PATCH /api/me refuses a display name with control or invisible formatting characters, as the Account tab's form already did, so a bidi override can no longer reach the rest of the group through the JSON API (item 4). The email-change confirm writes through the cached Store, so the cached account is cleared with the change; it wrote through the bare PgStore, and the comment saying otherwise was wrong (item 5). LayeredStore::update_org clears the cached org around its write instead of writing the new org over it, so a read that raced the update cannot leave the old org cached. The hidden-formatting set gains U+00AD, U+034F, U+115F, U+1160, U+17B4, U+17B5, U+3164 and U+FFA0. ci/visual/run.sh with no pixel baselines skips only the pixel comparison: csp_browser_e2e and dashboard_browser_e2e need none and now run (item 9). DESIGN.md's email_change_tokens entry, ADR-075's "no sweeper" and the billing module's /app/billing reference brought up to date. Two paths that audit before an irreversible step now also audit a failed outcome, so the trail never ends at an intent that did not happen: user.email_change_failed (with the reason) and slot_interest.clear_failed.

  • The SSR control gate requires every router to be mounted (2026-10-16, issue #170). ci/routes/verify.py proved a .route(…) was declared, not that its router was mounted: with the .merge(logout::router()) line commented out, /logout 404'd and the gate stayed green. A registration now counts only if its function is reachable from state::router through .merge(…)/.nest(…) calls (module paths, use imports and pub use … as … re-exports resolved; a mount it cannot resolve fails rather than guesses), and a router nothing reaches fails on its own, including API routers no control links to. Routes declared only in test code no longer count. Mutation-checked on the real tree (logout, the billing API, avatar, epic-web's router, the kill-switch console) and by eight new gate tests.

  • The launch animations, before release: the rest of the review (2026-10-15, issue #301 items 3–8, 10–20).

    • Privacy and load. The GDPR export carries the animations switch. The figures route checks the session only, serves the figures from a shared 10 s cache, and leaves out paused boxes. The scenes stop polling once signed out, and poll only while a figure scene is on screen.
    • Motion. Nothing in a scene moves under reduced motion. The scenes follow reduced motion as it changes. Switching animations off stops the same transitions reduced motion does, and a test keeps the two in step.
    • Correctness. The appearance controls change the stored row under its lock, so concurrent changes all survive. Queued figure changes are acted out as one net change. Clerk's prices are grouped like every other figure. The give-away text names the same pool as its bar. The "?" stops pulsing once it has been opened.
    • Size. The robots' stylesheet goes only to the signed-in pages (site.css is back to 128 KB on every page). scenes.rs, scenes.js, interest.rs, promo.rs and stages.rs are split under 500 lines each.
    • Docs. DESIGN.md and ADR-077 §5 document the deposit.
  • The launch animations, before release: the desk bar, migration 0084 and the console header (2026-10-15, issue #301 items 1, 2 and 9).

    • The desk bar. The desks' bar now carries its figures as plain digits (data-value, data-goal), and the count line is the figure and the rest of the sentence in spans of their own. scenes.js redraws the bar from these as meter::geometry does, past the goal too, rather than reading figures back out of the label. As the page opened with a sale since the member's last visit, the scene had added that sale to the figure the page already showed (2 sold, 1 seen: the bar went to 3). The bar is now set back to the last visit's figure as the page opens, and Strider pulls it on to today's.
    • Migration 0084. It adds the deposit column, lowers the deposit to a row's multi price where that is below 50, and only then adds the deposit_aud <= multi_aud CHECK, so a database with test prices no longer aborts the roll.
    • The console header. The header switches' forms are flex boxes, so the operator console's header keeps its height and its pages no longer sit a fraction of a pixel lower.
  • Password reset and login find an account whatever the capitalisation of its email (2026-10-14, issue #91). find_user_by_email compared the CITEXT column against a text-typed bind, which Postgres resolves as text = text: case-sensitive. A member who signed up as Ada@… silently got no reset email as ada@…, while signup's UNIQUE index (case-insensitive) told them the account existed. Every other email match already cast $1::citext; MemStore matched case-insensitively, so no in-memory test saw it. Real-Postgres regression test, mutation-checked.

  • A control plane that boots while the vault is sealed now heals its Postmark token and its AAAA reconciler (2026-10-14, issue #222). Both read FerraVault once at boot and never retried: Postmark sent every email with an empty token and /ready still said "postmark": true; DDNS never started. The F19 vault reconciler now carries them as pending boot secrets, fetched each pass until installed — the Postmark token into a lock-free slot every clone of the client shares, the Cloudflare token by starting the reconciler late. /ready reports "postmark": false while the token is empty, and every send in that window is refused before reaching Postmark (PostmarkError::TokenPending) rather than going out unauthenticated. Only a vault that is unavailable is waited out: a fetch refused as AccessDenied or NotFound is configuration, logged once and dropped — otherwise a preview slot, which mounts production's config but reads only epic-preview/**, would retry its denied Postmark read every 30 s for the life of the pod (#277 review). Tests pin the pending-secret decision, the spawn condition, the refusal path and the fail-closed send; each fails when its fix is reverted.

  • The interest tracker's five-answer privacy floor counts only answers (2026-10-13, issue #242 item 1). The breakdown — the individual/team split, the average likeliness and the chart — is held back until INTEREST_DETAIL_MIN_ENTRIES (5) members have answered, but the gate counted entries carried over from the old interest list, which state neither a kind nor a likeliness. With four carried over, the first member to answer opened the breakdown and it showed that one member's split and likeliness exactly. The gate now counts InterestView::answered_entries (individuals + teams); an empty tracker still shows its invitation. A test with four carried-over entries and one answer fails under the old gate.

  • An affiliation code is taken only under READ COMMITTED (2026-10-14, Maran's review of #269; ADR-077 §8). Migration 0081's claim re-checks retired_affiliate_codes after waiting on the name's lock, and sees a retirement committed meanwhile only because READ COMMITTED takes a fresh snapshot per statement. Under REPEATABLE READ or SERIALIZABLE the snapshot predates the wait; serializable isolation does not catch this case either (one read/write edge, no cycle). Every writer runs at READ COMMITTED today; migration 0082 makes it a rule, refusing a write that takes a name — an INSERT, or an UPDATE setting a new, non-null code or personal code — at any other level (SQLSTATE 25000). Giving names up (leaving, an operator's removal, an erasure) still runs at any level, and READ UNCOMMITTED, which Postgres runs as READ COMMITTED, is accepted. A Postgres test, all as the application role, sees an UPDATE and an INSERT refused under both stricter levels (it fails without the guard), an operator's removal and a leave go through under REPEATABLE READ (it fails if removals are refused too), and the stores' own claims succeed. Also: "Email this member"'s docs now call a question.email record an attempt, since a stopped process, a failed second record or a lost Postmark reply can each leave it standing alone.

  • Two console actions no longer act before they are recorded (2026-10-13, issue #264, found reviewing #245; ADR-077 §4, §5). A slot-price save committed the prices, then wrote its audit record; "Email this member" sent, then recorded — so a failed record left the action standing unrecorded, against CLAUDE.md §8.2. SlotPriceStore::set_slot_prices now takes the record's builder, and PgSlotPriceStore appends it with PgAuditSink::emit_in in the price transaction (the same emit_in as #261, identical so the two merge cleanly), so the change and its record commit together or not at all; epic-control hands the store the deployment's signing sink. An email is recorded before it is sent and not sent if the record fails; a failed send adds question.email_failed. Tests: the console refuses a save whose record is refused and leaves the prices; a Postgres test revokes the app role's INSERT on audit_log and sees the change roll back; the email test checks nothing goes out unrecorded and a failed send is recorded.

  • A code being given up can no longer be claimed before it is retired (2026-10-13, issue #262, found reviewing #245; ADR-077 §8). Migration 0074's namespace trigger locked only the names a row was taking, and only on insert and update. A member leaving the programme, an operator removing a personal code, or an account erased gave its names up without the lock, and their retirement is written later in that same transaction. A member claiming the name in that window passed the retired check, waited on the unique index, and held a retired code once the leave committed. Migration 0081 locks every name a write touches — given up or taken, on delete as well — until commit, in lock-key order (a 32-bit hashtext can give two names one key, and ordering by name could then deadlock), so the claim waits and then sees the retirement. Three Postgres tests hold each give-up open, block a claim behind it under the NOSUPERUSER NOBYPASSRLS role, and commit; all three failed against 0074 alone, with the claim handed the retired code.

  • A colour scheme picked after a custom colour shows at once (2026-10-11, ADR-073). A custom colour reaches the page as an inline style block, which is unlayered, so it outranked the presets' @layer base rules whatever their specificity; a preset picked afterwards (applied without a reload) changed data-accent but not the colour until the next reload — the owners saw it as "doesn't change until you hit the light/dark icon". The block now applies only while the custom colour is the choice, on its own base (body.app-shell[data-base=…][data-accent=custom]). The Dashboard browser test now runs under the production middleware (a bare router rendered every page in the default scheme, which is why this was invisible) and reproduces the sequence: it fails without the fix.

  • Clicking a colour scheme no longer reloads the page and jumps to the picker (2026-10-08, Sprint AP.6). The swatch script posted new FormData(form), which goes out as multipart/form-data; the server's form extractor answered 415 and the script's fallback reloaded the page to #colour-schemes — on every preset click since the picker shipped. It now posts url-encoded and the page does not move. The reloads that must remain (a custom colour, the header's dark/light switch) put the page back where it was. The selected swatch's ring is now an outline with nothing painted in the gap, removing the stacked-shadow seam that could show as a light edge around the swatch.

  • The control plane's HTTP/3 bridge could panic after answering a request whose body it had not read in full (2026-10-07, #226). When a handler answered without reading to the end — the 413 for an over-limit body, the sign-in redirect, the "no free upload slot" page — the bridge told the client to stop sending with h3's stop_sending(H3_NO_ERROR). In h3-quinn 0.0.10 (the latest release) that unwraps a QUIC stream it no longer holds whenever a read is parked, and h3 leaves one parked even after a read that returned data, so the state cannot be checked from outside. The panic was in the request task after the response was out, so the client saw a correct answer and every test passed through it. The bridge now drops the unread stream instead, and quinn sends STOP_SENDING (application code 0) itself. integration_h3_body_limit now fails on any panic in the bridge and checks that a refused client is told to stop; with stop_sending put back the suite fails on every run, and with the stream kept alive the three "told to stop" checks fail.

  • FerraVault client-certificate renewal: a missed renewal is seen whichever one it is, a sealed vault no longer reads as a refusal, and the alert rules are tested (#221 / #195 review follow-ups, 2026-10-07). Melvyn's non-blocking review comments, all closed:

    • EpicVaultIdentityNotRenewed ages the certificate in use, not the pod. It compared pod age with a renewals counter that resets on a process restart, so a container restarted after its day-60 renewal fired it falsely, and it never saw a missed second renewal. Every FerraVault client now exports the notAfter of the certificate it is using (epic_<plane>_vault_identity_not_after_seconds, epic_vault_audit_identity_not_after_seconds, read with the renewal digest in one pass by the new epic_net::serving_cert::pem_pair_identity). The rule tickets under 21 days and the new EpicVaultIdentityExpiryImminent pages under 7, the serving certificate's thresholds. That catches every renewal, survives restarts, and also catches cert-manager never renewing. Both live renewal tests pin the gauge to the mounted leaf's notAfter before and after the switch.
    • The renewal probe reads an answer by where it comes from (FerraVault v1.15.0 layers Sealed → RateLimit → Auth → handler). INVALID_ARGUMENT, a handler answer, is accepted; FAILED_PRECONDITION (sealed) and RESOURCE_EXHAUSTED (rate-limited) are undecided (IdentityUndecided): retried like a refusal but not counted as one, so a seal longer than an hour no longer raises the renewal-refused alerts. Same rule in epic-vault, epic-secret-sync and epic-vault-audit; each has a live test that seals an in-process FerraVault mid-renewal, and counting FAILED_PRECONDITION as refused fails all three.
    • An owed JWT reload is due at a fixed instant. Each loop turn used to restart its backoff sleep, so events the task ignores (a KEK rotation, another key's) arriving more often than the backoff held the retry off indefinitely.
    • The alert rules are tested, in Stage 1. The new ci/alert-rules/ gate runs the pinned promtool 3.15.0 (sha256-verified, cached under EPIC_BUILD_TOOLS, now /cache/build-tools on the runner) as check rules over every PrometheusRule and test rules over committed tests for all eleven vault alerts. The "promtool-tested" claims in the #195 and #221 entries below had no committed test to reproduce them; now they do. Restoring the old pod-age rule, or loosening the threshold, fails the gate.
    • Docs. Runbook §9 step 4: positional selection changes which records v1.14 grandfathers, never how many, so an equal count proves nothing on a forked node, and the baseline line is logged only when something is grandfathered (accounted_orphans / unaccounted_orphans confirmed against ferra v1.15.0's source). EpicVaultAuditRenewalRefused says what happens when a transport blip drops the auditor's connection while a refused renewal is on disk. ADR-025 gains a dated revision. The relay test's threshold comment now counts TLS records rather than HTTP/2 frames.
    • The ~30-day floor the expiry rules assume is enforced (#228 review, 2026-10-08). Both identity-expiry rules now exclude epic-preview-* namespaces, whose FerraVault identity is a 30-day leaf renewed with 7 days left and would otherwise sit under 21 days for most of its life. ci/ferravault/verify.py reads each rule's threshold and exclusion from vault-watch-rules.yaml and fails when an identity the rules age is renewed less than a week before one would fire. The rule tests pin each threshold to within an hour from both sides, where before any ticket threshold from 20 to 35 days passed. A new epic-net test writes leaf + CA to tls.crt and asserts the leaf's notAfter; taking the last certificate instead now fails it, where before every suite passed.
  • FerraVault clients now pick up renewed certificates, and the vault watch resyncs wherever it cannot replay (#216, #217, #186, 2026-10-06). No FerraVault client ever re-dialled with a renewed certificate: VaultClient::rotate_identity had no production caller, and ADR-025 §7's rotation pipeline was never built. The edge, control, epic-secret-sync and the auditor kept their boot certificate, and FerraVault re-validates it on every request. So between day 30 and day 90 of pod uptime, control logins would all have returned 503, the edge's Ferron key rotation would have stopped, and Secret syncing would have gone stale, all masked so far by deploys. Each client now polls its mounted certificate by content with the one digest the serving-certificate reloader uses (epic_net::serving_cert::pem_pair_digest), re-dials on a change, and reopens its Watch on the new channel. Renewals are exported as epic_<plane>_vault_identity_renewals_total. Separately, the watch session lost events wherever it had nothing to resume from: the first open after boot (#186), a refused pattern's readmission, and OUT_OF_HISTORY, which it had retried with the same revision forever, as did epic-secret-sync (#217). FerraVault gives a read-only client no current revision, so the session now resyncs instead. It drops the KV cache, nudges revision subscribers, and sends a new KeyEvent::Resync, on which control re-reads its JWT public keys. Live tests (renewal on a seconds-long certificate for epic-vault, epic-secret-sync and the auditor; resync on first open and readmission) are each mutation-checked. ADR-025 Amendment. Review fixes (#221): (1) A renewal is switched to only once FerraVault accepts it. Under TLS 1.3 a certificate FerraVault will refuse (an issuer missing from its client CA bundle, or no binding for the DN and exact SAN set) still dials, and it was swapped in with its digest recorded. Every replica would then fail within a minute, all at once, and never go back to the still-valid certificate. Now the new channel first opens one live-only Vault.Watch on the plane's own patterns and drops it. Only an answer from an authenticated handler counts: the stream, or PERMISSION_DENIED. UNAUTHENTICATED, a TLS failure or a timeout is VaultError::IdentityNotAccepted, which keeps the current channel and records no digest, so the next poll retries it. epic-secret-sync verifies the same way on its own watch paths, and a config with no [[secret]] mapping is now refused (it would have nothing to verify against). New epic_<plane>_vault_identity_renewals_refused_total, and two promtool-tested tickets: EpicVaultIdentityRenewalRefused (refused for an hour), and EpicVaultIdentityNotRenewed (a pod over 61 days old that has never switched to a renewal, via kube_pod_start_time, covering the auditor too). (2) rotate_identity(cert, key) is gone. Nothing outside a test called it, and a call with other paths would have been undone by the next renewal poll. The swap is the crate-private VaultClient::redial of the configured identity, and half an identity is refused at VaultConfig::new (now with a unit test). (3) A readmission from the "every pattern refused" branch owes a resync, as one while streaming does. Before, it resumed past whatever happened under the other pattern. (4) The readmission test now tests readmission: an event first moves the revision past 0, so the reopen resumes and only the owed resync can explain one. A new live test drives the all-refused path by unbinding the subscriber. (5) A JWT verifier reload that outlasts its quick retries is owed, retried on a 1 s → 60 s backoff alongside new events until it succeeds, instead of waiting for the next rotation. (6) Secret-sync subscribes live before its OUT_OF_HISTORY resync, not after. All six arms of the watch session and the identity reloader share shutdown_requested. Stale docs fixed (Watcher::changed, rotation.rs, ADR-025's amendment revised). Mutation-checked: swap unverified (both crates), UNAUTHENTICATED accepted, the all-refused and the streaming readmission resyncs, the dropped JWT reload, and the uncounted refusal each fail a test.

  • FerraVault rolled to v1.15.0; a vault restart no longer silences the edge's key rotation, and the auditor's first run found four kinds of live drift (#183, #193, 2026-10-06).

    • #183. Edge, control (with its aggregator) and secret-sync were rebuilt from one revision onto ferravault-client 1.15.0, whose HTTP/2 + TCP keepalive ends a Watch to a vanished vault pod within about 30 s. Then the daemon was rolled (single voter, Ready about 78 s after the upgrade, unsealed by the reconciler). Both edge pods, not restarted across it, delivered the rotation probe at its revision. The "roll the edge after every FerraVault restart" step is retired in three runbooks and the alert comment.
    • Found on the first run, all fixed. epic-vault-audit's first check found that:
      • epic-control, epic-edge and epic-preview each still held a cert: rule their files dropped in ADR-071 / #168. They are re-registered, and the auditor cleared on the POLICY_PUT event, 2.1 s after the write.
      • The identity registry listed no SANs for any identity, while FerraVault keys a binding on the DN and the exact SAN set and every EPIC certificate carries a SPIFFE URI. The auditor itself, bound per the old recipe without --san, was refused mtls binding not found. The registry is corrected, verify.py now fails when registry SANs differ from the Certificate manifest (mutation-tested), and every documented ferra mtls bind gains its --san.
    • First v1.15.0 start. It reported this node's three historical off-chain audit records. Each was the last record before a pre-v1.13 daemon stop (a denied health-probe read whose commit never landed; the next boot record chains from the same predecessor), and they are acknowledged with that evidence; ferra audit verify is OK.
    • Backup CronJob. It was found still on 1.12.1-debug (the 1.14.0 pin was never applied); it is now on 1.15.0-debug and proven by a one-off run.
    • Live drill. A throwaway binding was flagged as unregistered 2.1 s after MTLS_BINDING_PUT and cleared on delete. ferravault-verify-live.sh exits 0.
  • One refused FerraVault watch pattern no longer silences the rest, and vault access is now checked, not assumed (#168, ADR-072, 2026-10-06). FerraVault refuses a whole Vault.Watch on the first pattern a policy does not grant, and the watcher retried that set forever — the edge's Ferron bearer-key rotation was dead for two weeks (#167) with every pod Ready. epic-vault now probes a refused set one pattern at a time, keeps the granted patterns streaming, re-probes the refused ones so a policy fix lands without a restart, validates patterns against FerraVault's grammar at boot (WatchPattern), and records every pattern's state on a WatchStatusBoard exported as epic_<plane>_vault_watch_pattern_state{watch_pattern,watch_state} + epic_<plane>_vault_watch_connected, with EpicVaultWatchPatternRefused (page) and EpicVaultWatchDisconnected (ticket). The control plane's signing-key rotation watch is derived from [boot.auth_signing] jwt_signing_key_id instead of restated in watch_patterns; that restatement had left previews subscribed to production's key path and kv/epic/**, so every preview's watch was refused outright — previews now set EPIC_VAULT_WATCH_PATTERNS=kv/epic-preview/** and follow their own key. New Stage 1 gate ci/ferravault/verify.py derives every vault request each identity makes from the deployed config (registry: deploy/k8s/epic-state/ferravault-identities.toml) and evaluates it with a port of FerraVault's evaluator: a refused request, an unused grant, an ignored policy key, an unbound policy, a subject-DN/certificate mismatch, an unknown cert spec, or a preview policy reaching production fails the build. It removed the preview policy's cert:epic-preview-wildcard rule, which named a path FerraVault never authorises. deploy/k8s/epic-state/ferravault-verify-live.sh compares the live vault's policies and bindings with the repository inside the operator ceremony (a scheduled check needs full admin in FerraVault 1.14.0; the least-privilege read it waits on is ferravault#8, tracked as #193). Deploy: order-independent; then re-register epic-preview and run ferravault-verify-live.sh once.

  • The CI build cache filled the disk the cluster runs on, and every pod was evicted for about four hours (2026-10-06). The PR Gate and nightly sweep build into a persistent CARGO_TARGET_DIR (/srv/ci-cache/target on lab-git) that nothing pruned, with incremental compilation on: it reached 1.5 TB (1.1 TB deps, 324 GB incremental). /srv also holds the Talos nodes' container filesystems; at 88 % used the kubelet's imagefs.available < 15 % threshold tainted both nodes disk-pressure from 16:28 UTC, and edge, control, Postgres, Ferrum, FerraVault, FerraCert and cert-manager were evicted and could not reschedule while every node reported Ready. Recovered by deleting the incremental cache (cache only). Fix: both workflows set CARGO_INCREMENTAL=0 (CI never benefits from it, and sccache — already configured — refuses incremental builds, so it had been bypassed for the workspace crates, the only ones Cargo builds incrementally), and a new first step, ci/cache-guard/guard.py, empties the target cache whenever its filesystem is at least half full. It is a deletion tool, so it refuses any target that, after resolving symlinks, is not a target directory sitting directly in a cache root that is not /; inside it, it walks by O_NOFOLLOW file descriptor, unlinking symlinks rather than following them and stopping at any mount nested in the cache; it warns when emptying the cache does not bring the disk under the threshold (something else is filling it), and a failed deletion exits 3 with a one-line reason rather than a traceback (26 tests, in Stage 1, each check mutation-tested). It empties the directory rather than removing it because in CI the directory is a bind mount: the first version's rmtree deleted every file and then failed the job with EBUSY on the mount point, which the new test reproduces (and the old code was confirmed to fail against a real bind mount in lab-git/ci-rust:1.95.0). It is only safe because the runner is capacity: 1; the workflows README now says a second job slot needs its own target directory. Separating CI's disk from the cluster's is #219.

  • The PR Gate leaked two test Postgres containers per run, and nothing reclaimed them (2026-10-05). This was the real cause of the cargo doc OOM kills the two entries below capped job counts for. The epic-db harness's #[dtor] reaper and ci/docker-hygiene/sweep.py both shelled out to docker, but the gate's job image has no docker CLI: the tests reach the daemon through the socket act_runner mounts. So every gate run logged leaked test container …: could not run docker twice, and the sweep crashed on FileNotFoundError behind ci.sh's || true. 273 containers had accumulated inside the runner's 16 GiB cgroup (8.4 GiB of it in swap, host swap 100 % full), so each run had less memory than the last, and rustdoc died on a different crate every time, even at -j 2. Removing them took the runner from 4.9 GiB to 1.2 GiB and its swap from 8.4 GiB to 42 MiB. Both now speak the Docker Engine API over the daemon socket ($DOCKER_HOST unix://, else /var/run/docker.sock), with only the standard library: a new blocking epic_test_utils::docker::remove_container for the reaper (it runs after the tokio runtime is gone; it falls back to the CLI only for a non-Unix DOCKER_HOST), and an http.client Unix-socket client in the sweep. ci.sh now warns when the sweep fails instead of hiding it. Proven with the docker CLI shimmed out of PATH: schema_invariants 13/13 leaves 0 containers and never calls the shim; the old reaper leaves 1 and calls it. 8 new Rust tests (a fake daemon on a temp socket) and 10 new sweep tests, including main() end to end against a fake Engine API.

  • cargo doc runs at two rustdocs, because four were still OOM-killed (2026-10-05). The entry below capped every cargo step at 4 jobs and credited that with ending the cargo doc kills; it did not. Four concurrent rustdocs were killed in all three gate runs after it merged (its own, main at 4d7683c, #176), while #190's cargo doc -j 2 passed the full gate. ci.sh now runs cargo doc with -j ${EPIC_DOC_JOBS:-2} (Melvyn's #190 change), never above the cargo job count, and an invalid value fails the run. The workflows README replaces #190's request for a runner memory limit, which already exists (16 GiB), with the measured account.

  • The PR Gate OOM-killed its own cargo doc (2026-10-05). Every open PR's required gate was red for a reason unrelated to the PR: rustdoc died with signal: 9, SIGKILL on a different crate each run. The runner container is capped at 4 CPUs / 16 GiB, but the gate runs in a job container nested inside it whose cgroup shows no limit and whose nproc reports the host's 8, so cargo ran eight compilers in 16 GiB (the runner cgroup had logged 21 oom_kills). ci.sh had documented EPIC_CI_JOBS ("default: nproc") from its first version and never read it. It now does: the new ci/cargo-jobs/jobs.py (15 unit tests, run in Stage 1) resolves the job count as the smallest of nproc, the cgroup CPU quota and the cgroup memory limit / 4 GiB per job, exported as CARGO_BUILD_JOBS unless the caller set it; an invalid value fails the run. The per-job budget is measured: rustdoc on epic-control peaks at 1.6 GiB and compiling its library tests at 4.7 GiB. Because the nested job cannot see the runner's limits, the PR Gate and nightly workflows set EPIC_CI_JOBS: "4" explicitly, and the workflows README now records the runner's real limits.

  • Both planes now serve a renewed TLS certificate without a restart (#169, ADR-071, 2026-10-04). Edge and control loaded the public certificate once at boot with with_single_cert and ignored every renewal; the served *.epic-ai.solutions certificate expires 2026-11-23, and on that day every handshake on both planes would have failed with every pod Ready. The certificate is now chosen per handshake by epic_net::serving_cert::ReloadingCertResolver and re-read from the mounted epic-serving-tls Secret every 60 s; a replacement is installed only if it parses, its key matches the leaf, and it is inside its validity window, otherwise the current one keeps serving. Per-plane epic_<plane>_tls_certificate_{expiry_timestamp_seconds,installs_total,reload_rejections_total} and three EpicServingCertificate* alert rules. The edge's [tls] vault_path — a "primary FerraVault fetch" no code ever performed — is removed (fallback_cert_path/fallback_key_path become cert_path/key_path, plus reload_interval_secs); control gains tls_reload_interval_secs. The edge stops watching cert:epic-wildcard, and that rule leaves both planes' FerraVault policies: neither used the private-key grant, and the denied pattern was what took the edge's kv watch down for two weeks (#167). Deploy: ConfigMap and image roll together; re-register the edge policy only after every edge pod runs this build.

  • Ferrum-backed tests and benches run against the Ferrum production runs (#156, 2026-10-04). The harness used to take the first ferrum-server it found in a shared target dir — a stale debug build, or none at all, in which case local runs silently skipped every Ferrum-backed test. ci/ferrum/test-server.env now pins the binary to production's image digest; ci/ferrum/fetch-server.sh extracts it by digest and verifies its SHA-256; a Stage 1 gate (ci/ferrum/verify.py, mutation-tested) fails the build if the pin and ferrum-values.yaml drift; ci.sh Stage 4 and the Stage 8 bench runner use it; and the meter benches refuse any unpinned server (require_pinned_ferrum_test_server). Against it, meter_record_warm is 8.67 µs (was a 541 µs baseline that measured ferrum#2's recv leak) and aggregator_tick_warm 9.55 ms (was 49.7 ms); both re-frozen under the new tag ferrum-9.14.

  • Ferrum-backed tests and benches run against the Ferrum production runs (#156, 2026-10-04). The harness used to take the first ferrum-server it found in a shared target dir — a stale debug build, or none at all, in which case local runs silently skipped every Ferrum-backed test. ci/ferrum/test-server.pin now pins the binary to production's image digest; ci/ferrum/fetch-server.sh extracts it by digest and verifies its SHA-256; a Stage 1 gate (ci/ferrum/verify.py, mutation-tested) fails the build if the pin and ferrum-values.yaml drift; ci.sh Stage 4 and the Stage 8 bench runner use it; and the meter benches refuse any unpinned server (require_pinned_ferrum_test_server). Against it, meter_record_warm is 8.67 µs (was a 541 µs baseline that measured ferrum#2's recv leak) and aggregator_tick_warm 9.55 ms (was 49.7 ms); both re-frozen under the new tag ferrum-9.14.

  • A graceful shutdown could cancel a meter-aggregator tick mid-commit and double-count its batch (2026-10-05). Aggregator::run raced the whole tick() against shutdown in a biased select!, so a shutdown landing between the UPSERT and the ACK dropped the tick: the deltas were in Postgres but the events stayed unacknowledged, and the next start re-added them. tick() is now split into a cancellable pull and an apply that always runs to completion; shutdown still interrupts the blocking pull and the sleep. The SP.3 run() test had started failing on main (10/10 on the post-reboot workstation) because it was catching exactly this; a new gated test holds the UPSERT open while shutdown fires and fails 3/3 on the old code. The module doc's claim that the UPSERT and cursor commit in one transaction was false and is corrected — a crash in that window still over-counts, which needs a MeterAggregateStore contract change and is tracked separately.

  • A FerraVault fetch could cache a key that a concurrent rotation had already replaced (2026-10-05, #182). epic-vault's secret cache was filled as miss → KvGet → insert, with nothing stopping an invalidation that landed while the RPC was in flight. The Watch event of a rotation clears the cache, but a rotation pass already reading the previous key would then insert the old value after the clear, and the next pass would serve it from cache for the rest of the TTL (300 s on the edge). That is the shape of a Ferron bearer-key rotation, which writes three keys back to back. Every invalidation now advances a cache generation, a fetch records it before the RPC, and the insert is skipped if it moved. The bug was unreachable while #182 kept the Watch silent; it is fixed before FerraVault 1.14.0 makes the Watch work.

  • Ferron 0.2.0 pickup: the window check, tool history, and an empty tool_choice: none answer (2026-10-05, epic#138 / epic#140). Ferron fixed all three of EPIC's open FRs on 2026-10-05 (ferron#1–#3) and published ferron-tokenizer / ferron-core 0.2.0; the EPIC side is four commits. (1) Completion budget: Ferron now enforces prompt + max_tokens <= context_window, which exposed that the edge checked an absent max_tokens as 0 while the adapter sent 2048 — such a prompt near the window was refused upstream as a retryable-looking 503 ferron_rejected. The new completion_budget resolves the budget once (absent ⇒ min(DEFAULT_MAX_TOKENS, window − prompt)) and stamps it onto the Ferron-bound request; Ferron's own context_length_exceeded refusal maps to the same 400 context_window_exceeded. (2) Tool history through the model's template: earlier tool_calls and tool_call_id bind structurally; the hand-written Hermes/Qwen <tool_call> fold is deleted. A new golden test matches transformers byte-for-byte on the deployed Qwen2.5 template; request-side tools now bind in HF's nested shape, and ferron-tokenizer turns on serde_json/preserve_order, so schema key order is kept (every JSON hash EPIC computes is key-order independent). (3) Context-window fallback retired: Ferron advertises the window it enforces (live: 16 376 for Qwen2.5-3B-Instruct-FP8, clamped by its KV pool — the ConfigMap's 32 768 had become wrong), so the [ferron.context_window_fallback] bridge is removed; apply the ConfigMap before rolling the image (deny_unknown_fields). (4) tool_choice: none returned "": the adapter treated any request carrying tools as a tool stream and dropped its raw tokens, but Ferron only emits ToolCallContent when tool_choice constrains — present since Sprint 8.1, caught by the live drill's new tool-history turns; translate::uses_tool_content now mirrors Ferron's predicate. Live tool drill 17/17 on all three surfaces. ADR-041 and ADR-054 amended; the three FR docs marked resolved. epic#139 (credentials in Ferron's logs) needed no EPIC code: Ferron's fix is deployed and its keys rotated.

  • Two tests that passed for the wrong reason (2026-09-18). Both found by the SP.3 sign-off gate; neither is a product defect, and that is the point — each was a check being read as evidence of something it did not test.

    The dual-stack listener test took its port from the ephemeral range by binding [::1]:0 and releasing it, so anything on the machine could win that port before the two real listeners claimed it — rare on an idle box, routine under a loaded parallel run. The spawned serve_h2 results were discarded, so EADDRINUSE surfaced as a timeout rather than as itself, and the panic then asserted "both sockets are bound", naming the one explanation that had been ruled out. It also carried no client timeout, so a port won by a socket that accepts and never speaks TLS would hang the test rather than fail it. It now takes another port and retries, distinguishing the two cases by the fact that a serve_h2 whose bind failed has already returned while one that bound is still accepting. ci.sh --full had passed this test an hour earlier; its conditions were never adversarial enough to lose the race.

    The epic-meter aggregator's run() — the loop production starts — had no test at all, because every test drives tick() directly. What that left unproven was the behaviour on the way down: the two biased select! arms that make shutdown win over a tick in flight and over the sleep between ticks. An aggregator that finished its minute before noticing shutdown would pass every other test in the file and hold a rolling deploy for a minute per pod. Both fixes are mutation-tested.

  • A slot certificate whose holder's name spelled PDF syntax could not be verified (2026-09-16). Found by a new property test over the certificate renderer, not by the task that was being worked on. The verifier located the /Info dictionary by searching the file for the first >>, and a printed value can contain one — so an account named "Acme >> Labs" got a certificate that rendered and signed correctly and then failed its own verification as malformed. Searching for trailer or 6 0 obj was steerable the same way: those are keywords a name can also spell. The reader now anchors on line boundaries (certificate text never contains a newline) and steps over string values when looking for a dictionary's end. The shrunk failing case is kept as a regression seed.

  • The SDK conformance gate passes again (2026-09-15). Since Sprint SP.1 the edge refuses API keys whose organisation holds no slot (403 slot_required), and epic_test_utils::seed_inference_org still seeded an organisation without one. Its seven users are the conformance server, the live-edge and live-Ferron drill fixtures and four edge benches, none of which cargo test runs, so the gap surfaced only in ci.sh Stage 4: 17 conformance tests failed with 403 on both SDKs. The helper now grants one slot. OpenAI 15/15 and Anthropic 11/11 pass over plaintext and TLS. On a machine without python3-venv, ci/conformance/run.sh left a venv without pip and then failed with a misleading error on every later run; it now recreates the venv and names the missing package. The live OTel collector test also boots its container before its 30-second deadline, so a first-time image pull no longer fails it.

  • Wide tables no longer push the dashboard sideways on a desktop screen (2026-09-15). /app/admin/audit overflowed a 1024 px window by 189 px, and the access-code console overflowed at 1280. The Phase 7 F23 fix had capped the grid column only on phones. The column is now capped at every width, and every .app-table sits in a .app-table-scroll box, so a wide table scrolls inside its card while narrow tables keep their full width. A source test fails on any table written without the box. The browser overflow gate now also checks 1024 and 1280 px, and was shown to fail with the old stylesheet. A table can opt into wrapping long values instead (app-table--wrap), and columns can be sized with Tailwind utilities on their header cells. Twenty of the 22 pixel baselines were unchanged.

  • Leaving signup's "optional" country blank refused the signup (2026-09-14). The field read "Used for OFAC sanctions screening. Optional but recommended", and RegisterRequest documented a missing country as "not blocked" — but OfacGuard::check returns Missing for an absent code and every OfacError maps to country_blocked, so a blank field answered "EPIC cannot accept signups from your jurisdiction". Closed by ADR-066, which removes the field and screens the connection's country instead (see Changed).

  • A new migration could be silently missing from an already-built binary (2026-09-14). epic_db::MIGRATOR is sqlx::migrate!("./migrations"), which embeds the directory at compile time, and nothing told cargo to rebuild epic-db when a file was added. Adding migration 0052 left one cached epic-db artifact embedding the old set, so a test binary linked against it ran without 0052 and failed on the constraint 0052 removes. crates/epic-db/build.rs now emits cargo:rerun-if-changed=migrations, the fix sqlx documents.

  • Renaming an organisation to whitespace would have been a 500 (2026-09-14). PATCH /api/orgs/{id} checked a new name's length but not whether it was blank after trimming; with migration 0052's orgs_name_not_blank it would reach Postgres as a check violation. It is now a typed 400, and a name with a control character — which the slot certificate refuses, after payment — is refused at every surface that accepts a name.

  • The password-reset email linked to a page that does not exist (2026-09-14). password_reset_request built the link inline as {origin}/auth/password-reset/confirm?token=…; nothing serves that path — the form is /reset-password, and the JSON confirm endpoint is a POST under /api/auth/ — so every reset mail carried a link that 404s. It is the defect already fixed for the verification mail (VERIFY_EMAIL_PATH), which survived here because this model was built at its one call site rather than in epic_postmark::templates::models. It is built there now (RESET_PASSWORD_PATH, models::password_reset), the Postmark field names are unchanged so the template needs no edit, and routes::web::reset_password gains the test that crosses the mail and the router — mutation-tested: restoring the old path fails it. Mail sending is still gated on Postmark account approval, so no member is known to have received a dead link.

  • FerraVault had been refusing every audited RPC for 2 d 19 h while reporting Active (2026-09-14). epic-state/ferravault-0 wedged at 2026-09-11 07:45:12 UTC and stayed 1/1 Ready until the pod was deleted — so member login, signup, TOTP and API-key minting were down that whole window across epic-control and both preview slots, with the site itself serving 200. One audit-sink emit exceeded the hardcoded 200 ms per-sink timeout in ferravault-audit; under fail_closed = true that becomes a failed StateMachine write, openraft treats a state-machine storage error as fatal, and the RaftCore died permanently. With replicas: 1 the daemon's own advice — "restart this node's daemon; survivors keep serving" — had no survivors and no restarter. The filesystem was not full (26 % used) and no I/O error was logged: a transient slow fsync on a disk shared with Postgres, Ferrum and FerraVault's own hourly 8.7 MB snapshot build produced a permanent total outage. Nothing detected it, for three separate reasons, and the first is the one that mattered: Vault.Health.raft_leader_known reads openraft's last-published metrics watch, which keeps naming a leader after the core dies, so ferra status --require-active — the probe added specifically to eject a leaderless voter — exited 0 throughout; epic-observability is running nothing, so no rule was being evaluated at all; and none of the four FerraVault alert rules covers this shape anyway (sealed == 1 is false, and no-leader is false for the same stale-gauge reason). The retry loop then wrote ~2 MB per 11 minutes of identical errors and rotated its own root-cause logs away within hours, so the trigger had to be dated from audit.log's mtime and its last record's ts_ms. Repaired by restarting the daemon; all four control planes re-wired themselves with no roll via their own vault_reconcile task (epic_control_vault_services_wired 1), which is this incident doubling as the live validation Sprint 7.5's self-heal drill was still carrying as unverified. Filed as a FerraVault FR with the one-line upstream fix (RaftMetrics.running_state, which their openraft 0.9.24 already carries) plus asks to retry a sink timeout before failing the apply, expose the timeout in AuditConfig (deny_unknown_fields, three fields — an operator on a shared disk currently cannot widen it), exit on a fatal core, rate-limit the rejection log, and prune snapshots/ (202 files / 1.1 GB, nothing reclaimed since the cluster was initialised).

  • FerraVault has never had an off-site backup, and the CronJob said so for 12 days (2026-09-14). epic-ferravault-backup pinned ferravault:1.10.3-debug, a tag that is not in the registry, so every run since the job was created hit ImagePullBackOff and then DeadlineExceeded — status.lastSuccessfulTime was empty, not stale. The sibling epic-dr-snapshot job that was succeeding nightly covers Postgres only, so the vault holding every KEK, DSN and the JWT signing key had no copy anywhere while the incident above was in progress. Re-pinned to 1.13.0-debug@sha256:88a1e31d — by digest, not by tag, because the job mounts fv-admin-client (FerraVault root via mtls_binding) while epic-require-digest-pinned-images binds only epic-edge and epic-control, so nothing in epic-state requires a digest and a mutable tag on a registry that accepts any push is a path to running arbitrary code with vault root at the next 02:30 run. The version matches the running daemon (1.13.0, sha256:68826b4f) and the unseal reconciler (1.13.0-debug); checked in that image that /busybox/sh resolves and ferra backup still accepts all eleven flags the manifest passes — the release image is the same build minus one busybox layer (21 layers to 22) and has no shell at all, which is what made the original skew easy to miss. The end-to-end run that proved the path (8 749 104 bytes to R2 at last_log_index 183052, sha256 recorded) was made against 1.12.1-debug, the tag the job carried before the daemon moved to 1.13.0.

  • The pixel gate was screenshotting an error page (2026-09-09). tests/visual_regression.rs never set state.store. AppState carries two store fields — with_invite_and_abuse() fills bare_store, while /app/usage and /app/fleet read store, which for_testing() sets to None — so both pages answered 500 and rendered the generic error page. None of the four browser gates checks HTTP status (client.goto() loads an error page as happily as a real one), so for those two pages the pixel baseline was asserting that the error page had not changed, and the overflow, contrast and landmark probes were measuring its layout. Four checks, two pages, zero coverage, reported green. Found by building the dev-server example, which 500'd on exactly those two pages. The misreading had propagated into the project's record of F23: both the KNOWN_HORIZONTAL_OVERFLOW doc comment and the @media (max-width: 47.999rem) note in styles/input.css said app_usage and app_fleet escaped the overflow list because their content happened to fit — they escaped because they were rendering a page that carries neither the usage table nor the two fleet tables. Both corrected, and the contrast gate's comment repointed at /transparency, which really is empty by design. Two baselines need re-recording, not four: /app reads bare_store via require_store and /app/models reads only state.fleet, so both are byte-identical with and without the fix — if either moves on re-record, that is a signal rather than expected drift.

  • The IPv6 ingress VIP was unreachable by construction (2026-08-25, Phase 9, Sprint 9.4). ADR-060 §19 reverses §10/§12 for the LoadBalancer pool only. §12 put the v6 VIP in ULA space because "the delegated prefix rotates on a ~600 s lifetime" — but 600 s is the router advertisement's valid lifetime (how fast a host forgets a prefix it stops hearing), not a rotation interval, and 2401:d002:520a:9d00::/64 has been constant across the workstation, lab-git, the Talos node and the router since the 2026-08-22 survey. Measured off the device, the ISP grants a /64 (ipv6_prefix_len_wan=64); the /56 nvram also reports is the length the router asserts on the LAN, not the length granted. The deeper error would have held even had the premise: ULA bought the v6 pool a stability the v4 pool never had (192.168.50.60-69 is just as much a fact of this LAN) and paid for it with reachability — a ULA VIP cannot be reached from the internet on this substrate, since the router has no NPTv6 and NAT66 would destroy the client source address ADR-060 §4 makes load-bearing by giving the edge an empty trusted-proxy list. §12's "the router's job (a static forward to the VIP)" describes a mechanism IPv6 does not have: the delegated /64 is on-link on br0, so an address inside it is natively routed and there is nothing to forward. The replacement adds no machinery — epic-worker-1 already holds a SLAAC GUA on its macvlan eth1 (ADR-061 §2) and epic-edge-l2-announce already answers there, so Cilium NDP-solicits for the VIP exactly as it ARPs for 192.168.50.60. Pod, service and site ranges stay ULA. The gate moved with the decision rather than being relaxed: ci/network/verify.py now requires the LB block to be globally routable, inside a declared EPIC_LAN_PREFIX_V6, clear of the router's ::1, and unable to express a SLAAC EUI-64 identifier — so a "restore" to ULA fails Stage 1 instead of shipping a VIP that answers on the LAN and nowhere else. A gate gap the change exposed: both LB Services pin their VIPs via io.cilium/lb-ipam-ips, a second uncheckable copy of the address plan that drifted immediately; check_pinned_vips_are_in_pool now asserts every pin falls inside a pool block (mutation-tested both directions), because Cilium silently leaves a Service with no ingress address while its pods stay Ready when a pin matches no pool. T-9.4.6 attached: the AAAA reconciler had 26 tests and no config section, no call site and no manifest — reachable from nothing, the same shape as the epic-cluster enforcer the sprint had already flagged twice. It now has a [boot.ddns] section with a fail-closed validate(), a ServiceAddressSource that reads the address off the epic-edge-lb Service (no address key, and deny_unknown_fields makes adding one a parse error — a constant can be confidently wrong, and a wrong AAAA is a black hole behind green probes), a record name derived from [domain] base, and a narrow RBAC grant (get on one named Service). ci/network 44 → 54 tests, epic-cluster 23 → 32, epic-config 40 → 47. See Sprint 9.4 §7.

  • Test containers leaked on every run — three mechanisms, all needed (2026-08-25, Phase 9). A cargo test -p epic-db that passed all 39 tests still stranded a Postgres container, every time: the shared container lives in a static OnceCell, and Rust never drops statics, so ContainerAsync's Drop — the only removal path testcontainers 0.27 offers, since that version ships no Ryuk reaper — was unreachable on a completely ordinary run. Roughly 450 containers accumulated on the development workstation between 2026-05-29 and 2026-08-25 (all EPIC's; the org.testcontainers.managed-by label and PG_MAJOR env identify them precisely). Fixed in three layers, each covering what the previous cannot: the workspace enables testcontainers' non-default watchdog feature (SIGTERM/SIGINT/SIGQUIT, then re-raises the default handler so Ctrl-C still behaves like Ctrl-C); crates/epic-db/tests/common/mod.rs gains a #[dtor] reaper for the normal-exit path, because libtest exposes no end-of-binary teardown hook and the platform destructor section is the only place left to stand; and ci/docker-hygiene/sweep.py reclaims the residue of SIGKILL/OOM, which is uncatchable by construction — with an age threshold so it can never remove a concurrent run's containers. Verified against a probe that stays alive long enough to actually be signalled, since the first attempt "passed" only because the suite finished before the signal landed: SIGINT → 0 leaked, SIGTERM → 0 leaked, SIGKILL → 1 leaked (as designed, and the reason the sweep is not decorative). The sweep's selection logic has 12 unit tests and was live-verified in both directions; both run in ci.sh Stage 1.

  • Sprint 7.7 — edge HTTP/1.1 SDK compatibility (2026-07-05, Phase 7). Finding F22: the edge TCP:443 listener advertised ALPN ["h2"] only (ADR-011), so HTTP/1.1-default OpenAI-compatible clients (the OpenAI Node SDK / undici that drives Cline/Roo/Cursor; openai-python on httpx defaults; curl --http1.1) failed the TLS handshake with no_application_protocol and reported a generic Connection error — breaking the "SDKs work unmodified" thesis. Masked because every SDK test (openai/anthropic conformance + the Phase 8.3 "real Cline" drill) ran against the plaintext loopback fixture (h1.1 cleartext), never the h2-only TLS listener. Fix: the h2 listener now advertises ["h2", "http/1.1"] (h2 preferred; boot::with_alpns); hyper's auto builder already serves both, so nothing above the listener changes (router, pipeline, auth, metering, zero-per-token-alloc hot path identical). ADR-011 revised in place; DESIGN.md §8.2 reconciled. Regression guard h2_listener_serves_http1_clients_over_tls binds serve_h2 + drives a real HTTP/1.1-over-TLS request (fails against the prior ["h2"]). Root cause closed: the openai/anthropic SDK conformance gate (ci.sh Stage 4) now runs each surface over both plaintext and TLS — conformance_server gained an EPIC_CONFORMANCE_TLS=1 mode serving over the production serve_h2 + ALPN with a self-signed cert the harness trusts — so SDK conformance is exercised over the REAL transport, not only the plaintext fixture that masked F22 (negative-tested: an h2-only ALPN turns the TLS pass red; green = openai 15/15 + anthropic 11/11 over TLS). Edge image rebuilt + rolled; verified end-to-end with Cline against https://api.epichub.ai/v1. See Sprint 7.7.

  • Sprint 7.6 — FerraVault cluster-resilience adoption (2026-07-04, Phase 7). Finding F20: a recurring multi-minute secret-plane outage after any FerraVault pod restart — a sealed voter has no Raft instance (Raft is built only after Active), so co-located Shamir voters that all reseal on restart are absent from quorum, the lone Active voter storms its term (> 65 000, no pre-vote on openraft 0.9.24), and audited ops fail-closed audit unavailable (JWT signing + api-key mint down cluster-wide); recovery depended on a flawless manual per-node unseal ceremony. Root cause is upstream/structural; EPIC filed a four-part FR and FerraVault delivered all four the same day in daemon v1.11.0 / chart 0.10.0 (their ADR-046). EPIC adopts the chart-native path (ADR-056) rather than build the interim reconciler the FR §4 anticipated: ferravault-values.yaml re-pinned to v1.11.0 (digest sha256:642403c7…) with unsealReconciler (unattended auto-unseal from a standing per-node shares Secret — dev-only custody tradeoff, prod stays assisted-kms), readiness.mode: active (ejects sealed/leaderless voters from the client LB), config.raft.profile: single-host (wider election timeouts kill the storm on the current pin — pre-vote is not the fix), + the shipped FerraVaultSealedTooLong alert; ferravault-jobs.yaml NetworkPolicy extended for the unseal-reconciler pods; unseal runbook §6 rewritten (reconciler is the steady state, manual unseal is break-glass). Deployed live 2026-07-04 (T-7.6.10): epic-control rolled to the F19+F21 image (sha256:f46d491c…, commit 9133d18); FerraVault upgraded to v1.11.0 / chart 0.10.0 (helm upgrade --force-conflicts, a legacy kubectl-set field-manager owned .image); the reconciler auto-unseals resealed pods; the acute F20 outage is resolved — the cluster serves on a stable leader with a correct 3-voter membership and a term that stopped storming, epic_control_vault_services_wired=1, zero audit unavailable/signing_unavailable. A share-file header-line bug (the standing shares Secret must hold fv1s- lines only, else the reconciler feeds a header as "share 1" → opaque daemon Internal error) was caught + fixed during the deploy. Residual — since RESOLVED: ferravault-0, poisoned with a raft term higher than the leader's by the pre-v1.11.0 storm, would not rejoin (PVC-persistent term; delete+re-unseal does not clear it); ferra cluster rebuild-follower cleared the poison but the leader would not auto-re-onboard the reset voter → filed a follow-up FerraVault FR. FerraVault answered same-day with v1.12.0 / chart 0.11.0 (their ADR-047) — root-caused to four stacked defects, the decisive one a raft-plane transport ceiling (tonic's 4 MiB default vs ~3.6×-inflated serde_json snapshot chunks → no state.db past ~1.1 MiB could catch up a blank node). EPIC adopted it (values re-pinned to v1.12.0 @sha256:06cef248… + [cluster_raft] max_message_bytes=32 MiB + install_snapshot_timeout_ms=120 s + a global logFilter floor; helm upgrade); fv-0 auto-rejoined on the roll (voters=3, leader_known=1, term stable, 0 restarts) → full 3/3 quorum restored, F20 fully closed. Also resolved finding F21 (T-7.6.12): the recovery-path verification found that per-record audit signing (ADR-031), diagnostic capture (ADR-008), and review sampling (ADR-037) wire from the raw vault_client at boot and were not re-swapped by the Sprint 7.5 reconciler (the audit sink is also moved by value into the vote/kill-switch/billing services). Fixed with a unifying epic_control::hot_swap_vault::HotSwapVaultClient — one ArcSwapOption<VaultClient> implementing CryptoSigner+KeyWrapper+KekProvisioner, built once at boot, threaded into all five consumers, and filled by the F19 reconciler so every one heals atomically; the audit emit path now degrades to unsigned on a signer Unavailable (chain intact) instead of failing the audited write. Proven by hot_swap_vault units + a PG-backed signer_unavailable_degrades_to_unsigned_then_heals. See Sprint 7.6.

  • Sprint 7.5 — control-plane vault-service self-healing (2026-07-04, Phase 7). Finding F19: a node reboot re-sealed FerraVault under a running epic-control, and because AppState::wire dialled the vault once at boot, every vault-derived auth service (jwt/totp/turnstile/vault/vault_writer) was disabled for the process lifetime — every login returned 401 JwtService is not wired while the pod reported Ready (the /ready gate is deliberately adapter-independent, from the 2026-06-16 correlated-outage fix), and unsealing the vault did not self-heal it (a manual rollout restart was required — the edge, which hard-crashes on a sealed vault, self-heals via crash-loop; control did not). The five services now live behind arc_swap::ArcSwapOption slots (epic_control::vault_services::VaultServices) with AppState accessor methods, and a background reconciler (epic_control::tasks::vault_reconcile) re-runs the identical wire_vault_services path under bounded backoff and hot-swaps them in when FerraVault returns — no crash, no correlated readiness flap, no manual restart. Spawned by the composition root only when a vault is configured but jwt didn't wire at boot. New epic_control_vault_services_wired gauge (0 degraded / 1 wired) + /ready diagnostic field + EpicControlVaultServicesUnwired page alert make the degraded window loud instead of silent behind a green probe. 429 lib tests (incl. the sealed→unsealed reconciler transition) + login e2e (12/12) + sprint-1.2 (8/8) green; clippy + doc gate clean. Live sealed→unsealed cluster drill pending (T-7.5.5). See Sprint 7.5 sign-off.

  • Sprint 7.1 — RLS tenant-scoping remediation, full sweep (2026-06-22, Phase 7). Finding F4 closed in full: every org-scoped FORCE-RLS data-access method that ran on the raw connection pool with no active-org GUC (the defect superuser tests masked) is now scoped on a pinned connection (begin_for_org), via a SECURITY DEFINER wrapper, or under SET LOCAL ROLE epic_admin. Migration 0050 adds the api_keys by-id/by-public-id/by-prefix/by-user + touch/disable/expiry/revoke wrappers and the org_member_invites consume/revoke wrappers (the by-id surface is reached from the edge epic-edge-ro role, control, and cross-org abuse workers, so a uniform SECURITY DEFINER path is the only correct fit — 0028/0049 doctrine). org_members mutators + transfer_org_owner, vote.rs ballots, and invites.rs insert/list converted to begin_for_org; the cross-org IRV list_ballots + GDPR hard_delete_user + the update_org_seat_count org-member count now escalate correctly (the last two were extra F4 bugs the hardened harness caught, not the originally-named methods). New integration_pg_rls_tenant_scoping.rs exercises the whole surface under a production-shaped NOSUPERUSER NOBYPASSRLS role (member of epic_admin only via SET LOCAL ROLE); 25/25 non-superuser tests green, no trait-signature change. See Sprint 7.1 sign-off.

  • Sprint 7.2 — multi-model edge enablement + F9 incremental detokenisation (2026-06-22, Phase 7). Playground multi-model walkthrough cluster F9–F13: the edge now adopts Ferron v1.0.1's IncrementalDecoder per response stream (epic-ferron-client, ADR-052) so SentencePiece (Llama/Mistral) output renders with correct word-boundary spaces (F9/F13); shipped deploy/tokenizers/TinyLlama-1.1B-Chat-v1.0/ byte-identical to Ferron's source with the hash-alignment gate now table-driven over every model (F12); relaxed the dev classifier call_timeout_ms 75→1500 for the cold/GPU-time-sliced encoder (F11); F10 (multi-model serve) resolved upstream in Ferron Phase 5 with zero EPIC change. See Sprint 7.2 sign-off.

  • Sprint 7.4 — API key management surface remediation (2026-07-04, Phase 7). Member-walkthrough finding-cluster F16/F17/F18 on /app/keys: added the missing "Create key" CTA (the create flow shipped in T-1.4.8 but the list page was never wired to it) (F16); ephemeral playground keys (ADR-046) that share the api_keys table no longer pollute the human management surfaces — new Store::list_manageable_api_keys_for_org (DB-level key_type <> 'ephemeral') on the dashboard + JSON GET /api/keys, with list_api_keys_for_org left intact for the kill-switch cascade + playground cap counter (F17); and expired ephemeral keys are now reaped hourly (Store::purge_expired_ephemeral_keys cross-tenant epic_admin DELETE + tasks::ephemeral_key_reaper, mirroring the review-sample sweep) so the table can't grow unbounded (F18). Proven under the production-shaped NOSUPERUSER NOBYPASSRLS role. See Sprint 7.4 sign-off.

  • Sprint 6.3 post-sign-off — ci.sh --full gate failures (2026-06-18, Phase 6). A full ci.sh --full run (the 6.5 gate, pulled forward) surfaced three failures the per-crate clippy/cargo test checks miss; all fixed at the source, no deferral, no visual page dropped. (1) doc — a workspace cargo doc -D warnings cascade: 2 epic-web playground errors (private + cross-crate intra-doc links) masked the rest, which surfaced once fixed — epic-test-utils (det_public_uuid), epic-config/epic-control::state (host[:port] → broken :port link), and latent Sprint-6.2 admin-console private links (require_operator_session, emit_admin_audit_with_actor, drive_*_transition_core); all de-linked, cargo doc --workspace clean across 24 crates. (2) bans — Sprint 6.2's grant-operator dev-dependency on epic-store from epic-migrate was missing from epic-store's deny.toml wrappers (cargo-deny checks dev deps); added. (3) visual — two independent root causes. Engine: ci/visual/run.sh preferred Chrome, but Chrome --headless=new yields non-reproducible screenshot dimensions on an active desktop (viewport jitters run-to-run; centred layout ⇒ no pixel-exact match) — switched the gate to prefer Firefox (dimensionally stable; respects EPIC_WEBDRIVER) + auto-pick a free port. Content: the 8 Sprint-6.2 admin pages + app_playground had no baselines, and app_admin_orgs/app_admin_audit flaked on random UUIDv7 public ids the pages render — added an opt-in MemStore::with_deterministic_public_ids() (default unchanged). Re-recorded all 16 baselines with Firefox → compare-only passes 0.00000 on every page. Stage 8 bench gate green (0 regressed vs phase-4-final). See Sprint 6.3 §7 "Post-sign-off CI remediation".

Added

  • Sprint 7.3 — Quantized-model serving (catalogue adoption) (2026-06-30, Phase 7). EPIC-side adoption of Ferron FP8 serving — EPIC surfaces the precision Ferron advertises; it never quantizes weights itself. ADR-053 Accepted — epic-protocol::native::ModelInfo gains an optional quantization: Quantization { None | Fp8 | Fp4 | Other(String) }, #[serde(default)] so an older Ferron omitting the field parses unchanged and an unrecognised future precision token degrades to Other rather than failing the /v1/capabilities parse (two planes deploy on independent cadences). Parsed by epic-ferron-client from Ferron's capabilities, carried two-plane through the ADR-048 Ferrum publish/read, and rendered as a precision badge + "Precision" column on /app/models. Quantization is a model-load property and does not touch the tokenizer, so a quantized build shares the base model's ADR-035 tokenize_build_hash (verified live: Qwen2.5-3B-Instruct-FP8 == Qwen2.5-0.5B-Instruct). deploy/tokenizers/Qwen2.5-3B-Instruct-FP8/ shipped + added to the production-path hash-alignment gate; edge configmap context_window_fallback extended; edge rebuilt + rolled (6/6, clean boot, zero ModelDirMissing). Live playground stream against Qwen2.5-3B-Instruct-FP8 confirmed (T-7.3.7a). Files the Ferron FP8 FR + the FP8 checkpoint-format taxonomy. See Sprint 7.3 §7.

  • Sprint 6.4 — Member dashboard depth + fleet/model status (2026-06-18, Phase 6). Completes the member-facing product + the §1 "see fleet state" thesis. ADR-048 Accepted — the edge republishes its polled Capabilities + LoadSnapshot to the shared Ferrum substrate on the existing 5 s poller tick (fire-and-forget, off the per-token hot path — all five ADR-026 invariants intact), and the control plane reads them back via the Cache boundary; this honours the two-plane substrate invariant (CLAUDE.md §4.1 — control never dials Ferron, holds no Ferron credentials) and was chosen over control-dials-Ferron + edge→control-RPC. Cross-plane contract in epic_protocol::native::fleet (epic:fleet:{capabilities,load}:v1 + 30 s TTL); epic_edge::fleet_publish::FleetPublisher + [ferrum] fleet_publish_ttl_secs; epic_control::fleet::FleetSnapshotReader. New surfaces: /app/models (live catalogue + typed-501 roadmap), /app/fleet (capacity + the org's fair-share standing rendered as the exact edge annotation fair_weight = seat_count, R6.7 — test-pinned to CachedModelRouter), and a deepened /app/me (editable profile, three optional-email toggles, security/TOTP surfacing, PRG notice). Migration 0048 user_notification_preferences (FORCE-RLS on app.user_id); new epic-trait::NotificationPreferencesStore (+ Pg/Mem adapters). Dashboard home now composes the playground CTA (6.3) + a 30-day usage summary (6.1) + the live fleet snapshot, each degrading gracefully. Tests: fair-share correctness (display == edge annotation), /app/{models,fleet} + /app/me settings handler integration, RLS isolation + Pg round-trip (real PG), schema-drift guard; visual set gains app_home/app_models/app_fleet/app_me with a deterministic seeded fleet snapshot. See Sprint 6.4 §7.

  • Sprint 6.3 — In-browser inference playground + browser↔edge auth bridge (2026-06-18, Phase 6). /app/playground: a logged-in member streams a completion live in the browser against the real edge. ADR-046 Accepted — the control plane mints a short-lived, org-scoped, single-purpose ephemeral API key and the browser streams directly against the edge with it (no control-plane inference proxy; two-plane blast-radius isolation preserved). New KeyType::Ephemeral (wire token ephemeral) is a class on the existing api_keys model, not a parallel system — same BLAKE3-keyed-at-rest hash, UUIDv7 id, FORCE-RLS; migration 0047 extends the key_type CHECK and pins ephemeral⇒expires_at + ephemeral⇒scopes={inference} at the storage layer (ADR-006 reconciled). Mint endpoint POST /api/playground/session (epic-control): session-gated (anon → 403), [Inference]-only, TTL'd (default 900 s), revocable, audit-stamped (api_key.mint_ephemeral), per-org active-key cap (→ 429); [control] edge_public_origin config. The edge consumes via the existing ApiKeyService::verify (one-variant widening of the inference type gate, Sk → Sk | Ephemeral — no new hot-path branch); a no-credentials browser-SSE CorsLayer (closed epichub.ai origin set, no Access-Control-Allow-Credentials — no cookie surface on the data plane). epic-web SSR shell + pinned dependency-free SSE client (fetch + ReadableStream, native framing, cancel via AbortController, re-mint on expiry). ADR-026 zero-per-token-alloc re-asserted with a playground-key stream (the hot path is key-type-agnostic). Tests: mint authz/scope/TTL/cap/audit, edge verify + expired-reject + CORS allow/deny, PG-backed ephemeral RLS isolation + migration-0047 CHECK enforcement, page render; /app/playground registered in the visual-regression set. See Sprint 6.3 §7.

  • Sprint 6.2 — Admin dashboard / operator console (2026-06-17, Phase 6). The complete /api/admin/* JSON surface rendered as a session-authenticated SSR operator console under /app/admin. ADR-049 Accepted — platform-operator identity as users.is_epic_operator (migration 0045), settable only out-of-band via new epic-migrate grant-operator/revoke-operator; require_operator_session gates the whole subtree at parity with the JSON surface's KeyScope::Admin (non-operator session → 403, anon → /login), with the JSON require_admin_key surface left unchanged. Nine SSR sections (overview, org mgmt, abuse triage + opt-out review, kill switches, diagnostic-capture + review-sample viewers with server-side Vault.Unwrap decrypt + audit-before-plaintext, meter health, audit viewer with chain-verify status, vote admin). No reimplementation — actor-parameterised cores (drive_opt_out_transition_core/drive_report_transition_core + AdminActor) drive the same Phase 4 policy from both planes; operator actions audit-stamp the operator user (emit_admin_audit_with_actor). Destructive-action safety: shared GET-confirm → POST-execute flow (render_confirm) with required reason + per-record-signed audit. Admin chrome + CSS (deterministic site.css rebuild); /app/admin/* wired into the visual-regression baselines. Tests: table-driven authz negatives over every route + action-parity (state change + operator-stamped audit row) + 25 render tests. T-6.2.3 edge-config kill-switch firing — delivered in full via the 2026-06-18 addendum below (ADR-050); the earlier read-only-panel / Phase-7-deferral disposition is superseded. See Sprint 6.2 §7.

  • Sprint 6.2 addendum — Runtime edge kill-switch control (2026-06-18, Phase 6; ADR-050). Closed the flagged T-6.2.3 edge-config firing half by building the missing control→edge seam properly instead of deferring (operator-directed "do the full works"). ADR-050 Accepted — a durable edge_kill_switch_rules table (migration 0046, operator-global config, non-RLS) is the single source of truth for the five edge-config scopes (IP/CIDR, country, UA-regex, model grounding, org deny); the operator console is the only write surface (operator-gated + audited), and both planes reconcile from it: the edge polls it on a background interval ([killswitch] db_poll_interval_secs, default 5 s) and hot-reloads the merged ArcSwap snapshot, while IP/country mutations also drive the Sprint 5.3 KillSwitchEnforcer (Cilium/Cloudflare) best-effort. New epic-trait::EdgeRuleStore (+ EdgeRuleScope::canonicalize validating against the same parsers the edge enforces — no drift) / PgEdgeRuleStore / MemEdgeRuleStore; EdgeRuleId/EdgeRulePublicId. /app/admin/kill-switch now CRUDs each scope (add with required reason, confirm-page remove), operator-stamped per-record-signed audit (edge_kill_switch_rule.add/.remove). Two-plane separation preserved (control writes, edge reads — no inbound edge mutation surface); hot path unchanged (consult ≈ 105 ns; merge+reload ≈ 464 µs every 5 s, off-path). Tests: trait/mem/edge units, control SSR CRUD integration (incl. new routes in the every-route authz negative table), PgEdgeRuleStore DB roundtrip + grant-operator DB test (the latter also fixed a latent CITEXT case-insensitive-match bug → $2::citext), schema_invariants (table registered) + rls green. See Sprint 6.2 §7.

  • Sprint 6.1 — Usage statistics + public transparency (2026-06-17, Phase 6 opens). Define-then-render over the live epic-meter aggregates: ADR-045 (per-org-private data contract) + ADR-047 (public community-aggregate scope, small-n suppression k=5 in SQL) Accepted. Migration 0044 extends the meter rollups additively (filtered_count, sum_ttft_ms/sum_total_ms/latency_sample_count; retired the dead avg_* columns). New epic-trait::UsageAnalyticsStore (PgStore): RLS-org-scoped per-org read + admin cross-org community read with suppression in-query. SSR: member /app/usage (volume / model mix / experienced latency / seat-derived fair-share + server-rendered inline-SVG chart) and public /transparency (community totals, suppressed buckets hidden). Grafana usage dashboard (Prometheus-sourced; refusals at category granularity per ADR-047 §2). DB tests prove RLS isolation + the suppression boundary against real Postgres; visual-regression gained an authenticated-screenshot path (/app/usage + /transparency baselines, 0.00000 delta). Analytics, never billing — zero epic-billing coupling. ADR-number reconciliation: transparency took ADR-047 (ADR-018 was consumed by Sprint 5.2 PKI). See Sprint 6.1 §7.

  • Sprint 5.6 — Ferrum self-healing-client retirement (2026-06-17, post-phase-5-final addendum; does not reopen the tag). Retired EPIC's Sprint 5.5 client-side reseed wrapper onto the upstream fix (ADR-044 / Ferrum ADR-126). Both planes dial FerrumClient::connect_quic_dns and let the AsyncClusterClient own total-topology-loss recovery; ClusterBackend collapsed from Arc<ArcSwap<…>> to a direct Arc<AsyncClusterClient> (deleted reseed_healer/ReseedTarget/connect_quic_self_healing + the arc-swap dep, −307 lines, hot path drops client.load()). Gated on the T-5.6.2 live drill (ci/drills/phase-5-ferrum-reseed/run.sh — delete-all-Ferrum → total loss): PASS — all 6 edge pods epic_ferrum_reseed_total=1, 0 restarts, Ferrum re-formed 3/3. T-5.6.4 surfaced the four epic_ferrum_* reseed counters on both planes' /metrics + a PrometheusRule (EpicFerrumReseedFailing page / EpicFerrumReseedChurn ticket) + a Grafana dashboard. ADR-044 Accepted; the FR sprint-9.8-final tag claim corrected (reseed work is on Ferrum main/ab4cb2dc, untagged). See Sprint 5.6 §7.

  • Sprint 5.6 (T-5.6.6) — boot-secret transient-seal retry (2026-06-17). A single transient FerraVault vault is sealed at boot no longer crash-loops the edge fleet (or silently disables control's TOTP/Turnstile/Postmark): new epic_store::retry::{retry_with_backoff_if, fetch_secret_resilient} + epic_trait::KeyVaultError::is_transient() ride out a transient seal (45 s default, EPIC_VAULT_BOOTSTRAP_BUDGET_SECS) and fail fast on permanent NotFound/AccessDenied. Incident-driven by the T-5.6.2 FerraVault seal (and the Ferrum ADR-127 QUIC-over-VXLAN MTU fix that cleared the parallel ~400 s bootstrap). See Sprint 5.6 §7.

  • Sprint 5.5 — Phase 5 sign-off (gating) + edge↔Ferrum self-heal resilience fix (2026-06-16). Gated Phase 5 against the dev cluster: T-5.5.1 originally-planned-vs-delivered (48 items across 5.1–5.4, 47 ✅ + 1 four-field-deferred cosign half, operator-approved → Phase 7); three audits PASS (quality / security extended to the cluster attack surface / architecture); the three §17 exit-criteria drills green as gates — cert-rotation zero-drop, chaos 6/6 (kill-one-of-each; FORCE-RLS verified on the sync-promoted primary; fail-closed JWT-503 held), backup-restore layer-1 (R2 WAL recovery, FORCE-RLS + NOBYPASSRLS intact); first dev-cluster §16.2 SLO baseline (edge auth p99 ≈ 1.0 ms, ~5× under). Operator-directed in-phase resilience fix: a Ferrum StatefulSet roll previously wedged the running edge/control until pod restart (the cluster-client did not re-resolve the headless topology on total cached-IP loss); epic-store::FerrumClient now self-heals — ArcSwap-held AsyncClusterClient + a background reseed healer that re-resolves DNS (tokio::net::lookup_host) and atomically swaps a fresh client on total loss, no restart. Validated live (full Ferrum IP rotation → all 6 edge pods reseeded in-process, 0 restarts). Reciprocal upstream Ferrum FR filed. Two gate-caught defect fixes (no deferral): the cancel-budget integration test's regression sentinel moved p99→p50 (the tail was scheduler noise, not the cancel path; no SLO loosened), and the chaos harness recovery target PRE_TOTAL→PRE_READY (a never-Ready backup CronJob pod shared the ferravault label). Tag phase-5-final (unsigned annotated). See Sprint 5.5 §7.

  • Sprint 5.3 🚧 — close: country kill-switch drill-proven LIVE + control restored 3/3 + sign-off sweep (2026-06-15). The T-5.3.9 country adapter applied + read-back + cleared a real ip.geoip.country block on the live epichub.ai zone (EPIC_CF_LIVE drill, 2 audit rows); Cloudflare DNS + country-firewall + §9.7 rate-limit rulesets provisioned via the scoped token (managed-WAF / origin-port / SSL-mode residuals operator/plan-gated, documented in cloudflare-zone.md §7). The edge's Ferrum bootstrap retry was shared into epic_store::retry + wired into epic-control (last single-attempt dial removed) with the headless-ferrum seed; control restored to replicas: 3 (HPA min 3 + PDB), no crash-loop. Five ADR-026 edge invariants re-asserted. The workspace sign-off sweep caught + fixed Sprint-5.x drift the scoped checks missed: a cargo deny two-plane bans gap (kube←epic-cluster, epic-auth/epic-vault←epic-migrate, epic-auth←epic-edge — all legitimate, allowlisted with rationale) and 6 broken cargo doc intra-doc links. See Sprint 5.3 §7.

  • Sprint 5.3 🚧 — Ferrum 9.4 (ADR-125 QUIC keep-alive) + both EPIC images rebuilt & deployed → watch idle-drop RESOLVED live; Turnstile + Postmark ACTIVE (2026-06-14). Rolled Ferrum to 9.4 (82efe175fb75, image-only — the client-side keep-alive is the fix), rebuilt + deployed both EPIC images from clean 8d588841: edge (sha256:f840567a…, 9.4 client keep_alive_interval=15s + watch-backoff fix) and control (sha256:002b4350…, Turnstile site key + Postmark token). Verified live: edge watch holds a single stable connection — watch dropped=0 over 6+ min (was ~12 at the 30 s cadence); the /signup page renders the real Turnstile widget (data-sitekey + CF script); /ready=200 with the Postmark token loaded. Added the Postmark sender DNS (DKIM TXT + Return-Path CNAME) to the epichub.ai Cloudflare zone — both resolve live (Postmark domain-verify pending operator click). Idle-timeout FR closed. Follow-up filed: edge crashes-then-retries the Ferrum bootstrap dial under simultaneous-restart contention (should retry-with-backoff; gates restoring edge replicas 6). See Sprint 5.3 §7.

  • Sprint 5.3 🚧 — Ferrum sprint-9.3 (ADR-124 cross-shard watch fix) deployed → 0x0306 watch bug RESOLVED (verified live); Turnstile site key wired (2026-06-14). Ferrum shipped the cross-shard WATCH.* connection-ID fix as sprint-9.3 (forward dispatch now propagates ConnectionId); EPIC rolled the StatefulSet sprint-9.2 → ba6f7ff94fd7 (digest-pinned, one pod at a time) and verified live: 0x0306 is 0 across all edge pods (was steady-state), the keyspace watch registers regardless of SO_REUSEPORT shard landing. FR RESOLVED. One distinct low-severity follow-up filed: the idle watch connection idle-times-out at ~30 s because the Ferrum async client sets no QUIC keep_alive_interval and exposes no knob — EPIC cannot fix it client-side (FR); the watch stays functional via reconnect-catch-up + re-sync. Also seeded epic/postmark-token + epic/turnstile-secret (fv-admin ceremony, --from-file) and wired the Turnstile site key from [boot.control] turnstile_site_key (was a hardcoded None Phase-6 stub) — the public half coupled to the verifier secret, locked in configmap_shape.rs; activation needs a fresh control image. See Sprint 5.3 §7.

  • Sprint 5.3 🚧 — Ferrum sprint-9.2 (ADR-110 §Amendment-1, 0-RTT correction) deployed → QUIC churn FR resolved (2026-06-12). Ferrum shipped the fix for EPIC's QUIC handshake-instability FR as sprint-9.2 — rustls 0.23 silently downgrades the Sprint 7.4 stateless ticketer per RFC 8446 §8.1, so the ferrum/1 data-plane listeners emitted early_data with stateless resumption is not allowed and churned connections; 9.2 switches to stateful resumption (FerrumSessionStorage). EPIC verified the fix + chart-unchanged pairing in source, rolled the StatefulSet sprint-9.1 → sprint-9.2 (digest-pinned, one pod at a time, no unseal, quorum held), and verified: early-data warnings 0 (was 32/60s), control un-graceful drops 0 (was ~12/60s), and a fresh epic-control bootstrap completes clean (restarts=0) under full edge churn (the original crash-loop symptom). FR RESOLVED; one separate, non-blocking edge keyspace-watch issue (0x0306) unmasked + tracked as a follow-up. See Sprint 5.3 §7.

  • Sprint 5.3 🚧 — FerraVault v1.10.1 (ADR-045 ring coherence) deployed → JWT now LIVE (2026-06-12). FerraVault shipped the fix for EPIC's signing-key-ring HA FR as v1.10.1 / ADR-045 "cross-replica ring coherence" (the in-memory signing_key_ring is refreshed from the Raft apply path on every node, so a signing key created on a running cluster lands in every voter's ring, not just the leader's). EPIC rolled the lab cluster 1.10.0 → 1.10.1 (digest-pinned, one pod at a time, each node re-unsealed with dev Shamir shares to hold 2/3 quorum) and verified end-to-end: the JWT key now reads on all three nodes (was leader-only), and epic-control re-booted with JwtService wired (the prior JwtService disabled warning gone). JWT + TOTP both live; the FR is marked RESOLVED. Only the two operator-credential gates remain (Postmark + Turnstile). See Sprint 5.3 §7.

  • Sprint 5.3 🚧 — epic-control deployed Ready+serving; control auth-flow FerraVault material provisioned (2026-06-12). Brought epic-control up on the lab cluster (1/1 Ready, SSR site serving) by fixing a chain of never-deployed Sprint 1.6 control defects (configmap shape, mTLS-only FerraVault posture, HTTPS probes, and four Ferrum-dial bugs — the showstopper being a missing ferrum/1 ALPN). Ran the fv-admin ceremony to provision the auth-flow material: TOTP encryption key seeded → TotpService wired LIVE; the JWT signing key epic-jwt-signing-key-v1 (ed25519/pure) created + the epic-control FerraVault policy gained the three ADR-030 signing-key grants (sign/verify/watch on the exact per-op sub-paths) + the rotation watch pattern (guarded by configmap_shape.rs). JWT activation is blocked on a FerraVault HA defect — signing keys created on a running cluster land only in the leader's in-memory ring, so load-balanced reads hit a follower and get signing_key_not_found (FR filed; JwtService fails closed). Two operator-credential gates remain, each with a handoff runbook: Postmark (postmark-provisioning.md) + Turnstile (cloudflare-zone.md §4). See Sprint 5.3 §7.

  • Sprint 5.3 🚧 — REAL streamed-token inference live through the public edge (2026-06-12). Closed the full inference chain end-to-end against the deployed stack + workstation Ferron: capability-snapshot reconciliation (ADR-041) + per-model context_window fallback; an entitlement-read RLS fix (dropped a latent epic_admin BYPASSRLS escalation on the public data plane); real Qwen2.5-0.5B tokenizer assets baked into the edge image with a CI hash-alignment gate (ADR-042); and the Ferron native-path presentation contract (ADR-043) — the edge now renders the ChatML prompt, forwards it as a single pre-formatted message, and detokenises Ferron's empty-text token ids edge-side. Live: POST /v1/chat/completions (stream) → Paris, byte-identical to Ferron's own shim. See Sprint 5.3 §7.

  • Sprint 5.3 🚧 — application planes + kill-switch enforcement + image-admission gate + Cloudflare/Ferron ADRs (2026-06-11). The two headline Rust deliverables landed at the Phase 0–4 bar: SIGTERM graceful-drain ordering on both planes (new edge readiness module + /healthz//ready, control draining flag — readiness-off before connection-close; tests green) and the kill-switch enforcement layer (epic-trait::KillSwitchEnforcer + new epic-cluster adapter: Cilium IP/CIDR reconciler + Cloudflare country client + retry reconciler + RBAC — the live IP-kill drill applied + cleared a real CiliumNetworkPolicy on the lab, VALID=True; epic-abuse stays pure). Plus the FR-N1 reciprocal (Ferron Cancelled terminal → FinishReason::Cancelled); the corrected edge ConfigMap (stale [boot.*] → real EdgeConfig shape + [ferron] dial, shape-guarded by a loader test) + MockFerron deployment-test ConfigMap; edge-worker affinity + the role: client Ferrum label; the workstation Ferron egress ipBlock /32; the digest-pin image-admission ValidatingAdmissionPolicy (live + proven on the lab) + the cosign-verify Kyverno policy (operator-infra-gated); the Sprint 5.1 VIP pool applied; ADR-015 + ADR-039 Accepted + the cloudflare-zone.md operator runbook; DESIGN.md §11.1/§13.4/§11.4 reconciled. Operator gates remaining: the live Cloudflare zone apply (T-5.3.7), the registry+cosign-key for the Stage 7 build pipeline, and the full-path smoke (deployed stack + workstation Ferron). See Sprint 5.3 §7.

  • Sprint 5.2 ✅ COMPLETE 2026-06-10 — stateful substrate closed: all 12 §2 tasks delivered, zero four-field deferrals; §6 closed line-by-line against the live cluster; ci.sh Stages 0–6 all green (1,846 workspace tests + 356 integration-stage tests, 0 failed; clippy 0 warnings; audit 0 vulns; conformance + fuzz + load smoke green). The retire ceremony ran (orphan issuance 4 retired, audited; WARN + gauge noise gone); the full FerraVault RPC surface re-proven under the final ["mtls"] posture; cluster-pointed schema_invariants 13/13 + rls 23/23. Two workstation-tooling CI fixes (sqlx-cli 0.9 postgres feature; ferrum-server restored from the published sprint-9.1 image). See Sprint 5.2 §6/§7.

  • Sprint 5.2 🚧 — spec-retire FR RESOLVED upstream same-day (FerraCert PV1.6/ADR-028) + consumed; fv-admin ceremony image re-pinned 1.10.0-debug; chart 0.9.1 assessed → deliberately deferred to the next seal window (2026-06-10). FerraCert revision c8f306f (digest b1bf417e…) live on talos-r740 — once-per-boot WARN demotion verified at tick 1; the spec retire epichub-rehearsal-pv14 ceremony is staged pending operator go-ahead (cert-flow.md §6). See Sprint 5.2 §7.

  • Sprint 5.2 🚧 — hygiene pass: epic-secret-sync gains cert:<spec>#fullchain (live: epichub-tls tls.crt = served chain per the kubernetes.io/tls convention) + goes token-free (mtls sole credential); stale ConfigMaps pruned; FerraCert spec-retire FR filed (2026-06-10). See Sprint 5.2 §7 + docs/proposals/ferracert-spec-retire-verb.md.

  • Sprint 5.2 🚧 — vault v1.10.0 + ["mtls"] live; cert flow END-TO-END: production epichub-tls delivered to epic-edge (2026-06-10). The full window on talos-r740: __root__ admin binding (operator-authorized) → one-time VCT orphan-delete → chart 0.9.0 / v1.10.0 (the FR-170 snapshot self-heal fired live on all three voters) → unseal → final ["mtls"] posture (tokens ignored; admin via the committed fv-admin-pod.yaml ceremony manifest) → policy path-shape fix (* is single-segment; raw per-RPC paths) → FerraCert [delivery.vault] applied → issue-Job vault-mtls mount gap fixed → issuance #7 vault push: delivered attempts=1 → epichub-tls (kubernetes.io/tls, LE production, key-cert match) materialised in epic-edge 5 s later via the cert watch — renewals now flow hands-off. cert-flow.md + ferravault-unseal.md reconciled to the live posture. See Sprint 5.2 §7.

  • Sprint 5.2 🚧 — FerraVault 1.7.0 upgrade wedge: THREE stacked boot defects root-caused + recovered live; FR-170 RESOLVED upstream same day (FerraVault 8.13 → v1.10.0/chart 0.9.0); mtls bring-up rungs 1–2 done (2026-06-10). The 1.5.0→1.7.0 window wedged all three voters three distinct ways — FVSNAP02→03 magic mismatch, pre-ADR-043 poisoned-follower ciphertext, and a snapshot-AAD build-term/boot-term drift that wedged EVERY restart-after-election — each root-caused in FerraVault source, recovered via maintainer-blessed quarantine-move procedures, and filed as FR-170 (Findings 1/1b/1c/1d, nine asks, ALL answered by FerraVault Sprint 8.13: self-healing quarantine, root-fix FVSNAP04 build_term header, tooled ferra cluster rebuild-follower, Raft-availability readiness, version-free VCT labels, chart-native ferracert.deliveryPubkey). EPIC re-pinned to v1.10.0/chart 0.9.0. Auth semantics corrected (methods is AND/intersection, not alternatives — final posture ["mtls"] alone): policies + mtls bindings registered for ferracert-push + epic-secret-sync; epic-secret-sync carries its cert-manager client cert live; the FerraCert [delivery.vault] flip is staged. ferravault-unseal.md gains the version-conditional pre-restart scrub + rebuild-follower + post-flip admin-access procedures. See Sprint 5.2 §7.

  • Sprint 5.2 🚧 — Both sibling FRs RESOLVED upstream (FerraCert PV1.4 + FerraVault 8.12); backup-CronJob live-fix; T-5.2.10 plane egress NPs + T-5.2.12 runbooks closed (2026-06-10). FerraCert shipped the two-phase DNS-01 driver + settle knob (PV1.4) and FerraVault the golden-vector contract hardening (8.12) — every Sprint 5.2 FR is now resolved at source; the single remaining external ask is the FerraCert image publish (ferravault:1.7.0 is already in the registry). Live-fixed the FerraCert backup CronJob (base manifest lacks a writable /tmp for snapshot self-verify — overlay emptyDir patch, manual run green). Plane egress NetworkPolicies rewritten against the live substrate + applied (Ferrum lanes 7000-7003, CNPG labels, FerraVault in epic-state); docs/operations/cert-flow.md + docs/operations/migration-runner.md landed; DESIGN.md §11.4 reconciled. See Sprint 5.2 §7.

  • Sprint 5.2 🚧 — epichub.ai ACME staging rehearsal GREEN: Cloudflare token ceremony + the epichub-wildcard staging leaf issued; NEW FerraCert FR (DNS-01 multi-authz race) (2026-06-10). The Phase 5 Cloudflare operator ask landed: scoped token (Zone:Read + DNS:Edit, epichub.ai only) escrowed in FerraVault KV + ferracert-secrets; ACME CAs + the *.epichub.ai+apex spec flipped on; the staging leaf (serial 2c920720…, ECDSA-P256, pem+k8s) issued end-to-end through DNS-01. Two defects caught live: an EPIC NP gap (propagation checks query authoritative NSes on public 53 — fixed) and a FerraCert order-driver race (serialized prepare→set_ready fails every fresh apex+wildcard order — FR filed: docs/proposals/ferracert-dns01-multi-authz-race.md); the production-CA flip is gated on that FR. See Sprint 5.2 §7.

  • Sprint 5.2 🚧 — Ferrum Sprint 9.1 FR consumed: encryption.provider: manual + coherent sprint-9.1 image pair live; the ESO bridge retired (2026-06-10). Rolling 8.0→9.1 upgrade kept quorum; smart-client lanes restored; SET/GET green via the new CLI; epic-secret-sync is now the only FerraVault→K8s secret path (ADR-038 end state). Three new chart 0.2.0 findings appended to the FR doc (immutable volumeClaimTemplates labels; NP covers only shard_base_port; client-selector key rename). See Sprint 5.2 §7.

  • Sprint 5.2 🚧 — T-5.2.7 epic-secret-sync built + live on talos-r740 (2026-06-10, ADR-038). New binary crate: FerraVault→Kubernetes Secret reconciler (declarative kv/cert mapping, server-side apply, content-hash idempotence, fail-safe never-delete semantics, Vault.Watch-driven incremental reconciles + resync backstop). kube 3.1/k8s-openapi 0.27 confined to the crate by a cargo deny wrapper. 9 unit + 3 real-daemon integration tests. Live E2E on the cluster: scoped-token read of FerraVault KV → SSA Secret in epic-edge; rotation converges via the watch path in ~60 ms with no restart. Two FerraVault findings worked around EPIC-side (chart NP cross-namespace client gap → networkpolicies/ferravault-clients.yaml; policy path-shape asymmetry kv-raw vs kv/-prefixed-watch vs cert/). See Sprint 5.2 §7.

  • Sprint 5.2 🚧 — T-5.2.6 FerraCert live on talos-r740 (dev-CA mode) + NEW FR: the FerraCert↔FerraVault vault-push contract is wire-incompatible (2026-06-10). FerraCert 0.1.0 (e2774a9, operator-published, digest-verified + pull-tested) deployed via its official kustomize base + EPIC overlay (deploy/k8s/epic-state/ferracert/): dev CA local-dev minted (not-after 2036), Ed25519 delivery key minted (pubkey staged in ferravault-values.yaml), master key escrowed in FerraVault KV epic/ferracert/master-key, ferracert-vault-push client cert issued from epic-internal-ca. Vault push stays disabled: source audit found FerraCert pushes HTTP POST /v1/bundles + canonical-JSON Ed25519 while FerraVault only accepts gRPC Vault.CertIngest + prost-signed envelopes — wire-incompatible on every layer, with a boot-probe false-pass mode that could silently drop every push as PermanentRejection. FR filed: docs/proposals/ferracert-ferravault-delivery-contract-mismatch.md. See Sprint 5.2 §7.

  • Sprint 5.1 🚧 IN PROGRESS 2026-06-09 — Phase 5 cluster foundation: Talos on R740 (docker + lab) + Cilium + LoadBalancer/L2 ingress + storage; live-validated on talos-r740. Cluster-substrate work; zero EPIC Rust touched. ADR-016 Accepted, then revised the same day after live-lab validation: the original k3d-dev/Talos-prod + Gateway-API plan was reworked to Talos everywhere (one substrate, three provisioners: talosctl cluster create docker dev/CI · the live lab talos-r740 integration · R760 metal prod) with LoadBalancer + L2 ingress (not the Gateway API — its CRDs are absent on the lab and L7 is the wrong tool for an L4 QUIC edge). k3d/kind deleted as tech debt (a second non-Talos distro needs permanent version-syncing + reintroduces a fidelity gap; the docker provisioner gives the same ephemeral cluster on the same OS as prod). Stale version pins corrected to the lab's running versions (Cilium 1.16→1.19.4, k8s 1.31→1.36). Delivered: the talos-docker dev recipe (deploy/dev-cluster/up.sh/down.sh) + EPIC CiliumLoadBalancerIPPool (192.168.50.60–69); the R760 Talos controlplane/worker patches, talosctl validate --mode metal-clean + their portable subset exercised by the docker provisioner (deploy/talos/); the Cilium 1.19.4 values (render-verified against the real chart; gatewayAPI: false) (deploy/k8s/epic-system/); the §11.3 namespaces + default-deny across all five EPIC namespaces with DNS-egress pre-opened (deploy/k8s/baseline/); the epic-edge-lb/epic-control-lb LoadBalancer Services (deploy/k8s/ingress/); the epic-fast/epic-bulk StorageClasses (deploy/k8s/storage/); the rewritten bootstrap/recovery runbook (docs/operations/cluster-bootstrap.md); and the validation gate (deploy/validate.sh → ci.sh Stage 1). Live-validated on the lab talos-r740 cluster: 5 EPIC namespaces created, default-deny enforced (cross-namespace probe dropped, DNS egress resolved), epic-fast provisioned + bound a real PV. DESIGN.md §11.2/§11.3/§11.6/§11.7 reconciled. Remaining operator step: allocate the LB VIP pool onto the shared LAN. See Sprint 5.1.

  • Sprint 4.5 ✅ COMPLETE 2026-06-09 — Phase 4 sign-off (gating); tag phase-4-final (unsigned annotated). A gating sprint that did its job: a second operator-requested re-audit caught a silent cross-sprint drop and the one item built here (T-4.5.17) closes it rather than papering over it. Every Phase 4 §2 item across Sprints 4.1–4.4.5 delivered, zero four-field deferrals; the §2 originally-planned-vs-delivered table closes "(none)". T-4.5.17 — the ADR-013 §5 5% elevated-risk review-sampling control was found unbuilt and untracked (deferred 4.1 → "4.3's capture writer"; 4.3 never scoped it; the percentage was a stored flag with no code acting on it; no four-field block anywhere). Since it is testable (by direct analogy to diagnostic capture) the operator's "defer only if untestable" condition was not met, so it was built end-to-end at the production bar (ADR-037 Accepted): epic-trait::review_sample least-privilege boundaries + list_elevated_risk_orgs; the ADR-008 seal/open codec factored into a shared audited epic-store::envelope (diagnostic refactored onto it, byte-identical AAD, 5/5 re-proven); PgReviewSampleStore over a domain-tagged AAD + migration 0043_abuse_review_samples (FORCE-RLS + retention sweep, registered in EPIC_OWNED_TABLES); the edge deterministic-5%-of-prompt tee off the hot path (ADR-026 zero-alloc preserved) + opt-out-approve KEK provisioning (fail-safe 503 without it) + the operator review-queue API (list/decrypt-with-audit-before-plaintext/mark-reviewed) + hourly retention sweep. Tests: integration_review_sample 5/5 vs postgres:18-bookworm (round-trip, master-key-never-in-PG, mark-reviewed, sweep, delete, RLS), integration_abuse_gate edge-tee end-to-end + no-KEK fail-safe + non-elevated-no-sample, sampling proptest, diagnostic parity 5/5, edge zero-alloc green. Three audits ✅ PASS — quality (workspace line coverage 85.42 % ≥ 80 % floor; the listed critical surfaces — epic-abuse::preflight 100 %, killswitch 95.92 %, report 95.72 %, epic-vault::envelope 87.67 %, epic-vote::tabulate 94.79 % — meet their CLAUDE.md §5.3 reachable-code bars, with each residual missed line classified as a §5.3-excluded category), security, architecture. ci.sh --full Stages 0–6 green; re-gate (clippy -D warnings + cargo test --workspace, 116 suites / 0 failed + cargo doc -D warnings) green on the sign-off state. The three DESIGN.md §17 exit-criteria drills ran end-to-end: (1) classifier refusal → audit → metric on all three surfaces; (2) diagnostic capture of a real request round-tripped through real Vault.Unwrap; (3) a synthetic election. The live-Ferron classifier exit drill was run for real, not deferred — a genuine FerronEncoder.Classify encoder surface (Bearer + sigmoid head per ADR-033/034) was stood up against the real python -m ferron_worker.encoder worker on a CPU multi-label fixture; EPIC's FerronClassifier + edge gate proved reject/allow at the 0.9 boundary, 403 forbidden_category on all three surfaces, a Filtered meter event, benign→Ferron in up-mode, and fail-closed 503 abuse_screening_unavailable + zero Ferron calls in down-mode (ci/drills/phase-4-live-ferron/, the env-gated epic-edge integration_live_ferron_classifier_drill test that skips clean when the drill harness is absent). The Postgres-16→18 drift caught during sign-off was closed in full — a single source of truth (epic_test_utils::POSTGRES_TEST_IMAGE_TAG = "18-bookworm") now drives every testcontainer fixture, with ci.sh, DESIGN.md, the compose pin, and the workspace comment realigned; 72 DB-backed tests green on PG18. The five ADR-026 edge invariants were re-asserted under the classifier request-entry gate + the diagnostic-capture tee; Phase 4 criterion baselines frozen under ${EPIC_DATA_ROOT}/ci-artifacts/criterion/<crate>/phase-4-final/; aarch64 cross-build green; backup + DR drill green. See Sprint 4.5 sign-off.

  • Sprint 4.4.5 ✅ COMPLETE 2026-06-08 — Ferron integration reconciliation (Bearer transport + sigmoid head + tokenizer hash + admin/load). The Phase 4 analogue of the Phase 1 Ferrum/FerraVault substrate corrections: epic-ferron-client was built against proposed Ferron contracts that diverged from Ferron's shipped surface, so against a production-auth Ferron every call returned unauthenticated. Transport (ADR-033): EPIC now authenticates with authorization: Bearer <key> + x-ferron-tenant-id over server-side TLS (no client cert) — new auth.rs (BearerAuth + tonic interceptor), mtls.rs slimmed to server-CA-only, three per-surface FerraVault Bearer keys rotated header-source-only (no channel rebuild). Classifier (ADR-034): proto re-vendored to FerronEncoder/ClassifyRequest{model}; verdict_from_scores filters to the three known bright-line labels under the multi-label/sigmoid head (unknown/benign label can never cause or mask a reject). Tokenizer (ADR-035): RealTokenizer computes the per-model build hash via the shared ferron_tokenizer::tokenize_build_hash helper (wrapper crate version excluded); new CapabilityClient (HTTP/JSON over Bearer) replaces the heartbeat stub. admin/load (ADR-036): Bearer admin key per poll + degenerate single-instance/single-group snapshot tolerated. The loopback test fabric now runs auth-enabled — a valid key authenticates on infer/classify/admin and a missing/blank key ⇒ unauthenticated ⇒ fail-closed (the gap the empty-key loopback hid). ADR-033/034/035/036 Accepted; ADR-009/010 marked superseded-in-part; ADR-012 amended at filing. See Sprint 4.4.5 sign-off + docs/integration/ferron-fr-response-2026-06-08.md.

  • Sprint 4.4 ✅ COMPLETE 2026-06-08 — epic-vote: model proposal, ballot, seat-weighted ranked-choice tabulation. All 12 §2 items delivered, zero four-field deferrals. The control-plane-only model-governance subsystem (cargo tree -p epic-edge | grep epic-vote = 0). epic-vote::tabulate is a pure, total, deterministic seat-weighted instant-runoff function (lowest-proposal_id tie-break; Majority/LastStanding stop reasons; full per-round elimination trace) with a proptest suite asserting no-double-count, the majority criterion, unanimity, quorum-iff, and determinism/order-independence (explicitly not global monotonicity — an accepted, documented IRV property). Proposal admission (epic-vote::admission) gates on the founding-member plan + the GB300 hardware envelope; ballot validation (epic-vote::ballot) rejects empty/duplicate/off-ballot rankings; epic-vote::VoteService orchestrates propose/open/cast/close over epic-trait::VoteStore + AuditSink (every action hash-chain-audited; quorum-not-met is a first-class published result). Migration 0042_model_voting.sql reconciles the foundation schema (0009–0013) to the governance model: model_vote_ballots becomes one ballot per org ((vote_id, org_id) PK, org-scoped FORCE-RLS, user_id retained as cast_by) because seat_count is an org-level property (§9.8) — per-user ballots would let a multi-user org out-vote its seat share; model_proposals gains context_window+param_count_b for the admission gate. epic-store::PgVoteStore (cast-only-while-open + dedup-by-org ON CONFLICT UPSERT, legal-transition WHERE guards) + epic-test-utils::MemVoteStore. epic-web::pages::votes (list/detail/propose, no-JS ranked ballot) + epic-control::routes::web::votes (founding-member-gated) + routes::votes_admin (operator open/close); the closed-vote detail page re-tabulates the frozen ballots to render the elimination trace deterministically. Synthetic election (2,500 orgs, Pareto seats) green; tabulate_warm ≈ 36.5 µs; integration_vote/rls/schema_invariants green vs real Postgres. ADR-014 Accepted. Post-close self-review hardening (2026-06-08): the seat-fraction quorum was moved off f64 to an exact integer rational (SeatFraction { numerator, denominator: NonZeroU32 }, ceil(eligible × num / den) in overflow-safe u128) — a legally-load-bearing governance threshold no longer relies on IEEE-754 rounding, three cast_* lint suppressions removed, the bar reproducible from the audit record (admin …/close wire contract → quorum_numerator/quorum_denominator). See Sprint 4.4 sign-off + §7.

  • Sprint 4.3 ✅ COMPLETE 2026-06-07 — diagnostic capture end-to-end (envelope encryption via FerraVault Vault.Wrap/Unwrap). All 13 §2 items delivered, zero four-field deferrals. The per-org diagnostic-capture write path under ADR-008 / ADR-032: the epic-vault::envelope Vault.Wrap/Unwrap RPC surface (new epic-trait::key_wrap — KeyWrapper + KekProvisioner least-privilege split; unwrap returns Zeroizing); epic-store::PgDiagnosticStore::{seal,open}_capture (fresh ephemeral AES-256-GCM data key per capture → seal payload → Vault.Wrap the data key → zeroize; AAD binds version‖org‖public_id; migration 0041_diagnostic_captures_envelope.sql); the edge DiagnosticsHook real consumer teeing both pipeline paths with zero off-cost (capture_off_hot_path_warm ≈ 14 ns; capture_target → None short-circuits before any clone; ADR-026 pipeline modules stay runtime-clean); the per-org toggle + GDPR Art. 6(1)(a) explicit-consent surface (/admin/diagnostic-capture, who/when/7-day expiry, disable → delete_for_org purge); the FerronResponseMetadata diagnostic flag (include_diagnostics=17, riding EpicExtension.ferron_metadata); the hourly 7-day expiry sweep (tasks::diagnostic_sweep, hard-delete under epic_admin); the operator capture viewer (Vault.Unwrap → decrypt → Zeroizing JSON, audited before plaintext, per-record-signed); and the FerraVault/Ferrum sub-second keyspace-watch push cache watcher (FerrumDiagnosticSettingsWatch over the operator-accepted AsyncKeyspaceWatcher, ~0.13 s publish→notify) replacing the Sprint 2.3 5 s-TTL stopgap. The per-org master key never leaves FerraVault — the binaries hold only ciphertext + the wrapped data key; integration_diagnostic_capture (5/5 vs real Postgres) proves the round-trip + master-key-never-in-Postgres + expiry-sweep + revocation-purge. ADR-032 Accepted. (The Sprint 4.3 doc frontmatter + sign-off checklist + this CHANGELOG line were reconciled at the Sprint 4.5 sign-off, 2026-06-09 — the work shipped 2026-06-07 but the doc close-out had slipped; PHASE-4-OVERVIEW already recorded it ✅.) See Sprint 4.3 sign-off.

  • Sprint 4.2 ✅ COMPLETE 2026-06-07 — kill-switch primitives + abuse-report state machine + anomaly detection. All 14 §2 items delivered, zero four-field deferrals. The response half of the abuse layer. Kill switches (DESIGN.md §13.4): the three identity scopes (epic-abuse::KillSwitch over Store/SessionStore/SessionLookup/Cache/AuditSink — no DB import) fire a cross-tenant cascade (key→revoke; user→keys+sessions; org→suspend+keys+member-sessions) + a hot-cache tombstone + actor/target/reason audit; the three edge-config scopes (epic-edge::EdgeKillSwitches, ArcSwap snapshot: IP/CIDR + country geofence + UA regex + model-grounding + org-deny) hot-reload (compile-then-swap) and are consulted per request (no Postgres on the request path). Abuse-report state machine (DESIGN.md §13.5): migration 0040_abuse_reports_lifecycle.sql (additive — the table exists since 0016; reconciles the key_leak CHECK latent bug + adds immediate_harm/source/content_hash); the ReportStatus legal-edge machine + epic-abuse::report decision policy; six operator triage routes; the public Turnstile /abuse form (epic-web SSR, no auth, 5/hour/IP, BLAKE3 content-hash idempotent); the abuse@ Postmark inbound parser (idempotent on MessageID, dead-letter on unparseable — R4.9); the 24 h SLA monitor + immediate-harm escalation paging (all three intake paths). Auto-disable link (DESIGN.md §13.4): MeterEvent gains api_key_public_id (protobuf tag 12); the edge emits a Filtered event on pre-flight reject; epic-trait::MeterObserver lets epic-meter::Aggregator fan events to epic-abuse::AutoDisableTracker, firing the API-key kill switch (system actor) at 5 rejects/key/hour. NCMEC quarantine (DESIGN.md §14.5): epic-trait::QuarantineWriter boundary + the …/ncmec-quarantine route (preserve evidence under the per-org key → record ncmec_report_ref → org kill cascade); envelope encryption is Sprint 4.3, live filing Phase 6. Seven §13.7 anomaly dashboards + Loki/Prometheus rules under ops/observability/, none auto-action. Benches: killswitch_consult_warm ≈ 99 ns, killswitch_invalidate_warm ≈ 246 µs. See Sprint 4.2 sign-off.

  • Sprint 4.1 ✅ COMPLETE 2026-06-07 — epic-abuse substrate + pre-flight classifier + opt-out (Phase 4 opens). All 14 §2 items delivered, zero four-field deferrals. Lands the surface-agnostic request-entry pre-flight abuse gate (DESIGN.md §13.2): epic-abuse::PreflightClassifier (pure over ClassifierBackend; GateDecision::{Proceed,Forbidden,FailClosed}) consulted once in the edge pipeline ahead of route selection — one gate covers native + OpenAI + Anthropic; a reject → typed 403 forbidden_category carrying the category name only (structured category field, no prompt content) mapped into all three surface envelopes; a classifier outage fails closed → 503 abuse_screening_unavailable (no prompt reaches Ferron). The real epic-ferron-client::FerronClassifier adapter dials Ferron's encoder-plane unary Classify RPC over mTLS (vendored ferron_classifier.proto, Sprint 2.1 channel-pool/rotation pattern, EPIC-side 0.9 threshold, hard per-call timeout → fail-closed) — proven against a real in-process mTLS loopback; production boot wires it behind the [abuse] classifier_endpoint config flip (empty → allow-all stand-in until Ferron's plane ships, ADR-012 §8). The documented opt-out workflow (ADR-013): abuse_opt_out_requests (migration 0039) + AbuseOptOutStore (Pg/Mem) with the requested→under_review→approved/denied, approved→revoked state machine; the authenticated org-admin /admin/abuse-opt-out SSR form; operator review at /api/admin/abuse/opt-out-requests/*; the elevated-risk-review queue (classifier-disabled + 5 % sampling flag + quarterly review + suspend-on-signal); a 5 s-TTL edge opt-out hot cache refreshed from the store; every decision per-record-signed-audited. The ClassifierBackend trait latency reconciled to < 15 ms p99 (classify_gate_warm bench ≈ 165 ns EPIC-side overhead); the zero-per-token-allocation forward path (ADR-026) untouched (request-entry gate). The 5 % sampled-content write + evidence envelope bind on Sprint 4.3's ADR-008 capture writer (cross-sprint dependency per ADR-013 §5). ADR-012 + ADR-013 Accepted. See Sprint 4.1 sign-off.

  • Sprint 3.5 ✅ COMPLETE 2026-06-07 — Phase 3 sign-off (gating). phase-3-final tag lands. All 15 §2 items closed: T-3.5.1 (63-row originally-planned-vs-delivered table; 63/63 ✅ Delivered; zero deferrals) + T-3.5.2/3/4 (quality / security / architecture 34/34 audits ✅ PASS) + T-3.5.5 (Phase 3 criterion baselines frozen under ${EPIC_DATA_ROOT}/ci-artifacts/criterion/<crate>/phase-3-final/ — shim translate/reframe 106 ns–1.5 µs, hot-path overlay 146/164 µs vs 154 µs native, jwt_issue_via_crypto_sign_warm ≈ 184 µs, audit_sign_per_record_warm ≈ 9.9 µs) + T-3.5.6 (Stage 1 1576 tests green / clippy / doc / fmt; Stage 2 audit + deny green; Stage 4 conformance; Stage 6 shim load smoke) + T-3.5.7 (aarch64 cross-build, both bins ARM ELFs) + T-3.5.8 (coverage — translate::* + error 96–100 %; completion.rs lifted 71 %→99.59 % via 3 added tests) + T-3.5.9 (this entry + DESIGN/ADRs/CLAUDE.md §0 reconciled) + T-3.5.10 (conformance 11/11 + 7/7) + T-3.5.11 (cancel budget through both shims, p99 ≈ 2.1 ms) + T-3.5.12 (JWT signing key gone from epic-auth memory) + T-3.5.13 (backup + DR drill, 8/8 commits tree-identical) + T-3.5.14 (phase-3-final unsigned annotated tag → backup) + T-3.5.15 (Phase 3 → Phase 4 hand-off). Two real defects caught + fixed by the gating sprint: a 19-file rustfmt-drift cleanup (committed pre-fmt across 3.1–3.4) and a cargo deny two-plane bans regression (Sprint 3.3's dev-only epic-vault → epic-auth test edge was not allowlisted; production graph always clean). See Sprint 3.5 sign-off.

  • Sprint 3.4 ✅ COMPLETE 2026-06-07 — Cross-shim conformance/hardening + the visual-regression baselines (T-1.4.16) closed in full. All 9 §2 items delivered, zero deferrals. The official-SDK conformance suites became ci.sh Stage 4 gates (ci/conformance/run.sh — builds the conformance_server once, runs both suites in exact-pinned venvs); the SDK pins were bumped to latest stable (openai==2.41.0, anthropic==0.107.0) and both shims are green unchanged (the server-side wire contracts are stable across the bump — no epic-shim-* / route / test change). The typed 501 posture was verified against the real OpenAI SDK across embeddings / audio / images / assistants / moderations — the last was a 404 gap (no route), now a typed 501 (/v1/moderations added to the unsupported table + is_openai_path). New edge integration suites: cross-surface byte-for-byte forward parity (one MockFerron script → identical concatenated content across native / OpenAI / Anthropic, streaming + non-streaming, UTF-8/emoji/whitespace/JSON-escape stress) and error-envelope parity under FerronError fault injection (every variant, pre-stream HTTP status+envelope + mid-stream in-band frame, per surface — OpenAI server_error vs Anthropic api_error taxonomy, shared EPIC-stable code). The ADR-021 Rust load harness gained OpenAI chat.completions + Anthropic messages request shapes (per-surface edge p99 < 5 ms, TTFT p99 < 1 s, zero meter loss, uniform model_id); the Sprint 2.4 cancel-chaos drill re-ran through both shim overlays (p99 ≈ 2.4 ms — the overlay does not buffer). Cross-shim criterion sweep frozen under phase-3-progress/ (no interaction regression). The project's longest-running carry-forward — the visual-regression half of Phase 1 T-1.4.16, deferred through Phase 1 → 2 → 3 — closed in full (operator directed "Resolve Tailwind + land it now"): the Tailwind precompile pipeline (crates/epic-web/styles/, pinned standalone tailwindcss v3.4.19, sha256-verified, no Node.js/npm dep; compiles input.css → committed src/assets/site.css; build-css.sh --check is a byte-determinism gate in ci.sh Stage 1; zero visual drift via preflight: off + @layer) and the visual-regression gate (ci/visual/, crates/epic-control/tests/visual_regression.rs: headless-browser screenshots of the static public SSR pages via fantoccini, pixel-delta gate; driver-agnostic geckodriver/chromedriver; baselines out-of-tree under ci-artifacts/visual-regressions/; validated at 0.0 pixel-delta on 5 pages). See Sprint 3.4 sign-off.

  • Sprint 3.3 ✅ COMPLETE 2026-06-07 — Auth-hardening: the FerraVault Vault.Sign JWT migration + audit-log per-record signing. The headline security win: the plaintext JWT signing key leaves epic-control process memory entirely. epic-auth::JwtService issues each access token via a per-issuance Vault.Sign RPC (a new epic-trait::CryptoSigner, implemented by epic-vault::VaultClient over FerraVault v1.1.0) — a memory dump no longer yields forge-a-token material. Verification stays in-process and RPC-free against an ArcSwap cache of Ed25519 public keys, refreshed on the KEY_ROTATED watch event (epic-control::tasks::jwt_key_rotation, which ignores KEK / unrelated-key rotations) by re-reading FerraVault's new non-sudo Vault.SigningKeyPublic RPC (added by the operator for this consumer so a sudo token stays off the verify-reload path). Issuance fails closed (bounded transient retry via RetryingCryptoSigner → 503 signing_unavailable + Retry-After; no local-key fallback). The audit_log hash-chain gains optional per-record Ed25519 signing (migration 0038, signed tuple (id, prev_hash, row_hash) via a dedicated epic-audit-signing-key-v1, config-gated): epic-store::PgAuditSink::with_signing pre-allocates the id via nextval (no UPDATE — the append-only trigger is untouched) and verify_persisted_chain checks linkage + signatures (3/3 vs real Postgres, green with signing on AND off). epic-config gains [auth.signing] + [audit.signing]; LocalEd25519Signer backs the FerraVault-less dev/test posture. ADR-030 + ADR-031 Accepted. Two-plane invariant holds (epic-edge → epic-auth = 0). All 13 §2 items delivered, zero deferrals: the real-ferravault integration_signing suite (4/4) + the jwt_issue_via_crypto_sign_warm bench (warm median ≈ 186 µs over loopback FerraVault TLS, ~54× under the ADR-030 < 10 ms budget) went green once the operator attached the ferravault-test-utils v1.1.0 signing-engine ring — the suite passed unchanged. See Sprint 3.3 sign-off.

  • Sprint 3.2 ✅ COMPLETE 2026-06-02 — The Anthropic Messages surface end-to-end + the cross-tool tokenizer byte-parity gate. epic-shim-anthropic becomes a set of pure, total translators over the stable EPIC native surface (ADR-027, reused verbatim): MessagesRequest → InferRequest (folding the top-level system slot, normalising string-or-array content, enforcing the required max_tokens, image/tool-use blocks → typed 501), InferResponse → the non-streaming Message, and the EPIC-native StreamEvent sequence → the six-event Anthropic SSE state machine (message_start → content_block_start → content_block_delta* → content_block_stop → message_delta → message_stop, framed event:\ndata:\n\n, 3 allocs/token = Phase 2 steady state) with timer-driven event: ping keep-alives (ADR-029). epic-edge::routes::anthropic mounts POST /anthropic/v1/messages (streaming + non-streaming), GET /anthropic/v1/models (Anthropic-shaped, per-org filter), and the 501 posture for the deprecated /complete — reusing the middleware tower + dispatch_streaming_with_framer with no re-wiring; the Phase 2 StreamFramer trait gained defaulted on_open/keepalive_interval/write_keepalive hooks + one lowest-priority producer keep-alive arm (no-ops for native + OpenAI). x-api-key auth added so the official anthropic SDK works unmodified. Fixed a Phase 2 seam where a mid-stream upstream error leaked the native error shape onto shim streams (now vendor-framed on all surfaces). Retired the Phase-0 epic-protocol::anthropic stub (fuzz target relocated). Cross-tool tokenizer byte-parity (T-3.2.9, closes the Phase 2 carry-forward): 256 prompts × 3 models render byte-identically and bill identically across native / OpenAI / Anthropic. 46 shim tests + 13-test edge integration suite + counting-allocator zero-alloc test green; benches (anthropic_request_translate_warm ≈ 1.26 µs, anthropic_event_reframe_warm ≈ 119 ns, overlay ≪ 500 µs); anthropic SDK conformance 7/7 against anthropic==0.105.2 (ci/conformance/anthropic/). ADR-029 Accepted. See Sprint 3.2 sign-off.

  • Sprint 3.1 ✅ COMPLETE 2026-06-02 — Shim translation substrate + the OpenAI Chat Completions surface end-to-end. Phase 3 opens with the surface most customers use. epic-shim-openai becomes a set of pure, total translators over the stable EPIC native surface (ADR-027): OpenAI ChatCompletionRequest → InferRequest, InferResponse → ChatCompletion, and the EPIC-native StreamEvent sequence → chat.completion.chunk + data: [DONE] SSE re-framing (zero per-token allocation preserved — the Phase 2 dispatch_streaming producer was factored into a monomorphised StreamFramer strategy, the substrate Sprint 3.2 reuses). epic-edge::routes::openai mounts POST /v1/chat/completions (streaming + non-streaming), POST /v1/completions (legacy, complete), GET /v1/models (OpenAI-shaped, per-org entitlement filter), and the typed 501 posture for embeddings / audio / images / assistants — reusing the existing middleware tower + dispatch_streaming with no edge-pipeline re-wiring. Every typed failure (translation-stage ShimError + pipeline-stage EdgeApiError, incl. auth + rate-limit) maps to the OpenAI {error:{message,type,param,code}} envelope with EPIC-stable codes + HTTP status parity. Retired the unused Phase-0 epic-protocol::openai stub (a vendor schema is a surface concern) and relocated its fuzz target. /v1/models resolution (ADR-028): OpenAI-shaped canonical on the edge; richer native catalogue at /v1/capabilities (new epic_router::build_catalogue_router). 50 shim unit/property tests + 13-test edge integration suite + strict counting-allocator zero-alloc test green; criterion benches (openai_request_translate_warm ≈ 1.85 µs, openai_chunk_reframe_warm ≈ 223 ns, < 500 µs overlay); openai-python conformance suite 7/7 against openai==1.109.1 (ci/conformance/openai/ + the conformance_server example). ADR-027 + ADR-028 Accepted. See Sprint 3.1 sign-off.

  • Sprint 2.5 sign-off remediation pass — 2026-06-02 (after the initial close) — operator-requested production-grade follow-up to address every finding from the honest-assessment readout. Three follow-ups landed: (a) aarch64 link-stage closed cleanly, retiring the original four-field deferral — root cause was a missing linker = "aarch64-linux-gnu-gcc" in /.cargo/config.toml, not a toolchain gap; with that one toggle cargo build --target aarch64-unknown-linux-gnu --release -p epic-edge -p epic-control succeeds in 2m 19s; both binaries verified as ARM aarch64 PIE ELFs (epic-edge 73 MB, epic-control 80 MB) and smoke-tested under qemu-aarch64-static -L /usr/aarch64-linux-gnu. (b) cargo geiger ran to completion across every workspace member, refreshing the unsafe inventory in docs/audits/phase-2-quality-audit.md §3 — every EPIC-owned crate carries the ? glyph except epic-crypto (already-documented Secret::<T>::into_inner ManuallyDrop discipline). (c) Edge-side tokenizer determinism gate added at crates/epic-edge/tests/integration_tokenizer_determinism.rs — 5/5 tests covering in-process determinism (1 024 invocations), instance independence, role-prefix stability, Unicode safety (CJK + emoji + RTL), and fixed-backend rendered-prompt determinism. The §9 hand-off Phase 3 carry-forward was narrowed from "tokenizer byte-parity property test" to "cross-tool byte parity at the shim layer" to reflect what actually remains. phase-2-final tag moved from 27bdc23 to the remediated HEAD so the tag points at the actual closing state.

  • Sprint 2.5 ✅ COMPLETE 2026-06-02 — Phase 2 sign-off (gating) — phase-2-final tag lands. All 15 §2 items closed: T-2.5.1 (82-row originally-planned-vs-delivered table; 67 distinct work-contract items all ✅ Delivered; zero substantive deferrals) + T-2.5.2 (docs/audits/phase-2-quality-audit.md ✅ PASS — 12 new #[allow(...)] with reasons; 4 new unsafe test/build-script-only with SAFETY comments; banned-shortcut grep clean; zero-per-token-alloc invariant defended at 3 allocs/token) + T-2.5.3 (docs/audits/phase-2-security-audit.md ✅ PASS — BLAKE3 + subtle::ConstantTimeEq, mTLS rotation drill, ADR-009 admin-cert custody, ADR-010 tokenizer parity, kill-switch propagation within 60 s Ferrum cache TTL) + T-2.5.4 (docs/audits/phase-2-architecture-review.md ✅ 27/27 PASS — Phase 0's 10 + Phase 1's 7 + Phase 2's 10 questions; one in-audit cleanup of dead axum-extra workspace-dep declaration; ADR-026 pipeline-module runtime-agnosticism invariant verified) + T-2.5.5 (criterion baselines frozen under ${EPIC_DATA_ROOT}/ci-artifacts/criterion/<crate>/phase-2-final/) + T-2.5.6 (ci.sh --full green) + T-2.5.7 (aarch64 cross-build green) + T-2.5.8 (coverage workspace ≥ 80 %; 100 % on listed Phase 2 critical surfaces) + T-2.5.9 (this entry + DESIGN.md / ADRs / root CLAUDE.md §0 reconciled) + T-2.5.10 (cancellation budget under 100 RPS sustained load — 500 cancel samples, p50 ≈ 2.5 ms, p99 ≈ 9.6 ms, p99.9 ≈ 11.7 ms; well under 100 ms wire SLO and 21 ms regression budget) + T-2.5.11 (org-pooled annotation correctness over 1 024 fixture orgs — every priority_class / fair_weight / kv_tier_hint / request_id / user_id_hash matches DESIGN.md §9.8) + T-2.5.12 (10/10 random Phase 2 commits backup-verified) + T-2.5.13 (DR drill green) + T-2.5.14 (phase-2-final unsigned annotated tag pushed to backup) + T-2.5.15 (Phase 2 → Phase 3 hand-off note in sign-off §9). Two new integration tests land: crates/epic-edge/tests/integration_cancel_budget_under_load.rs (sustained-load cancellation budget) and crates/epic-router/tests/integration_org_pooled_annotations.rs (1 024-fixture-org property test).

  • Sprint 2.4 ✅ COMPLETE 2026-06-02 — per-token streaming + biased-tokio::select! cancellation + 100 ms budget + Stage 6 load smoke — Phase 2's correctness slice ships at the production-grade bar. See Sprint 2.4 sign-off. 18 named sprint scope items shipped + 2 supplementary tests (chaos drill, fixture self-test). New epic_edge::pipeline::streaming::dispatch_streaming replaces the Sprint 2.3 collect-then-flush tail with a mpsc::channel(1) + Body::from_stream per-token forward; producer task runs tokio::select! biased over client-disconnect → classifier-rejection → shutdown-grace → forward arm. New foundation: epic-core::FinishReason::ShutdownCancelled, epic-protocol::MeterEvent.ttft_ms (protobuf tag 11), epic-trait::StreamRejectionWatch + NeverRejects + epic-test-utils::MemStreamRejection, ShutdownToken::cancelled_after_grace. 20-test wire-level integration suite + 5 criterion benches (forward_token_p99 ≈ 5 µs/token, cancel_propagation_p99 ≈ 2.1 ms — 50× under the 100 ms p99 budget DESIGN.md §8.6 names) + boot bench (~36 µs warm) + counting-allocator zero-alloc test (3 allocs/token in steady state). ADR-021 revised in place (Phase 2 revision: grow the Rust harness, do NOT swap to k6; future-revisit Phase 5). New ci.sh Stage 6 streaming load smoke (tests/load_smoke_streaming.rs) runs the 60 s × 500 ms-synthetic-Ferron-TTFT envelope alongside the original Phase 0 control-plane smoke. Sprint 2.3's 19-test suite stays green end-to-end against the new pipeline.

  • FerraVault cross-project feature commitments accepted + EPIC consumption committed (2026-06-01 / 2026-06-02) — sibling project FerraVault (fully under our control) confirmed delivery of four cryptographic-RPC features on 2026-06-01, with end-of-week 2026-06-05 ship date confirmed on 2026-06-02. EPIC commits to consuming all four — the consumer surface and phase assignment for each is recorded in DESIGN.md §20 — External commitments — FerraVault. (1) Vault.Wrap / Vault.Unwrap — server-side envelope-encryption RPCs; per-org KEK never leaves FerraVault. Resolves the Sprint 1.7 footer fork in ADR-008 §References — option (a) selected (FerraVault adds the RPCs to the proto) rather than option (b) (work around with KvGet-cached master keys). Phase 4 diagnostic-capture endpoint sprint consumes these RPCs directly. (2) Vault.Sign / Vault.Verify (Ed25519 + HMAC) — sign/verify-as-a-service; signing keys never leave FerraVault. Phase 3 auth-hardening sprint migrates epic-auth::JwtService from in-process ed25519-dalek to per-issuance Vault.Sign RPC per §9.2; same sprint adds the audit-log per-record signing surface per §13.6. (3) KEY_ROTATED watch event in Vault.Watch — carries {key_id, prev_version, new_version}. Phase 3 auth-hardening sprint pairs this with the JWT-signing migration (same sprint, since both touch the JWT-key lifecycle) — epic-auth JWT-key reload moves from "every KvGet revision bump triggers a reload" to "fires only on actual key rotation." (4) TTL_EXPIRED watch event — fires when a TTL'd KV entry expires. Phase 4 secrets-watch hardening sprint consumes for Postmark API token rotation per §15 and billing-provider API token watching per ADR-007 revision sprint; lands alongside the billing-provider live-merchant pick since the billing token is the first real consumer. Rejected: policy/role bundles for Admin.TokenIssue (FerraVault declined — ergonomics-only, not security; EPIC continues to issue tokens with flat policy lists). DESIGN.md §9.2 amended with the committed Phase 3 migration note; §20 table flipped from "TBD / if-when" hedges to definite consumer assignments. No new EPIC ADRs land at this checkpoint — each consuming sprint writes its own integration ADR if the shape is load-bearing. No code touched.

  • Sprint 2.0 post-close rework, second-pass self-audit (2026-06-01) — Operator follow-up self-audit ("please confirm that everything was done according the highest quality production grade long term solutions?") — same prompt that surfaced 6 gaps in Sprint 1.6's second pass and 4 gaps in Sprint 1.7's second pass — identified 5 real "we believed but never verified" gaps in the Sprint 2.0 initial close. All five closed in-place:

    • G1 — meter_record_warm measured against debug-mode Ferrum with INFO-level logging. The dev script tools/dev/start-ferrum.sh defaults to --debug --log-level=info, and Ferrum's linearizability_fork INFO trace fires on every STREAM.PUB (~600 chars per command). Re-measured against fresh /data/ferrum-target/release/ferrum-server --threads 4 --log-level warn on isolated socket — median lands in the 486–920 µs range with high run-to-run variance. The 500 µs Sprint 2.0 §5 SLO budget is borderline against this dev-machine setup; recorded as a Sprint 2.3-handoff item rather than silently masked.
    • G2 — aggregator_tick_warm bench harness was structurally wrong. The docstring claimed "Pre-load 100 events so each iteration measures the consume + UPSERT cost" but the for-loop ran INSIDE iter(), so the measurement was 100 record() + tick() (~80 ms) rather than tick() alone (~10 ms). Fixed by switching to criterion::Bencher::iter_custom with the pre-load outside the timed region. Post-fix aggregator_tick_warm lands at 9.75 ms median (SLO < 50 ms ✅, ~5× headroom).
    • G3 — ci.sh Stage 4 readiness check polled a removed wiremock. Sprint 1.7 removed the 4th wiremock (FerraVault on :8093) from ci/integration/docker-compose.yaml per ADR-025 §4 because FerraVault speaks gRPC + uses the in-process ferravault-test-utils::server::TestServer instead of an HTTP stand-in. The compose-file comments got updated, but ci.sh stage_4_integration still curl-polled :8093/__admin/mappings — stack was healthy (Postgres + 3 wiremocks all up) but the readiness loop timed out forever. Fix: remove the :8093 curl + the EPIC_VAULT_BASE_URL export. Stage 4 now passes end-to-end in 1m 50s. This was a real Sprint-1.7-introduced ci-rot bug that Sprint 2.0's first attempt to run the full stage gate surfaced.
    • G4 — strict-mode integration tests were not run during the initial close. Sprint 1.6 §F8 added EPIC_FERRUM_INTEGRATION_REQUIRED precisely because silent-skip integration tests had previously masked 3 real production bugs. Initial Sprint 2.0 close ran cargo test --test integration_aggregator without the strict env var. Second-pass run against all 4 Ferrum-touching suites with EPIC_FERRUM_INTEGRATION_REQUIRED=1 FERRUM_SERVER_BIN=/data/ferrum-target/release/ferrum-server: integration_aggregator 7/7, integration_rate_limit 5/5, integration_meter_roundtrip 3/3, integration_adapter_stack 2/2 — 17/17 strict-mode green.
    • G5 — criterion baselines were not archived to the canonical location. Sprint 2.0 §6 sign-off explicitly requires ${EPIC_DATA_ROOT}/ci-artifacts/criterion/{epic-router,epic-meter}/phase-2-sprint-2.0/. Initial close left the criterion artifacts at the default /data/ferrum-target/criterion/<bench>/ (cargo bench target dir, not the archive). Fix: cp -r of all four post-fix benches into the canonical archive directory.

    Full ci.sh stage gates on 2026-06-01: Stage 1 ✅ 1m 26s, Stage 2 ✅ 8m 25s (cargo audit + deny + geiger all clean), Stage 3 ✅ 36s (23 RLS tests pass), Stage 4 ✅ 1m 50s (post-G3 fix), Stage 5 ✅ 1m 15s (5/5 fuzz targets clean at FUZZ_SMOKE_SECS=15). Stage 6 (load smoke) and Stage 7 (release) are phase-sign-off-only; Stage 0 (backup) skipped (no backup target wired this dev box).

    Final baseline numbers (release-mode Ferrum, post-bench-fix): router_route_warm ≈ 669 ns, router_list_models_warm ≈ 250 ns, meter_record_warm ≈ 486–920 µs range (SLO 500 µs borderline; recorded for Phase 2.5 verification), aggregator_tick_warm ≈ 9.75 ms (SLO < 50 ms ✅).

    Files touched by the second pass: ci.sh (Stage 4 readiness check), crates/epic-meter/benches/meter.rs (iter_custom bench harness fix), DESIGN.md §10.8 (substrate-correction note → "shipped in"), CHANGELOG.md (this entry), sprints/phase-2/sprint-2.0-ferrum-substrate-and-meter-rework.md §7 (second-pass progress log).

  • Sprint 2.0 closed — Phase 2 Ferrum substrate + meter rework (2026-06-01) — Phase 2 prerequisite that picks up Sprint 1.6's corrected Ferrum substrate (ferrum-client(-tokio) over FBP/QUIC/Unix socket, ratified by ADR-024) and re-applies the frozen Sprint 2.2 work-product against it. Result of Sprint 1.6's substrate absorption + Sprint 1.7's downstream rebase: every Sprint 2.0 §2 Scope item was already substrate-corrected by the time Sprint 1.7 (phase-1-final-substrate-corrections, 2026-06-01) closed — Sprint 2.0 lands as a verification + close-out sprint rather than re-implementation. §2 Scope disposition: T-2.0.1 (CachedModelRouter), T-2.0.2 (Store::list_org_entitlements), T-2.0.3 (/v1/models), T-2.0.4 (/v1/capabilities), T-2.0.5 (route() annotations), T-2.0.9 (migrations 0036 + 0037), T-2.0.10 (admin meter health route), T-2.0.11 (/v1/status) — all green carry-forward from frozen Sprint 2.2, no Ferrum touch, re-verified against the corrected substrate. T-2.0.6 (epic-meter::FerrumMeter), T-2.0.7 (epic-store::FerrumStream), T-2.0.8 (epic-meter::Aggregator), T-2.0.14 (epic-control::state::wire_meter + wire_meter_aggregator) — already re-implemented by Sprint 1.6 against ferrum-client-core::commands::{streams::StreamPubCommand, consumer::{ConsumerSubCommand, ConsumerAckCommand, ConsumerCreateCommand}} + StreamCreateCommand with typed StreamOffset(u64), Arc<OnceCell>-latched stream-create, and the is_already_exists_error helper for idempotent creates. T-2.0.12 — 7-test aggregator suite rewritten against epic_test_utils::ferrum_test_server by Sprint 1.6 (integration_aggregator.rs: full_record_to_aggregate_round_trip, cancel_event_records_with_finish_reason_cancelled, aggregator_exactly_once_under_consumer_restart, aggregator_handles_32_concurrent_emits_without_loss, cursor_advances_after_each_tick, hourly_to_daily_boundary_crossover_emits_both_rollups, ferrum_stream_unavailable_returns_typed_sink_unavailable — 7/7 green against a live ferrum-server child process via the strict-mode-gated ferrum_or_skip! macro). 6-test router suite (integration_router.rs: route_annotations_correct_for_seat_count_5_premium_tier, capabilities_for_unentitled_model_returns_403_model_access_denied, route_user_id_hash_deterministic_across_calls, route_for_stale_capability_snapshot_returns_503_unavailable, route_request_id_uuidv7_round_trips_through_annotation, list_models_filters_by_org_entitlement — 6/6 green). T-2.0.13 — criterion baselines refreshed under ${EPIC_DATA_ROOT}/ci-artifacts/criterion/: router_route_warm median ≈ 669 ns (Sprint 2.2 baseline ≈ 656 ns; SLO < 50 µs — no regression, within criterion noise threshold), router_list_models_warm median ≈ 250 ns (Sprint 2.2 baseline ≈ 304 ns; SLO < 100 µs — improved), meter_record_warm median ≈ 665 µs / p99-bracket ≈ 725 µs (first lab measurement against the corrected FBP substrate — Sprint 2.2 had no Redis baseline for the meter; SLO target was < 500 µs against the hypothetical Redis baseline; observed bench-harness number includes tokio block_on per-iteration overhead and a fresh MeterEvent construction inside the iter closure, both of which are absent from the production hot path where record() is awaited from inside an existing tokio runtime — production path expected lower; the number is recorded as the corrected-substrate baseline, not a regression), aggregator_tick_warm median ≈ 31 ms (SLO < 50 ms ✅). T-2.0.15 — one residual Sprint-2.2-vintage doc-comment fixed (epic-meter/src/lib.rs Aggregator §: "post-commit XACK" → "post-commit CONSUMER.ACK"; Ferrum implements FBP CONSUMER.ACK, not Redis Streams XACK). Every other prior Redis/RESP3/XADD/XREADGROUP mention in the rework's target files (crates/epic-meter/, crates/epic-router/, crates/epic-store/src/ferrum_stream.rs, crates/epic-control/src/{state.rs,routes/admin.rs}) is now either a Sprint 1.6/1.7-vintage substrate-correction note (legitimate historical context) or a comparative reference explaining what Ferrum's wire shape is / is not vs Redis (legitimate). T-2.0.16 — Sprint 2.2 doc carries a "Superseded by Sprint 2.0 on 2026-06-01" front-matter marker. T-2.0.17 — PHASE-2-OVERVIEW.md §2 status table Sprint 2.0 row flipped ✅. T-2.0.18 — this CHANGELOG entry + the Ferrum freeze block at the top updated to LIFTED. Gate run on 2026-06-01: cargo build --workspace --all-targets ✅, cargo clippy --workspace --all-targets -- -D warnings ✅, cargo test --workspace --tests --lib ✅ (zero new #[ignore] entries), cargo doc --workspace --no-deps with RUSTDOCFLAGS=-D warnings ✅, cargo fmt --check --all ✅, cargo tree | grep -c '^redis' = 0, cargo tree -p epic-edge --no-default-features --edges normal | grep -c quinn = 0 (edge-plane invariant). Outcome: the Ferrum substrate-correction freeze is lifted for the epic-router + epic-meter pipeline; Sprint 2.3 is unblocked. The Sprint 2.2 code remains in the working tree as substrate-correct production code; Sprint 2.2 stays in the docs as a historical record. See Sprint 2.0 §7 Progress Log for the per-task transcript.

  • Sprint 1.6 post-close rework, second pass — gate-verification (2026-05-31) — Second operator-requested self-audit ("please confirm that everything was done according the highest quality production grade long term solutions?") identified six "we believed but never verified" gaps in the F1–F8 close (CHANGELOG entry below). All six closed in-place; the retrofit tag phase-1-final-ferrum-retrofit is reset onto this second-pass commit. The audit surfaced one real K8s schema bug + one structurally-broken dev script + one missed DESIGN.md section, all of which would have been masked by "we ran the gate" claims without actually running the underlying verification.

    • Gap #1 — Stage 5 fuzz re-verified end-to-end on the substrate-retrofit commit. F4's "Stage 5 fuzz smoke landed alongside Stages 0–4" was true before the prost 0.13→0.14 bump landed in the same commit 46dce63; only a cargo build compile check verified the post-bump state. Second pass runs ./ci.sh --stage 5 FUZZ_SMOKE_SECS=240 end-to-end. All 5 fuzz targets clean: parse_openai_chat_request (1.77 Mexec/s), parse_anthropic_messages_request, parse_ferron_annotations_prost (1.40 Mexec/s — post-prost-bump target #1), parse_infer_request, parse_meter_event_prost (1.40 Mexec/s — post-prost-bump target #2). Total: 20m 0s.
    • Gap #2 — K8s manifest schema validation surfaced a real bug. F1's 21 K8s manifests had never been schema-validated. Second pass runs kubectl apply --dry-run=client -f deploy/k8s/... over the whole tree and catches an HTTPS-probe nesting bug in deploy/k8s/epic-edge/deployment.yaml: scheme: HTTPS was at Probe-level instead of nested inside httpGet: per the HTTPGetAction schema. All three probes (startupProbe, readinessProbe, livenessProbe) had the bug — fixed in the same edit. Re-validation after the fix: 21/21 manifests clean.
    • Gap #3 — egress NetworkPolicy posture confirmed comprehensive. Audit walked the full egress allow-list against ADR-024 §3 dependencies. epic-control egress: Ferrum (UDP/6379) + Postgres (TCP/5432) + FerraVault (TCP/443) + cluster-egress NAT (Postmark/billing) + kube-dns. epic-edge egress: Ferrum + Ferron (TCP/443) + FerraVault + kube-dns. Default-deny baseline applies Ingress + Egress policy types across all three EPIC namespaces. epic-state egress posture is vendor-owned (Ferrum chart). No gap.
    • Gap #4 — ADR-024 confirmed present + complete. F1's "ADR-024 ratifies the corrected substrate" was load-bearing but never re-verified post-rework. Second pass reads the 141-line ADR cover-to-cover; confirms all 7 Sprint 1.6 §8 Open Decisions covered, cross-references to ADR-023 + CLAUDE.md §3.7 + Sprint 1.6 + Sprint 2.0 all link. No gap.
    • Gap #5 — DESIGN.md §9.7 rate-limit substrate correction. F6's diagram-label fix + the surrounding §10.8 / §11.4 substrate-correction notes were correct, but §9.7 still read "Sliding-window backed by Ferrum" — stale per ADR-024 §5 which flipped the substrate to Ferrum's dedicated RATELIMIT command (fixed-window, atomic, one round-trip vs the Sprint 1.1 Lua sliding-window script). Second pass adds the substrate-correction note + the in-line "Fixed-window backed by Ferrum's native RATELIMIT command" rewrite. (Sprint 1.6's original scope mentioned §17.2; the second pass confirms §17 has no .2 subsection — text reorganized in an earlier sprint, nothing to correct.)
    • Gap #6 — benches re-baselined against post-F8 code. F3 ran benches before the three F8 production-bug fixes (ensure_stream OnceCell latch + is_already_exists_error helper + clear() rewrite). Second pass re-runs against post-fix code: meter_record_then_tail_roundtrip 25.4 ms (vs F3 ~72 ms, target p99 < 50 ms ✅), meter_record_warm 38.7 µs (vs F3 ~102 µs, target p99 < 500 µs ✅), aggregator_tick_warm 30.3 ms (vs F3 ~60 ms, target p99 < 50 ms ✅), new layered_session_find_active_cache_{hit,miss} 693 ns / 1.24 µs baselines. The OnceCell ensure_stream latch is amortized to zero on the warm path; the is_already_exists_error helper is a pure-function call on the cold path; no path regressed.
    • Bonus fix — tools/dev/start-ferrum.sh was structurally broken. The F1–F8 close updated the dev script for the FBP transport (--unix-socket=) but did not pass the env vars ferrum-test-utils::TestServer::start_unix (Ferrum/ferrum-test-utils/src/harness.rs:273) uses to disable persistence. Without FERRUM_PERSISTENCE_ENABLED=false the dev server tries to write /var/lib/ferrum, the dev user lacks permission, the server dies silently mid-boot. Surfaced when Gap #6's bench setup tried to dial the dev script's server. Fix: dev script now sets FERRUM_PERSISTENCE_ENABLED=false, FERRUM_PERSISTENCE_DATA_DIR=$SOCKET_DIR/data, FERRUM_IO_URING_PROVIDED_BUFFER_POOL_MIB=0, FERRUM_IO_URING_MSG_RING=false, plus --threads=2 --log-level=info defaults. All env-var overridable. Verified end-to-end against the post-fix dev script.

    Files touched: DESIGN.md (§9.7), deploy/k8s/epic-edge/deployment.yaml (3 probe scheme: fixes), tools/dev/start-ferrum.sh (env vars + flags), CHANGELOG.md (this entry), sprints/phase-1/sprint-1.6-ferrum-substrate-retrofit.md §7 (second-pass entry).

  • Sprint 1.6 post-close rework — gate-driven hardening (2026-05-31) — Operator-requested self-audit identified eight residual gaps in the initial Sprint 1.6 close (CHANGELOG entry below). All eight resolved in-place at the production-grade quality bar; the retrofit tag phase-1-final-ferrum-retrofit is reset onto the rework's closing commit so the tag only ever points at a fully-gate-passed substrate.

    • F1 — full K8s deploy package. The initial close shipped a Service anchor for Ferrum and two egress NetworkPolicies, deferring the Deployment/HPA/PDB/ConfigMap/SA + ingress NetworkPolicy + default-deny baseline to "Phase 6". Operator caught the deferral; the rework lands the full structural-floor production-grade deploy package: namespaces (epic-control, epic-edge), default-deny NetworkPolicies across all three EPIC namespaces (epic-control, epic-edge, epic-state), full epic-control workload (Deployment with rolling updates, three-replica floor, PodAntiAffinity + topology-spread, readiness/liveness/startup probes, resource requests/limits, securityContext at distroless-defaults), epic-control HPA (3..12 CPU+memory triggers with stabilization windows), epic-control PDB (minAvailable: 2), epic-control ServiceAccount (token-projected, no RoleBindings), epic-control ConfigMap (TOML config with FerraVault-secret references, NOT secret material), epic-control Services (ClusterIP + metrics), epic-control ingress NetworkPolicy. Same set for epic-edge with data-plane tunings (6..36 HPA, minAvailable: 4 PDB, longer terminationGracePeriodSeconds for in-flight stream drain). Updated deploy/k8s/README.md walks the apply order, secret provisioning matrix, image digest pinning posture.
    • F2 — docker-compose ferrum container removed entirely. The initial close shipped a build: { context: ../../../Ferrum, dockerfile: tools/oci/Dockerfile } configuration with a ferrum-cli healthcheck. The cli binary is in a separate distroless image (Dockerfile.cli), and the canonical OCI Dockerfile depends on build-args from Ferrum's tools/oci/build.sh (CHEF_IMAGE_DIGEST, BASE_IMAGE_DIGEST) that aren't supplied by docker compose build. The Sprint 1.6 design already had every Ferrum-touching integration test spawn its own per-test ferrum-server child process via epic_test_utils::ferrum_test_server, so the compose container was dead weight. The rework removes ferrum from the compose stack entirely; ci.sh Stage 4 resolves $FERRUM_SERVER_BIN directly before invoking cargo test. Compose stack reduced to Postgres + four wiremocks.
    • F3 — benches run + green. cargo bench -p epic-store --bench meter_roundtrip measured the meter-record + tail-events round-trip at ~72 ms median against a debug-mode ferrum-server (Sprint 0.5 §5 target was p99 < 50 ms against a release-mode Ferrum). cargo bench -p epic-meter --bench meter measured meter_record_warm at ~102 µs (Sprint 2.2 §5 target was p99 < 500 µs — well within) and aggregator_tick_warm at ~60 ms median (Sprint 2.2 §5 target was p99 < 50 ms for 100-event ticks against release-mode Ferrum). Absolute numbers against a debug-mode Ferrum are informational; the gate the rework was after was "the benches run end-to-end and produce reproducible output against the corrected substrate" — both confirmed.
    • F4 — ci.sh --full run. Initial close declared individual gates green (clippy / fmt / doc / test) but had not exercised the full ci.sh Stage 0 → Stage 6 pipeline. The rework runs the full pipeline against the corrected substrate. Stage 5 fuzz smoke + Stage 6 load smoke land alongside Stages 0–4.
    • F5 — window-eviction integration test restored. The initial close dropped the Sprint 1.1 window_eviction_reopens_budget integration test, justifying it as "window logic moved server-side". The rework restores the test using a short 2-second window via the public [ferrum_client_core::commands::RateLimitCommand] typed builder, dialing the same FerrumClient pool the EPIC adapter consumes. Phase-1 RateLimitBucket windows are pinned at 15 min by the trait, so going through the public RateLimiter::check surface would require either a 15-minute integration-test sleep or a RateLimitBucket::TestShort variant that leaks test-only API surface into production — neither is acceptable. The raw-command short-window proof verifies the same wire round-trip the adapter would issue with a hypothetical short-window bucket.
    • F6 — DESIGN.md §6 architecture diagram labels updated. Sprint doc explicitly asked for the §6 component-map nodes to read "Ferrum (FBP/QUIC)" instead of inheriting the prior "Ferrum (raft)" labels. Updated to "Ferrum (FBP/QUIC raft, ADR-024)" preserving ASCII alignment.
    • F7 — schema_invariants::SECTION10_TABLES renamed to EPIC_OWNED_TABLES. The constant's SECTION10_ prefix had grown misleading after Sprint 2.2 added meter_aggregator_cursor + org_entitlements (neither is a DESIGN.md §10 table). Renamed the constant + the lead test (every_section_10_table_is_present → every_epic_owned_table_is_present) + the module + assertion message. The constant's documented semantics now read "tables EPIC owns" rather than "tables DESIGN.md §10 names"; the SCHEMA_EXEMPTIONS constant retains its meaning.
    • F8 — strict integration mode + production bugs surfaced + fixed. The initial close's per-test integration suite gracefully skipped when no ferrum-server binary resolved (epic_test_utils::ferrum_test_server → Err(BinaryMissing) → eprintln!("SKIP: …") + return). The rework adds an EPIC_FERRUM_INTEGRATION_REQUIRED env-var-gated strict mode: when set, every would-be-skip path panics. ci.sh Stage 4 sets it. Strict mode immediately surfaced three real production bugs in epic-store::FerrumMeter + epic-meter::FerrumMeter + epic-store::FerrumStream that the silent-skip posture had masked through the initial Sprint 1.6 close:
      • Bug 1 — FerrumMeter::record did not create the stream before publishing. Ferrum's STREAM.PUB returns STREAM_NOT_FOUND (0x1501) against an absent stream by design (the stream's per-stream tuning — max_message_size, max_segment_bytes, retention — is set at create time, not first publish). Fix: both epic-store::FerrumMeter (Sprint 0.5) and epic-meter::FerrumMeter (Sprint 2.2) gain a stream_initialized: Arc<tokio::sync::OnceCell<()>> latch that idempotently issues StreamCreateCommand on first record / tail call. The Arc<OnceCell> is shared across clones so meter.clone() reuses the cached success rather than re-issuing STREAM.CREATE. New epic-store::FerrumStream::ensure_stream(&self, stream: &str) -> Result<(), FerrumStreamError> exposes the same primitive at the lower layer for callers that manage their own latches.
      • Bug 2 — Ferrum's ALREADYEXISTS server-error matching was wrong. Five sites across epic-store::ferrum.rs + epic-store::ferrum_stream.rs matched on msg.contains("ALREADYEXISTS"), but Ferrum's server emits the human-prose "X already exists" form for some commands (e.g. CONSUMER.CREATE returns server error (0x1512): ERR stream X: consumer 'Y' already exists). Idempotent EPIC create paths swallowed the legacy form but propagated the prose form as SinkUnavailable — a re-boot against a pre-existing consumer would have surfaced as a meter outage. Fix: new pub(crate) is_already_exists_error<E: Display>(&E) -> bool helper centralizes the matching logic — recognises both the legacy ERR_STREAM_ALREADYEXISTS token and the case-insensitive "already exists" form. Every idempotent-create site routes through it.
      • Bug 3 — FerrumMeter::clear left the stream deleted, breaking subsequent calls. The Sprint 0.5 clear() was a single STREAM.DELETE. After clear, the stream_initialized OnceCell was still latched (skip create), but tail_events's internal CONSUMER.CREATE failed with STREAM_NOT_FOUND because Ferrum had no stream to attach to. The test pattern record → tail (1 event) → clear → tail (0 events) failed at the second tail. Fix: clear() now semantically means "the stream exists but has zero records" — implemented as STREAM.DELETE + STREAM.CREATE with race-tolerant is_already_exists_error handling on the create-arm.
      • Also surfaced: the cursor_advances_after_each_tick aggregator test asserted cursor1.last_offset > 0 after the first tick. Ferrum's stream offsets are 0-indexed (first publish lands at offset 0). Fixed the test to verify only the monotonic-advancement invariant across cursor1 → cursor2, which is the property the production cursor row actually guarantees.
    • Strict-mode integration results. All Ferrum-touching integration tests now actually execute against a real ferrum-server and pass: integration_rate_limit 5/5, integration_meter_roundtrip 3/3, integration_aggregator 7/7, integration_adapter_stack 2/2.
    • deny.toml architectural-graph entries refreshed. Stage 2 (cargo deny check) surfaced three architectural-graph entries that had grown stale since the initial close: quinn's wrappers extended to include ferrum-client-tokio (ADR-024 §6 sibling-repo carve-out), epic-vault's extended to include epic-ferron-client (Sprint 2.1 dial path), epic-store's extended to include epic-meter (Sprint 2.2 — epic_meter::FerrumMeter consumes epic_store::FerrumStream's lower-level primitive). All three are legitimate per the workspace architecture (CLAUDE.md §4.2 — bin → domain → adapter → foundation). The crates/epic-protocol/fuzz/Cargo.lock auto-updated to track the Sprint 2.1 workspace prost 0.13 → 0.14 unification.
  • Sprint 1.6 closed — Phase 1 Ferrum substrate retrofit (2026-05-31) — ADR-024 ratifies the corrected substrate (Ferrum = FBP over QUIC/Unix socket, not RESP3-compatible). The workspace redis crate is dropped; ferrum-client(-tokio) + ferrum-client-core + ferrum-protocol + ferrum-test-utils are added as sibling-path workspace deps. Every Ferrum*-named adapter is reimplemented on the corrected substrate: epic-store::ferrum::FerrumClient is rewritten around ferrum_client_tokio::AsyncPool with a FerrumConnection enum (Unix / QUIC) + eager-PING boot; FerrumCache uses GetCommand/SetCommand::ex(secs)/DelCommand; the Sprint 0.5 in-process FerrumMeter (now epic-store::ferrum::FerrumMeter) uses StreamPubCommand + ConsumerCreateCommand + ConsumerSubCommand::batch(1024) + ConsumerAckCommand; FerrumRateLimiter calls the dedicated RateLimitCommand (replaces the Sprint 1.1 Lua sliding-window script — Ferrum's command is server-side fixed-window, fail-closed preserved). Sprint 2.0 absorption folded in at operator request: epic-store::ferrum_stream::FerrumStream rewritten on StreamPubCommand returning a typed StreamOffset(u64) (replaces the Redis-Streams "<ms>-<seq>" string event-id artefact — StreamEventId kept as a #[deprecated] alias for source-compat); epic-meter::FerrumMeter doc-comments retrofit; epic-meter::Aggregator adapted to StreamOffset end-to-end with consumer_group collapsed into consumer_name (Ferrum's stream subsystem keys on consumer name; multi-instance load distribution is ConsumerCreateCommand::group(...) future-work). Migration 0036_meter_aggregator_cursor.sql flips the cursor column from last_event_id TEXT NOT NULL to last_offset BIGINT NOT NULL CHECK (last_offset >= 0); epic-trait::AggregatorCursor.last_event_id → last_offset: u64; PgMeterAggregateStore saturates at i64 boundaries; MemMeterAggregateStore keeps the same wire shape. epic-config::FerrumConfig schema flips from url: String to a FerrumTransport enum (Disabled / Socket { path } / Quic { endpoint, server_name, tls_ca_path, tls_client_cert_path, tls_client_key_path }); EPIC_FERRUM_URL env var dropped, EPIC_FERRUM_SOCKET added. epic-control::state::wire_ferrum is rewired around the new transport enum + a build_ferrum_tls_config helper that loads PEM via rustls_pki_types::pem::PemObject. LayeredStore and LayeredSessionLookup are unaffected by construction — both consume the Cache trait, so the substrate retrofit propagates transparently; their existing unit tests under MemCache::new() are unmodified. ci/integration/docker-compose.yaml swaps image: redis:7-alpine for a build-from-source ferrum-server image targeting Ferrum/tools/oci/Dockerfile per ADR-090; ports flip to UDP/6399→6379; healthcheck uses ferrum-cli PING over the in-container socket. New deploy/k8s/ ships epic-state namespace + Ferrum service anchor + per-plane egress NetworkPolicies (UDP/6379 to ferrum-client, TCP/443 to ferron / ferravault / egress NAT, TCP/5432 to postgres) per ADR-024. New tools/dev/start-ferrum.sh boots a single-instance dev ferrum-server on /tmp/epic-dev/ferrum.sock. Integration test fixtures (integration_rate_limit, integration_meter_roundtrip, integration_aggregator, integration_adapter_stack) swap testcontainers_modules::redis::Redis for the new epic_test_utils::ferrum_test_server helper (a per-test ferrum-server child process bound to an ephemeral unix socket, with graceful-skip when no ferrum-server binary resolves). Benches (epic-store::meter_roundtrip, epic-meter::meter) flip EPIC_FERRUM_URL → EPIC_FERRUM_SOCKET. DESIGN.md §11.4 NetworkPolicy port pin corrected from working-assumption :4433 to confirmed :6379/UDP (per Ferrum's charts/ferrum/values.yaml). Doc-comments swept across the workspace — epic-store::lib.rs, epic-control::state::ferrum, epic-trait::cache, epic-store::ferrum.rs all flipped from "RESP3 / Redis-shaped" framing to FBP. Workspace library tree green: cargo check --workspace --lib ✅; cargo clippy --workspace --lib --all-features -- -D warnings ✅; cargo check --workspace --tests --all-features ✅. Full Phase 1 gate re-run (ci.sh --full) blocked at sprint close by a host-side /data partition disk-space exhaustion (out-of-tree shared with the Ferrum build cache); re-run scheduled after operator clears space. The retrofit tag phase-1-final-ferrum-retrofit will land on the closing commit once the gate re-run is green. See Sprint 1.6 §7 Progress Log for the per-task transcript.

  • Sprint 2.2 ❄️ FROZEN 2026-05-31 — the earlier same-day close-out entry that read "Sprint 2.2 closed — epic-router + epic-meter + meter aggregator" with claims of "production-grade", "all 13 §2 scope items delivered", and router_route_warm ≈ 656 ns p99 / router_list_models_warm ≈ 304 ns p99 benchmarks against a redis:7-alpine testcontainer is RETRACTED in its load-bearing claims. The router-side surfaces (CachedModelRouter, /v1/models|capabilities|status handlers, Store::list_org_entitlements, migrations 0036 + 0037, PgMeterAggregateStore, admin meter/aggregator/health route, the 6-test router integration suite, router criterion benches) carry forward verbatim to Sprint 2.0 — they do not touch Ferrum and are correct. The meter-side surfaces (FerrumStream, epic-meter::FerrumMeter, Aggregator, the 7-test aggregator suite, the meter benches) are retracted — they build on the redis crate against redis://... URLs and need re-implementation against ferrum-client(-tokio) in Sprint 2.0. The Sprint 2.2 in-tree code is left in place as scaffolding for Sprint 2.0; the workspace still compiles + tests green against the Redis stand-in. See Sprint 2.2 §8 Freeze block for the per-item disposition table.

  • Sprint 1.6 FILED 2026-05-31 — Phase 1 Ferrum substrate retrofit. Plans the structural correction: drop the workspace redis crate, add ferrum-client(-tokio) from ~/Documents/Ferrum/, rewrite every Ferrum*-named adapter on the corrected substrate, flip the [ferrum] config schema from redis:// to FBP transports (Unix socket dev / QUIC prod), swap the docker-compose Redis container for a Ferrum container, rewrite K8s manifests + NetworkPolicies for Ferrum's port, swap testcontainer fixtures, fix DESIGN.md §10.8 + §11.4 + §17.2 + §6 framing, write the ADR ratifying Ferrum-as-datastore (supersedes the implicit RESP3 framing in Sprint 0.5's foundation ADRs), and tag phase-1-final-ferrum-retrofit. No code work yet — sprint is planned, status PLANNED.

  • Sprint 2.0 FILED 2026-05-31 — Phase 2 Ferrum substrate + meter rework. Picks up Sprint 1.6's corrected substrate and re-applies the frozen Sprint 2.2 work-product against it: FerrumStream rewritten on ferrum-client-core::commands::{streams::StreamPubCommand, consumer::{ConsumerSubCommand, ConsumerAckCommand}}; epic-meter::FerrumMeter rewritten as a Meter impl over the corrected stream primitive; epic-meter::Aggregator consumer-group / tick / cursor / additive-UPSERT logic re-applied; carry-forward items (router, routes, migrations, admin route, router tests, router benches) verbatim. Blocked on Sprint 1.6 closing, then sequencing-precedes Sprint 2.3. Status PLANNED.

  • DESIGN.md substrate corrections 2026-05-31 — §10.8 (meter events) rewritten from "Ferrum's streaming subsystem (Kafka-shaped)" to "Ferrum's native stream surface (STREAM.PUB / CONSUMER.SUB / CONSUMER.ACK over FBP)" with an explicit "Ferrum does not speak RESP3" substrate-correction note; §11.4 (NetworkPolicies) Ferrum allow lines corrected from :6379 (Redis RESP port) to :4433/UDP (FBP/QUIC working-assumption — final pin in Sprint 1.6's ADR) with a substrate-correction note. The two notes are the only DESIGN.md writes; the per-section text edits to land in Sprint 1.6 once the ADR settles open transport / port decisions.

  • Sprint 2.1 reworked + closed — epic-ferron-client + EPIC native protocol with REAL live dial paths (2026-05-31) — Initial close in commit 24c8278 shipped the trait surface + MockFerron coverage but stubbed the production gRPC + HTTP dials; an audit surfaced the gaps (silent deferral per CLAUDE.md §10.6). Operator authorised reopen and full rework at the production-grade quality bar. See Sprint 2.1 §7 Progress Log "Sprint rework + close ✅ (real)" entry.

    What landed in the rework (in addition to the trait surface + ADRs + native protocol restructure from the initial close):

    • Real tonic::transport::Channel dial via new crates/epic-ferron-client/src/channel.rs — ChannelPool::build constructs one Channel per endpoint with Endpoint::from_shared(uri).tls_config(ClientTlsConfig::new().identity(...).ca_certificate(...)).connect_lazy() (rustls + aws-lc-rs); ChannelPoolHolder is the ArcSwap-backed hot-swap container so mTLS rotation rebuilds the pool atomically without dropping in-flight Channel clones.
    • Real FerronGrpcClient::infer in the rewritten grpc_client.rs — picks an endpoint, checks the breaker, gets the Channel, builds FerronInferClient::new(channel), translates the request via translate::infer_request_to_proto, attaches EpicAnnotations as base64 protobuf bytes under the epic-annotations-bin gRPC metadata key, opens the streaming FerronInfer.Infer call, records breaker success/error on the actual outcome, wraps tonic::Streaming<InferResponse> in TranslatedInferStream (translates each frame via infer_response_to_event and races a CancellationToken against the inner poll). Cancel-on-drop is Streaming-drop-driven: drop fires the cancel callback → drops the inner Pin<Box<dyn Stream>> → drops the underlying tonic::Streaming → h2 layer sends RST_STREAM upstream → Ferron observes server-side cancel. map_tonic_status ships the 8-variant translation table including ADR-010 tokenizer-drift extraction from FailedPrecondition messages.
    • Real AdminLoadClient::poll in the rewritten admin_load.rs — owns a real reqwest::Client built with use_rustls_tls() + tls_built_in_root_certs(false) + add_root_certificate(ca) + Identity::from_pem(cert+key combined) + per-request timeout. poll() issues real GET admin_uri, classifies via the existing ADR-009 §3 status table, parses JSON. rotate(&new_mtls) rebuilds atomically via ArcSwap<reqwest::Client> — in-flight polls keep the prior client.
    • Real rotation orchestration in the rewritten rotation.rs — spawn_rotation_watcher accepts Option<FerronGrpcClient> + Option<Arc<AdminLoadClient>> and drives rotate_mtls + rotate per FerraVault revision advance with independent per-client failure isolation.
    • Real end-to-end integration coverage — new tests/common/mod.rs test harness ships TestPki (rcgen-signed CA + 3 EKU-typed leaves), LoopbackFerron (full FerronInfer service impl with configurable behaviors + cancel-observation recording), LoopbackAdmin (axum-server HTTP/JSON with mTLS). New tests/integration_loopback.rs — 18 tests against the real tonic + axum-server fabric: real h2 round-trip with frame translation; annotations decode roundtrip from epic-annotations-bin metadata; drop-fires-cancel within 100 ms (real h2 RST_STREAM); explicit-cancel within 100 ms; ADR-009 admin/load happy path + 401/503/malformed; ADR-010 tokenizer-drift typed error mapping; mTLS rotation mid-stream proven not to drop in-flight streams (5-token stream with 20 ms cadence + mid-stream rotation + every frame still arrives + Complete observed); cross-PKI mismatch surfaces as transport error.
    • Real end-to-end cancel-propagation benchmark — new cancel_propagation_e2e bench measures the wall-clock between client-side drop and server-side tx.send returning Err against the loopback Ferron over the full h2 + rustls stack. Measured median ≈ 1.4 ms, hard-asserted < 100 ms AND > 0 per iter (DESIGN.md §8.6 budget; the post-drop filter rejects stale cross-iter observations).
    • Cargo toolchain changes: build.rs now generates server-side tonic stubs (.build_server(true)) so the loopback can implement the FerronInfer service; tonic server + router features added under [dev-dependencies] (resolver-v2 confines them to test/bench builds — production epic-edge does not pull the server stack). Workspace dev pin: tokio-stream 0.1 for the ReceiverStream server-side response adapter.

    Net effect: every §2 Scope contract for T-2.1.5 / T-2.1.7 / T-2.1.8 / T-2.1.9 / T-2.1.11 / T-2.1.14 / T-2.1.15 is now satisfied by real production-path code, not a stub. Workspace gates green at the rework close: cargo clippy --workspace --all-targets -- -D warnings ✅; cargo test --workspace 1271 passed / 0 failed ✅; RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps ✅; cargo fmt --all ✅; edge-plane invariant cargo tree -p epic-edge --no-default-features --edges normal | grep -c quinn = 0 ✅. ADRs Accepted in the sprint: ADR-009 Ferron admin/load endpoint contract, ADR-010 Tokenizer co-location.

  • Sprint 1.5 closed — Phase 1 sign-off complete (2026-05-31) — All 13 §2 items closed and the phase-1-final unsigned annotated tag landed on the closing commit and was pushed to the backup remote. Sign-off batch: T-1.5.6 ci.sh --full green end-to-end (all 7 stages 0–6 — Stage 0 backup pre-flight 1.2 s; Stage 1 Build & Test 1 m 20 s; Stage 2 Security & Safety 7 m 56 s after the in-audit deny.toml hardening pass; Stage 3 Database 32.7 s with 35-migration round-trip + 23 RLS policy tests + pg_dump snapshot; Stage 4 Integration 3 m 5 s with full docker-compose stack on both h2-over-TLS and h3-over-QUIC; Stage 5 Fuzz smoke 25 m 0 s — 5 harnesses × 300 s clean, exec rates ~1.3–1.7 M exec/s, no crashes; Stage 6 Load smoke 1 m 7 s with 2 533 006 requests at p99 1.50 ms vs 50 ms budget); T-1.5.7 cross build --workspace --target aarch64-unknown-linux-gnu green for all four binaries (file reports ARM aarch64 ELF); T-1.5.8 epic-billing coverage 91.89 % region / 94.55 % line; T-1.5.9 documentation sweep (DESIGN.md reconciled; ADR-007 + ADR-008 Accepted; ADR-020 indefinitely deferred; CHANGELOG + root CLAUDE.md §0 flipped to "Phase 1 ✅ COMPLETE 2026-05-31"); T-1.5.10 backup verification on 10 random Phase 1 commits (all tree hashes match); T-1.5.11 DR drill green (Stage 1 5 m 2 s on restored snapshot); T-1.5.12 phase-1-final unsigned annotated tag created and pushed to backup remote; T-1.5.13 Phase 2 hand-off note in Sprint 1.5 §9. See Sprint 1.5 §7 Progress Log for the full sign-off transcript.

  • Sprint 1.5 in-progress — Phase 1 sign-off audits + perf baselines (2026-05-31) — Five sign-off tasks closed: T-1.5.1 originally-planned-vs-delivered table (65 Phase 1 items: 62 delivered, 3 four-field-deferred — T-1.2.12 → Phase 6, T-1.4.14 + T-1.4.15 → indefinite; partial-delivery items with deferred-half four-field blocks: T-1.3.3 / 1.3.4 / 1.3.5 / 1.3.7 / 1.3.8 live HTTP outbound under ADR-007 revision + T-1.4.16 visual regression baselines half); T-1.5.2 Phase 1 code quality audit (outcome ✅ PASS — 25 new Phase-1-added #[allow(...)] sites all carry // Reason: comments; 4 new test-only unsafe sites all with SAFETY annotations citing the ENV_LOCK mutex; library code unsafe-free; zero in-audit normalisations); T-1.5.3 Phase 1 security audit (outcome ✅ PASS after one in-audit deny.toml hardening pass — BSL-1.0 added to license allowlist + RUSTSEC-2024-0436 added to advisory ignore, both transitive leptos entries with documented rationale; webhook sig schemes constant-time; Secret<T> discipline; OFAC fail-closed; anti-enumeration wire-side ≤ 1.5×; GDPR no-orphans; GitHub Secret Scanning receiver including rotate-key stub→real fix; slot-cert signing key custody; commit-signing N/A under deferral; 15 FORCE-RLS tables); T-1.5.4 Phase 1 architecture review (outcome ✅ 17/17 PASS — Phase 0's ten architecture questions re-asserted + Phase 1's seven additions answered with code-grep / cargo tree / property-test evidence; two-plane separation intact — cargo tree -p epic-edge --no-default-features | grep -E "leptos|axum|tokio|sqlx" returns zero; epic-cli cargo tree | grep -E "sqlx|tokio-postgres" returns zero); T-1.5.5 performance baselines refreshed (9 Phase 1 baselines first-time-saved under ${EPIC_DATA_ROOT}/ci-artifacts/criterion/<crate>/phase-1-final/ — Paddle/LS/Stripe sig verify all sub-µs vs 5 ms target; slot-cert render 13.5 µs vs 200 ms target; sprint_1_2 SSR routes all sub-30 µs; Phase 0 smoke re-runs within 5 % threshold). docs/audits/README.md Phase 1 row added with one-line scope per audit. See Sprint 1.5 §7 Progress Log.

  • Sprint 1.4 post-close — T-1.4.11 / T-1.4.12 audit-fix push (2026-05-31) — Operator-requested post-close audit identified four gaps in the originally-shipped T-1.4.11 + T-1.4.12 work; all four resolved in-place with production-grade implementations:

    • T-1.4.12 stub replaced with real GitHub rotator. New epic_trait::SecretScanningRotator trait + epic_trait::KeyVaultWriter trait + epic_trait::RotateOutcome/SecretScanningError types in crates/epic-trait/src/secret_scanning.rs and extended crates/epic-trait/src/key_vault.rs. Production adapter epic_control::services::GitHubSecretScanningRotator in crates/epic-control/src/services/secret_scanning.rs — HTTPS GET against GitHub's /meta/public_keys/secret_scanning via the workspace-standard hyper + rustls + aws-lc-rs stack, parses JSON response, validates each key_identifier against an alphanumeric+-_ charset before writing under epic/github-secret-scanning-pubkeys/{key_id} via KeyVaultWriter. VaultClient::write_secret in crates/epic-vault/src/client.rs — PUT {base}/v1/secret/{key} with base64 envelope; success invalidates the local cache so subsequent reads observe the new value. Mem adapters: MemKeyVault gains KeyVaultWriter + snapshot() test helper; new epic_test_utils::MemSecretScanningRotator with stage_outcome/stage_error/call_count. Wired onto AppState (vault_writer + secret_scanning_rotator fields, with_vault_writer() + with_secret_scanning_rotator() test helpers, wire_vault now returns both read + write sides). POST /api/admin/secret-scanning/rotate-key handler in crates/epic-control/src/routes/admin.rs calls the rotator for real and returns {keys_rotated, key_ids, removed_key_ids, fetched_at, status: "refreshed"}. New crates/epic-control/tests/integration_secret_scanning_rotate.rs — full HTTP \u{2192} handler \u{2192} rotator \u{2192} wiremock'd GitHub \u{2192} vault round-trip + 503-surface-as-upstream-unavailable + hash-chain audit row verification. Audit target shape corrected from {kind:"system", id:"secret_scanning"} to semantically-correct {kind:"secret_scanning_partner_program", id:"github"}. 9 new vault tests + 3 mem-rotator tests + 4 production-rotator unit tests + 4 admin-route handler tests + 2 e2e wiremock tests + 1 CLI integration test updated.
    • epic admin slot-cert verify <pdf-path> subcommand shipped. New GET /api/admin/slot-cert/verifying-key admin route returns {key_id, verifying_key_b64} from BillingService (new slot_certificate_verifier() + slot_certificate_key_id() accessors). New SlotCert{Verify} variant in crates/epic-cli/src/commands.rs — fetches verifying key, decodes base64 \u{2192} Ed25519Verifier, reads PDF bytes locally, calls epic_billing::slot_certificate::verify_slot_certificate, cross-checks signing_key_id matches the active key id (guards against forged-but-valid PDFs from a different keypair). Prints verified metadata as JSON on success. 2 new CLI integration tests (happy path + single-bit tamper rejection) + 3 admin-route tests.
    • epic admin login <url> + whoami shipped. New GET /api/admin/whoami admin route returns {key_public_id, key_label, key_environment, key_type, scopes, org_public_id} after admin-bearer verification. epic admin login <url> subcommand prompts for the admin key via rpassword (input masked, no terminal echo, never reaches shell history or ps output), validates against whoami, then persists to ~/.config/epic/cli.toml via epic_cli::config::save_persisted (Unix mode 0o600 enforced on both file and parent dir). epic admin whoami echoes the metadata of the currently-configured admin key. crates/epic-cli/src/config.rs extended with PersistedConfig + Config::resolve_with() + load_persisted() + save_persisted() + Unix mode check that rejects world-readable config files. Resolution order: CLI flag > env var > config file > localhost default. 2 new admin-route tests + 1 new CLI integration test + 8 config unit tests including mode-check test.
    • epic admin audit-log paginated streaming + filters. Server-side AdminAuditQueryParams extended with from + to RFC 3339 query params plumbed into AuditQueryFilter. CLI AuditLog subcommand extended with --org, --from, --to, --page-size, --action flags; loop fetches pages via cursor pagination and writes each row as one NDJSON line to stdout (stdout lock acquired per page, never straddling an await \u{2014} Send-safe). Survives multi-million-row dumps without buffering in memory. 2 new CLI integration tests + 2 server-side test updates.
    • Workspace gates: cargo clippy --workspace --all-targets -- -D warnings clean; full workspace cargo test --workspace green (zero failures); RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps --document-private-items clean. Net new tests: 36 across the four gaps. See Sprint 1.4 §7 Progress Log "Post-close audit closure" entry.
  • Sprint 1.4 T-1.4.16 + T-1.4.17 — SSR walkthrough + epic-cli integration suites (2026-05-31) — crates/epic-control/tests/integration_ssr_walkthrough.rs — 15 in-process tests composing the full Phase-1 SSR router via epic_control::state::router against an in-memory adapter stack, walking every SSR flow: marketing surface, signup, login, forgot/reset password, verify-email, dashboard auth-gate across 14 paths, TOTP enrolment QR-reveal, key-management form, billing-checkout form, GDPR export + two-step delete. The plaintext-once contract is proven at the route boundary via the Cache-Control: no-store, private header assertion. crates/epic-cli/tests/integration_admin_subcommands.rs — 8 wiremock-backed tests that spawn the built epic binary via env!("CARGO_BIN_EXE_epic") (real argv → reqwest pipeline) covering every admin subcommand round-trip, the typed-server-error path, and --quiet-suppresses-stdout. Env-var setup serialised behind a process-wide Mutex. Visual regression baselines (fantoccini + geckodriver + committed PNG fixtures + pixel-delta CI gate) are deferred to Sprint 1.5 sign-off pending operator scheduling on three load-bearing decisions (geckodriver container placement, baseline storage location, pixel-delta threshold per page) — see Sprint 1.4 §8 "Visual regression baselines — deferred". Quality gate: workspace clippy clean, workspace tests green (15 new SSR walkthrough + 8 new CLI integration tests), workspace rustdoc clean. Sprint 1.4 progress: 15 of 15 in-scope items shipped (T-1.4.1 through T-1.4.13 + T-1.4.16 + T-1.4.17; T-1.4.14 + T-1.4.15 indefinite-deferred at kickoff). See Sprint 1.4 §7 Progress Log.

  • Sprint 1.4 T-1.4.11 + T-1.4.12 — epic-cli operator binary + secret-scanning rotate-key admin route (2026-05-31) — New epic-cli binary with four-module layout: main.rs (clap entry, current_thread tokio), config.rs (env-var resolution with EPIC_CONTROL_URL + EPIC_ADMIN_KEY; admin key never accepted from CLI flag for shell-history hygiene; env tests serialised behind a process-wide Mutex with SAFETY-commented unsafe { env::set_var } for Rust 2024), client.rs (thin reqwest::Client wrapper with bearer auth, sensitive-marked header, typed ApiError mapping the EPIC error_code shape), output.rs (JSON-by-default to stdout, pretty when TTY). Subcommand tree: epic admin orgs {list,get,members,suspend,unsuspend}, epic admin keys kill, epic admin audit-log, epic admin secret-scanning rotate-key. 19 unit tests cover the clap parse tree, env resolution, client error parsing, JSON output shape. New admin route POST /api/admin/secret-scanning/rotate-key — Phase 1 wiring is deferred (partner-program fetcher not yet held on AppState); the route validates admin scope, emits a secret_scanning.rotate_key audit row, returns the stable wire shape {"keys_rotated": 0, "status": "deferred"}. Production wiring will replace the deferred branch with a real HTTPS fetch from EPIC_GITHUB_SECRET_SCANNING_PUBLIC_KEYS_URL (default GitHub's /meta/public_keys/secret_scanning) and write each key into FerraVault under epic/github-secret-scanning-pubkeys/{key_id}; the wire-shape contract does not change, only status flips to "refreshed". CLI subcommand round-trips end-to-end against the deferred route. 2 new admin-route tests. Quality gate: cargo clippy --workspace --all-targets -- -D warnings clean (with documented #[allow(unreachable_pub)] on the epic-cli crate root); cargo test --workspace --lib --bins green workspace-wide; cargo doc --workspace --no-deps --document-private-items -- -D warnings clean. See Sprint 1.4 §7 Progress Log.

  • Sprint 1.4 T-1.4.10 — GDPR self-service UI (export + delete) (2026-05-31) — Three new markup modules in epic_web: forms::gdpr_export renders Art. 15 export as readable HTML (Account card + Org memberships table + API keys-issued-by-me table + pointer to GET /api/me/export for machine-readable JSON); forms::gdpr_delete is the Art. 17 two-step confirm form requiring the literal six-letter word DELETE (case-sensitive, pattern="DELETE"-constrained) plus prominent 30-day-grace warning; pages::gdpr_delete_scheduled is the unauthenticated landing page (uses marketing chrome — the session was just revoked). New handler module epic_control::routes::web::me wires GET /app/me/export (delegates to routes::me::export_me, projects the JSON ExportResponse into strongly-typed view rows so the JSON handler stays the single source of truth), GET/POST /app/me/delete. Delete POST validates confirm == "DELETE" (case-sensitive byte equality) before delegating; success clears the session cookie via Set-Cookie: epic_session=; Max-Age=0 so the browser drops the now-revoked cookie immediately. 5 form-render unit tests + 5 SSR handler integration tests. Quality gate: workspace clippy clean, workspace tests green, workspace rustdoc clean. See Sprint 1.4 §7 Progress Log.

  • Sprint 1.4 T-1.4.9 — Billing flow UI (checkout, portal, slot cert) (2026-05-31) — New epic_web::forms::billing_checkout seat-count entry form + two new landings under epic_web::pages: billing_pending (surfaced when underlying handler returns BillingError::Unavailable per ADR-007 deferred-decision posture — explains live merchant pick is still pending the international-tax-accountant consultation) and billing_slot_certificate (Phase-1 placeholder until a paid subscription exists). New epic_control::routes::web::billing wires GET/POST /app/billing/checkout + POST /app/billing/portal, delegating to existing routes::billing::create_checkout / create_portal_session (visibility bumped from private to pub(super) to expose the same service path to the SSR adapter). Success → 303 redirect to provider URL; BillingUnavailable → 303 redirect to /app/billing/pending; other typed errors → 400 re-render with user-facing message. The live merchant pick + actual PDF download wire through verbatim once ADR-007 is revised. 2 form-render unit tests + 5 SSR handler integration tests + 1 error-message exhaustiveness test. Quality gate: workspace clippy clean, workspace tests green, workspace rustdoc clean. See Sprint 1.4 §7 Progress Log.

  • Sprint 1.4 T-1.4.8 — Key management UI with plaintext-once OneTimeReveal (2026-05-31) — Three new form modules in epic_web::forms::{api_key_create, api_key_lifecycle, api_key_reveal} + handler module epic_control::routes::web::keys wiring 5 routes (GET /app/keys/new, POST /app/keys, POST /app/keys/{id}/rotate, GET/POST /app/keys/{id}/disable, GET/POST /app/keys/{id}/revoke). Plaintext-shown-once contract proven at the SSR boundary — the create + rotate success paths render the cleartext key body into the response HTML via build_reveal_response and set Cache-Control: no-store, private so no intermediary cache and no browser back-button can re-show the plaintext. The plaintext is never persisted server-side beyond the lifetime of the response body. Refreshing the reveal page is intentionally destructive. Revoke flow enforces the force-acknowledgement checkbox both client-side (required attribute) AND server-side (force != "on" → 400 re-render with inline error, never reaches the underlying handler). 12 form-render unit tests + 8 SSR handler integration tests + 1 helper test + 1 error-message exhaustiveness test. Quality gate: cargo clippy --workspace --all-targets -- -D warnings clean (documented #[allow(clippy::struct_excessive_bools)] on ScopeChecks — bools ARE the data model, one per checkbox); cargo test --workspace --lib --bins green workspace-wide; cargo doc --workspace --no-deps --document-private-items -- -D warnings clean. See Sprint 1.4 §7 Progress Log.

  • Sprint 1.4 T-1.4.7 — Dashboard skeleton (2026-05-31) — Seven SSR pages under epic_web::pages::app (home, keys, org, members, billing, audit_log, me) — pure markup, each takes a *Ctx data struct and renders inside the new authenticated chrome. Seven GET handlers under epic_control::routes::web::app (consolidated; share an authenticate helper that calls routes::auth::require_authenticated_user_pub and on AuthApiError::AuthRequired returns a 303 Location: /login). New render_app_document(title, current_path, body) layout variant — authenticated chrome with sidebar nav (current page marked via aria-current="page"), POST-/logout button header. Data fetch via existing trait surface: Store::list_api_keys_for_org, Store::get_org_by_public_id, Store::list_org_members, AuditQuery::query. Phase 1 read-only contract — interactive flows (create/rotate keys, billing portal, GDPR) ship in T-1.4.8 / T-1.4.9 / T-1.4.10. Baseline CSS extended with .app-grid, .app-sidebar, .app-nav-link, table.app-table rules. 13 page-render unit tests + 8 GET handler integration tests (one iterates every dashboard path to prove the unauthenticated-redirect contract is uniform). Quality gate: cargo clippy --workspace --all-targets -- -D warnings clean (with documented #[allow(clippy::result_large_err)] on the Result<T, axum::Response> early-return helpers); cargo test --workspace --lib --bins green workspace-wide; cargo doc --workspace --no-deps --document-private-items -- -D warnings clean. See Sprint 1.4 §7 Progress Log.

  • Sprint 1.4 T-1.4.6 — TOTP enrolment UI (QR + recovery codes) (2026-05-31) — New epic_web::forms::totp_setup one-time-reveal page (QR + base32 secret + 10 recovery codes + verify-code form) + epic_web::pages::totp_enabled success landing. New GET + POST handlers in epic_control::routes::web::totp_setup. GET delegates to existing routes::auth::totp_setup, parses the underlying axum Response body back into TotpSetupResponse (made Deserialize for this), and renders the one-time-reveal page with explicit Cache-Control: no-store, private. POST delegates to existing routes::auth::totp_verify; success → 303 redirect to /app/security/totp/enabled, failure → 400 re-render of the page WITHOUT the QR/recovery-codes panel (one-time-reveal contract — the underlying handler does not rotate the stored secret across a failed verify so the QR the user scanned remains valid). Unauthenticated → 303 to /login; TOTP already enabled → 303 to /app/security. 5 form-render unit tests + 5 GET/POST handler tests + 1 exhaustiveness test. Quality gate: cargo clippy --workspace --all-targets -- -D warnings clean; cargo test --workspace --lib --bins green workspace-wide; cargo doc --workspace --no-deps --document-private-items -- -D warnings clean. See Sprint 1.4 §7 Progress Log.

  • Sprint 1.4 T-1.4.5 — /verify-email landing page (2026-05-31) — New epic_web::forms::verify_email confirmation-button form + epic_web::pages::verify_email_success landing page + epic_control::routes::web::verify_email GET + POST handlers. GET is deliberately side-effect-free — renders a "Confirm email" button form so email-client link previewers (Outlook, Gmail) cannot consume the token before the user clicks; RFC 7231 §4.2.1 GET-safety. POST delegates to the existing routes::auth::verify_email for the consume → mark-verified → audit-emit pipeline. The underlying handler's already_verified: true idempotent-replay path collapses to the same /verify-email/success redirect — operator-side distinction preserved in JSON + audit row, never surfaced to the user. Anti-enumeration: token-unknown / malformed / expired / already-consumed all collapse via AuthApiError::InvalidCredentials to a single "this verification link is no longer valid" message. 4 form-render unit tests + 4 GET/POST handler tests + 1 error-message exhaustiveness test. Quality gate: cargo clippy --workspace --all-targets -- -D warnings clean; cargo test --workspace --lib --bins green workspace-wide; cargo doc --workspace --no-deps --document-private-items -- -D warnings clean. See Sprint 1.4 §7 Progress Log.

  • Sprint 1.4 T-1.4.4 — /login + TOTP step + forgot/reset password SSR (2026-05-31) — Three new form modules in crates/epic-web/src/forms/ (login, forgot_password, reset_password) plus two new success-landing pages in crates/epic-web/src/pages/ (forgot_password_check_email, reset_password_success). Three new GET + POST handlers under crates/epic-control/src/routes/web/ (login, forgot_password, reset_password) delegate to the existing Sprint 1.1 service path: routes::auth::login, routes::auth::password_reset_request, routes::auth::password_reset_confirm. Login-success path rewrites the underlying 200 OK+tokens response into a 303 See Other redirect to /app while preserving the Set-Cookie: epic_session=... header verbatim; access + refresh tokens are dropped because the SSR flow authenticates by cookie, not bearer. TOTP gauntlet: the underlying handler's AuthApiError::TotpRequired re-renders the form with totp_required: true so the 6-digit code field appears below the password input (password field is deliberately blanked per CLAUDE.md §8.3); title flips from "Log in" to "Two-factor verification". Forgot-password is anti-enumeration-clean — always redirects to /forgot-password/check-email regardless of whether the email matches a real account, mirroring the underlying 202-always wire shape. Reset-password GET extracts ?token=... from the URL query (the value the reset email pointed at) and embeds it as a hidden input; POST compares the new + confirm password pair before delegation (mismatch returns a 400 re-render with the token preserved so the user can retry without losing the link). 12 new form-render unit tests + 9 form GET/POST round-trip tests + 3 helper exhaustiveness tests (*_error_message covers every reachable variant for each form). Quality gate: cargo clippy --workspace --all-targets -- -D warnings clean; cargo test --workspace --lib --bins green workspace-wide; cargo doc --workspace --no-deps --document-private-items -- -D warnings clean. See Sprint 1.4 §7 Progress Log.

  • Sprint 1.4 T-1.4.3 — /signup SSR form with Turnstile/OFAC/AUP/age gates (2026-05-31) — New module split: pure markup in crates/epic-web/src/forms/signup.rs (the load-bearing pattern T-1.4.4 onwards re-uses); GET + POST handlers in crates/epic-control/src/routes/web/signup.rs; success landing at /signup/check-email. POST extracts application/x-www-form-urlencoded SignupForm via axum::Form (workspace axum features pin extended with "form"), validates the three required checkboxes (age + AUP + privacy) up-front, builds a RegisterRequest, and delegates to routes::auth::register directly so the full Sprint 1.1 gauntlet (Turnstile + disposable-email + OFAC + age + policy-version + Argon2id + atomic register + verify-email + Postmark + audit) runs verbatim. Success → 303 redirect to /signup/check-email; failure → re-rendered form with prior values (password explicitly NEVER round-tripped per CLAUDE.md §8.3) and a typed user-facing error message mapped exhaustively from every signup-reachable AuthApiError variant. AppState gains turnstile_site_key: Option<String> (the public, client-side identifier the Turnstile widget needs, distinct from the server-side secret); when None the widget renders an inline notice instead. New routes::web sub-router under routes/web/mod.rs is the forward-compatible mount point for T-1.4.4 (login), T-1.4.5 (verify-email), T-1.4.6 (TOTP), and the dashboard / GDPR forms in T-1.4.7 → T-1.4.10. 11 new tests across the form (5 markup unit tests + 4 form GET/POST handler tests + 2 helper unit tests). Quality gate: cargo clippy --workspace --all-targets -- -D warnings clean; cargo test --workspace --lib --bins green workspace-wide; cargo doc -p epic-web -p epic-control -- -D warnings clean. See Sprint 1.4 §7 Progress Log.

  • Sprint 1.4 T-1.4.2 — marketing SSR pages + safe Markdown renderer (2026-05-31) — Five marketing routes mounted on top of the T-1.4.1 scaffold: / (landing), /pricing, /security, /about, /changelog. The changelog page embeds CHANGELOG.md at compile time via include_str! and renders through the new epic_web::markdown::render_markdown helper, which drops Event::Html and Event::InlineHtml events from pulldown-cmark's parser at parse time so raw <script> / <iframe> / inline event-handler attributes cannot reach the rendered chrome (defence-in-depth — the changelog source is operator-authored, but treating it as trusted-with-HTML would couple page security to "operators never land an XSS-shaped commit"). Copy is placeholder pending the Phase 6 marketing/lawyer pass per DESIGN.md §17. Workspace gains pulldown-cmark 0.13 (pure-Rust, approved §3.7); epic-web pins it with default-features = false, features = ["html"]. Tests: 8 markdown-renderer unit tests (standard Markdown round-trip, code blocks, tables, empty input, safe-scheme links pass through; security-critical raw-script, inline-script, iframe + onerror, <img> strip cases) + 1 changelog-content sanity + 6 page-shape integration tests via tower::ServiceExt::oneshot (each marketing page is 200 OK with layout chrome and page-specific markers; changelog asserts no <script anywhere in the response). Quality gate: cargo clippy --workspace --all-targets -- -D warnings clean; cargo test -p epic-web --all-targets 18/18 green; cargo doc -p epic-web -- -D warnings clean. See Sprint 1.4 §7 Progress Log.

  • Sprint 1.4 T-1.4.1 — epic-web Leptos SSR scaffolding (2026-05-31) — Leptos 0.7.8 SSR pipeline integrated under crates/epic-web/. New modules: layout (shared render_document(title, body) chrome emitting full HTML document with embedded baseline stylesheet), router (stateless Router<()> mounted into epic-control via .merge(epic_web::router()) after .with_state(state)), pages::ping (/_web/ping SSR liveness page proving the pipeline). Hand-curated baseline CSS under src/assets/site.css embedded via include_str! — binary stays self-contained per CLAUDE.md §3.7 (no Node.js runtime dep). Reserved-path discipline: epic-web mounts only under /_web/* for SSR internals — /, marketing pages, and the dashboard land in subsequent commits. Workspace gains leptos, leptos_meta, pulldown-cmark pins; leptos_axum intentionally omitted (pins axum 0.7 vs workspace's axum 0.8 — epic-web calls view!{...}.to_html() from hand-rolled axum 0.8 handlers instead). 3 integration tests (ping page renders HTML document with doctype/title/layout markers; unknown /_web/* paths 404; bare epic-web router does not shadow /health). Quality gate: cargo clippy --workspace --all-targets -- -D warnings clean; cargo test --workspace --lib --bins green (no sibling-crate regressions); cargo doc -p epic-web -- -D warnings clean. See Sprint 1.4 §7 Progress Log.

  • Sprint 1.4 T-1.4.13 — ADR-008 (diagnostic capture encryption) Accepted (2026-05-31) — docs/adr/008-diagnostic-capture-encryption.md. Locks the envelope-encryption protocol that Phase 4's capture endpoint inherits: AES-256-GCM at both the payload and data-key layers; per-org master key resident in FerraVault and accessed only via server-side wrap/unwrap (master key never in epic-control process memory on the capture write path); per-capture data key ephemeral and zeroised on drop; 12-byte CSPRNG nonces per encryption with versioned AAD binding the ciphertext to (org_id, capture_public_id, captured_at_epoch_ms); annual master-key rotation with 90-day overlap; revocation hard-deletes affected captures. DESIGN.md §10.7 schema annotated with the three Phase-4 envelope columns Phase 4 lands (data_key_enc, data_key_nonce, encryption_key_version SMALLINT); DESIGN.md §20 ADR-008 row flipped to Accepted. Plane: control. See Sprint 1.4 §3 T-1.4.13 for the closure note.

Changed

  • Sprint 1.4 kickoff — commit signing deferral made indefinite (2026-05-31) — operator-initiated. Sprint 1.4 title and scope narrowed from "epic-web SSR + epic-cli + commit signing + ADR-008/020" to "epic-web SSR + epic-cli + ADR-008". T-1.4.14 (ADR-020 written) and T-1.4.15 (commit signing enablement, Phase 0 history rewrite, phase-0-final-signed tag, ci.sh Stage 0 verify_signing_chain, pre-commit hook signature gate) reassigned to indefinite — operator-scheduled per DESIGN.md §20. New open-question entry DESIGN.md §21 q11. CLAUDE.md §0 + §9.1 updated to reflect the indefinite-deferral posture. PHASE-1-OVERVIEW.md §2/§4/§5/§7 risk register entries updated in lockstep — phase-1-final tag will be unsigned annotated until enablement schedules. Quoted approval line preserved in Sprint 1.4 §8.

Added

  • Sprint 1.3 slice 2 — billing routes + Postgres-backed WebhookEventStore + OrgExpiryWarningStore + orchestration service + plan-expiry task + e2e/RLS suite (2026-05-30) — substrate side of Sprint 1.3 now complete end-to-end. Refined the Stripe parser to walk items.data[].quantity (sum across multi-item subscriptions) with top-level fallback for legacy shapes — the prior unwrap_or(1) silent mis-record path is replaced by an explicit BillingError::Rejected when no quantity field is reachable; 4 new Stripe parser tests cover items-level extraction, multi-item sum, legacy fallback, and the rejection case. Two new traits in epic-trait: WebhookEventStore (insert-replay-mark idempotency contract over webhook_events) + OrgExpiryWarningStore (exactly-once contract over org_expiry_warnings); both ship with type-doc invariants + non-exhaustive #[non_exhaustive] enums for forward compatibility. Store trait extended with find_org_by_billing_customer_ref, update_org_seat_count (atomic with SeatCountUpdateOutcome::{Updated, BelowAssigned} outcome + per-org pg_advisory_xact_lock matching migration 0033's discipline), cancel_org_subscription, list_orgs_in_expiry_window. The epic_trait::Org struct gained billing_provider, billing_customer_ref, billing_email, seat_count, plan_started_at, plan_expires_at, country_code — every Org decoder + builder + constructor (Pg adapter, Mem adapter, OrgBuilder, register_user path, the 3 test fixtures) updated in the same PR. Postgres adapters: PgWebhookEventStore + PgOrgExpiryWarningStore — both run SET LOCAL ROLE epic_admin inside their transactions to satisfy FORCE ROW LEVEL SECURITY without bypass. Pg update_org_seat_count takes per-org pg_advisory_xact_lock mirroring the trigger discipline; pg list_orgs_in_expiry_window backs the task. LayeredStore extended for the four new methods + cache invalidation on update_org_seat_count/cancel_org_subscription. In-memory adapters: MemWebhookEventStore + MemOrgExpiryWarningStore with the same contract (parking_lot Mutex-serialised) for the merchant-agnostic e2e. Routes: new billing.rs module exposes POST /api/billing/checkout + POST /api/billing/portal (both org:admin-scoped + BillingError::Unavailable → 503 billing_unavailable mapping via a new typed AuthApiError::BillingUnavailable variant) and POST /api/billing/webhook/{provider} with path-segment dispatch onto the three concrete adapter handles. Webhook handler: verifies signature (typed AuthApiError::WebhookSignatureInvalid → 400 on failure with the invalid body still persisted to webhook_events for operator triage), parses event, dedupes via WebhookEventStore::insert_unprocessed, dispatches to the new services::BillingService orchestrator, marks processed. Orchestration service: process_event switches on SubscriptionEvent variant — Created/Updated → update_org_seat_count with SeatReassignmentBlocked arm emitting billing.seat_drop_blocked audit row + Postmark seat_reassignment_required template; Canceled → cancel_org_subscription with default 30-day grace; TransactionCompleted → slot certificate render (hand-rolled PDF + Ed25519 sig from slice 1) + payment_receipt + slot_certificate Postmark emission with PDF attachment; every arm emits a stable billing.* audit action. New tasks::PlanExpiryWarningsTask with run_once running the exactly-once-per-(org, kind) Postmark fan-out across 3 windows (90d / 30d / 7d); two-phase send-then-mark contract documented in module docs (Postmark failure ⇒ skip mark ⇒ next run retries; concurrent runs serialised by mark_sent's ON CONFLICT DO NOTHING). AppState extended with paddle_provider, lemonsqueezy_provider, stripe_provider, billing_provider (generic-checkout slot wired to MemBillingProvider in tests), webhook_events, expiry_warnings, billing_service. Test surface: integration_billing_e2e.rs covers 11 e2e scenarios via the in-memory adapter stack including signature-invalid bounce, idempotency single-thread + 32-task concurrent race (asserting exactly one Inserted + 31 Replays land), seat-drop-blocked audit emission, plan-expiry exactly-once + window-overlap correctness + clear-on-renewal; 5 new RLS tests in epic-db/tests/rls.rs cover webhook_events + org_expiry_warnings invisibility-to-app-role + admin-role-can-read (with GRANT epic_admin TO epic_app_rls_test added to the test provisioning so the SET LOCAL ROLE discipline is exercised) + UNIQUE / composite-PK enforcement; fixtures/regenerate.md authored documenting the future ADR-007 revision sprint's procedure for refreshing the chosen merchant's fixtures from live-sandbox traffic. Quality gate after slice 2: cargo clippy --workspace --all-targets -- -D warnings clean; cargo test --workspace green workspace-wide (0 failures); cargo deny check clean (advisories, bans, licenses, sources OK); cargo audit clean (526 deps scanned vs 1099 advisories, no vulnerabilities); RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps clean. Sprint 1.3 substrate complete end-to-end — every §2 item shipped or covered by the four-field deferral on the live HTTP outbound paths (T-1.3.3/4/5 explicitly defer ONLY the live HTTP plumbing); the next sprint opens against a fully-wired billing surface. See sprints/phase-1/sprint-1.3-billing-and-slot-certificate.md §7 Progress Log.

  • Sprint 1.3 slice 1 — epic-billing merchant-agnostic substrate + three adapter shells + slot certificate + ADR-007 (deferred-decision form) (2026-05-30) — Option B locked at sprint open (Sprint 1.3 §1 framing the four-field deferral of the live merchant pick per DESIGN.md §21). ADR-007 Accepted as deferred-decision form — three-candidate evaluation locked, symmetric-shells posture locked, live merchant pick reassigned to a future ADR-007 revision sprint post the Australian international-tax-accountant consultation. epic_trait::BillingProvider extended with provider_kind, create_customer_portal_session, verify_webhook_signature, parse_subscription_event, lookup_customer, lookup_subscription. BillingError widened with SignatureInvalid, EventReplayed, SubscriptionNotFound, UnsupportedEvent. New types: SubscriptionEvent, SubscriptionStatus, CurrencyCode, Customer, Subscription, PortalSession. Three symmetric adapter shells landed under crates/epic-billing/src/: paddle::PaddleProvider (ts=...;h1=... HMAC-SHA256, 5-minute replay window), lemon_squeezy::LemonSqueezyProvider (X-Signature HMAC-SHA256), stripe::StripeProvider (t=...,v1=... HMAC-SHA256 with replay-window check + multi-v1 rotation support). All three: signature verifier real + constant-time via subtle::ConstantTimeEq; event parser real against the providers' published wire shapes; live HTTP outbound methods return typed BillingError::Unavailable("<provider> live adapter pending ADR-007 revision") uniformly. Shared signature module factors the HMAC + constant-time compare primitives. Recorded fixtures for every event arm under crates/epic-billing/tests/fixtures/{paddle,lemonsqueezy,stripe}/ + conformance test suite (15 tests). Slot certificate PDF generator under epic_billing::slot_certificate — hand-rolled minimal PDF 1.4 encoder in pdf.rs (printpdf rejected at implementation time on byte-reproducibility grounds — see ADR-007 §"Alternatives" point B). Renders deterministically; embeds Ed25519 signature in /Info dict as /EpicSignature; verifier in the same crate; round-trip + reproducibility + tamper-rejection tests all green. Migrations 0034_webhook_events.sql (idempotency ledger, UNIQUE(provider, event_id), admin-only RLS mirroring audit_log) + 0035_org_expiry_warnings.sql (per-(org, kind) warning-fired ledger, admin-only RLS). epic_postmark::Template gains ExpiryWarning90d, ExpiryWarning30d, ExpiryWarning7d, SeatReassignmentRequired. New PostmarkClient::send_template_with_attachment helper takes a PostmarkAttachment<'_> struct (Name + ContentType + bytes), base64-encodes into the Postmark Attachments field. epic_test_utils::MemBillingProvider extended to satisfy the full trait surface: real HMAC-SHA256 signature verifier on a test-only x-mem-signature header; JSON envelope parser; synthetic success responses on the live methods so the merchant-agnostic e2e is exercisable without binding to a provider's wire shape. Six criterion benches scaffolded (3× sig-verify per adapter, parse warm path, slot-cert render, slot-cert verify). Schema-drift guard updated for both new tables. Sprint 1.2 closing migration head was 0033; this slice numbers from 0034 forward. New workspace dep: sha2 (already transitive via ed25519-dalek + aws-lc-rs, promoted to a direct pin for explicit Sprint 1.3 usage). epic-test-utils gains http, hmac, sha2, hex, serde, serde_json as non-dev deps to back the in-memory billing provider. epic-billing gains hmac, sha2, subtle, hex, ed25519-dalek, blake3, chrono, uuid, serde, serde_json, base64, thiserror, tracing as direct deps. Quality gate after slice 1: cargo clippy --workspace --all-targets -- -D warnings clean; cargo test --workspace green workspace-wide; epic-billing 52 unit + 15 conformance tests; epic-postmark 6 integration tests (1 new); epic-test-utils 80 unit tests (6 new); epic-trait 5 unit tests (3 new). Slice 2 scope note (Sprint 1.3 §7 Progress Log): the remaining §2 items (T-1.3.7, T-1.3.8, T-1.3.9, T-1.3.10, T-1.3.13, T-1.3.14 — control-plane routes + Postgres-backed WebhookEventStore adapter + merchant-agnostic e2e suite) land in subsequent commits of this same Sprint 1.3 work-stream. Substrate-side scope unchanged. Plane: control. See sprints/phase-1/sprint-1.3-billing-and-slot-certificate.md + ADR-007.

  • Sprint 1.2 post-close audit closure (2026-05-30) — operator-approved rework pass on all 11 self-review findings rather than carrying them into Sprint 1.3. New migration 0032_audit_log_org_id.sql (first-class tenant scope column with composite (org_id, occurred_at DESC, id DESC) index; canonicalise() intentionally excludes org_id so pre-migration row_hash values continue to verify); fixes the audit-log handler's silent cross-tenant read (filter.org_id now binds to the new column). New Store::list_users_pending_erasure(cutoff, limit) trait method + Pg + Mem adapters wired into ErasureSweep::candidates() — the sweep is no longer a no-op. MemAuditSink::matches_filter now honors actor_user_id + org_id (was silently ignoring both). MemStore::hard_delete_user cascade extended to drop email_verification_tokens, password_reset_tokens, and seat_assigned entries (was partial). Two new cross-tenant admin routes: GET /api/admin/orgs (paginated org listing backed by Store::list_orgs_admin) + GET /api/admin/audit-log (cross-tenant dump with optional ?org_public_id= narrowing). 21 new unit tests across routes::me (7), routes::admin (6), routes::abuse (1 idempotency + 3 proptests), routes::audit and services::gdpr (1 proptest + 3 cases). 1 new concurrency test in epic-store/tests/integration_pg_store.rs (concurrent_seat_assignments_respect_enforce_seat_limit_trigger). 8 new wire-side integration tests in epic-control/tests/integration_sprint_1_2.rs against fresh Pg containers covering the named T-1.2.17 flows (signup→key, GDPR export, GDPR erase no-orphans, audit-log chain intact, audit-log chain broken returns 500, GitHub secret-scanning auto-revoke, abuse-report ingestion, admin-scope enforcement). New benches/sprint_1_2_routes.rs with 3 criterion benches (me_export, me_delete, audit_log_query) matching sprint §5 SLO budgets. AppState::store and AppState::vault widened from concrete Arc<LayeredStore<…>> / Arc<VaultClient> to Arc<dyn Store> / Arc<dyn KeyVault> so test fixtures can wire MemStore / MemKeyVault directly — every call site uses trait methods only, performance impact is control-plane only. AppState::with_vault_and_audit_query test helper added. Quality gate after closure: cargo clippy --workspace --all-targets -- -D warnings clean; epic-control unit tests 235 → 256 (+21); epic-test-utils 72 → 76 (+4); integration_sprint_1_2 8/8 pass; integration_auth_e2e 12/12 pass (no regressions). See sprints/phase-1/sprint-1.2-org-keys-and-secret-scanning.md §7 "Post-close audit closure".

  • Sprint 1.2 — Org / members / keys / me + GDPR + audit-log query + admin + GitHub secret-scanning receiver (closed 2026-05-30) — 16 of 17 §2 items shipped; one explicit four-field deferral (T-1.2.12 → Phase 6); one wire-side suite scope-realignment (T-1.2.17 → Sprint 1.5). Three new migrations: 0029_org_member_invites.sql (new customer-data table, FORCE-RLS'd, CHECK forbidding role='owner' so invites cannot transfer ownership), 0030_epic_admin_role.sql (provisions epic_admin Postgres role with BYPASSRLS NOLOGIN, idempotent via DO $$ EXCEPTION WHEN duplicate_object block), 0031_users_display_name.sql (adds nullable display_name TEXT with 1..=256-char CHECK). Trait substrate expansion: epic-trait::Store gains 14 new methods (org update / transfer-owner / suspend / status / list-members / role / seat / delete / count-memberships / find-by-prefix-and-last4 / soft-delete-user / display-name / email / hard-delete); new epic-trait::invites::InviteStore (insert / list-pending / atomic consume-by-token-hash / revoke); new epic-trait::abuse::AbuseStore + AbuseCategory (adds KeyLeak); new epic-trait::audit::AuditQuery (filter + cursor pagination + intra-page chain verify). epic-core gains OrgInviteId/Public, AuditLogId. epic-security::ActorKind::EpicAdmin variant. epic-postmark::Template adds KeyRotated, KeyDisabled, KeyRevoked, OrgInvite, AupUpdate, GdprEraseScheduled, AbuseReceived. Parallel Pg + in-memory adapter impls land for every new trait method (PgInviteStore, PgAbuseStore, PgAuditSink::AuditQuery, MemInviteStore, MemAbuseStore, MemAuditSink::AuditQuery). epic-auth::ApiKeyService<S> gains S: ?Sized so the route layer can hold ApiKeyService<dyn Store>. Six new route modules in epic-control: orgs.rs (T-1.2.1–T-1.2.4, nine handlers including 2FA-gated transfer-owner with TOTP step-up + cascade soft-delete on sole-org member removal), keys.rs (T-1.2.5–T-1.2.6, six lifecycle routes; Turnstile gate on org-typed keys; force-flag-gated permanent revoke), me.rs (T-1.2.7–T-1.2.9, GDPR Art.15 export + Art.17 erasure scheduling; soft-delete + 30-day grace; verify-email re-issuance on email change), audit.rs (T-1.2.10, cursor-paginated query with chain verification on read; 500 + audit_chain_broken on break), abuse.rs (T-1.2.11 GitHub Secret Scanning Partner Program webhook receiver with Ed25519 signature verify via FerraVault-stored partner public key + auto-revoke via find_api_key_by_prefix_and_last4 + audit + Postmark security_alert; T-1.2.13 public abuse-report ingestion), admin.rs (T-1.2.14, five operator routes gated by KeyScope::Admin API key; ActorKind::EpicAdmin on every audit emission). New service services::gdpr::ErasureSweep — the hourly background hook the production binary invokes to hard-delete soft-deleted users past the 30-day grace. New AppState wiring fields: bare_store, invites, abuse_store, audit_query. Axum query feature added to the workspace pin to enable the Query extractor for the audit-log route. Deferral: T-1.2.12 (live GitHub partner-program filing) → Phase 6 per DESIGN.md §4.3 + the Sprint 1.2 four-field block. The receiver code itself lands in this sprint and runs against the GitHub registration sandbox locally throughout Phases 1–5; live wiring flips on when Phase 6 lands the production HTTPS endpoint. Scope-realignment: T-1.2.17 (wire-side integration suite extension) folded into Sprint 1.5 sign-off so the full Phase 1 surface area runs as one suite — the Sprint 1.2 unit-test surface (235 passing across epic-control) covers every new route via tower::ServiceExt::oneshot against the in-memory adapter stack. Quality gate at sprint close: cargo check --workspace clean; cargo clippy --workspace --all-targets -- -D warnings clean; cargo test -p epic-control --lib → 235 passing; cargo test -p epic-auth → 111 passing; cargo test -p epic-test-utils --lib → 72 passing; cargo test -p epic-db --test schema_invariants -- --test-threads=1 → 12 passing; cargo test -p epic-db --test rls -- --test-threads=1 → 18 passing. Plane: control (zero edge changes; the data-plane invariant from Sprint 1.1 is preserved). See sprints/phase-1/sprint-1.2-org-keys-and-secret-scanning.md.

  • Sprint 1.1 — epic-control HTTP surface + auth routes (closed 2026-05-30) — full §2 scope, zero deferrals. All nine /api/auth/* handlers shipped: register (T-1.1.3, 11-step gauntlet), verify-email (T-1.1.4), login (T-1.1.5, anti-enumeration + TOTP + session/refresh issue + JWT mint), logout (T-1.1.6, session + bound family revoke), refresh (T-1.1.7, cookie + refresh + RFC 6819 §5.2.2.3 replay detect), password-reset/request (T-1.1.8, always-202), password-reset/confirm (T-1.1.9, single-use + rotate + revoke-all-sessions + revoke-all-families), totp/setup + totp/verify + totp/disable (T-1.1.10, QR PNG + recovery codes). HTTP/3 over QUIC primary + HTTP/2 over TLS-on-TCP fallback in the same process (T-1.1.1) — h3 via the ADR-023 quinn carve-out following the Ferrum precedent (control-plane only; edge plane keeps strict quinn-proto-only posture). Middleware tower mounted onto the production composition root via router_with_middleware (T-1.1.2). Turnstile (T-1.1.12), OFAC (T-1.1.13). Hash-chained audit_log via the new production PgAuditSink (T-1.1.14, advisory-xact-lock matches migration 0015's trigger). FerrumRateLimiter (T-1.1.11) wired in AppState::wire from the FerrumClient. 12-test wire-side integration suite (T-1.1.15) — every test over real wire on both transports (h2 over TLS + h3 over QUIC) against the real Postgres-backed adapter stack booted per-test via testcontainers. Three criterion benches (T-1.1.16). NEW production PG-backed adapters in epic-store: PgStore::AuthFlowStore (auth_flow.rs, atomic register_user transaction across users + orgs + memberships; DELETE … RETURNING for single-use token consume; CAS recovery-code consume), PgSessionLookup (session.rs, write-through revoke), PgSessionStore (session.rs, FOR UPDATE row lock in consume_and_rotate for RFC 6819 §5.2.2.3 family revoke), PgAuditSink (audit.rs, advisory-xact-lock chain serialisation matching the 0015 trigger). 18-test PG adapter integration suite proves each method end-to-end against a real Postgres testcontainer. Migration 0027 reshapes refresh_tokens (BIGINT family_id via sequence + public_id per ADR-002 + rotated_out/family_revoked explicit booleans) to match the trait surface — closes the "Phase-2 PG-side audit item" migration 0026 had flagged. SessionRecord::refresh_family_id trait extension + migration 0026 (sessions ↔ refresh family linkage). SessionCookie::parse_hex, AuthFlowStore::find_user_by_id / find_primary_org_for_user / register_user trait extensions. require_authenticated_user helper. RegisterUserPayload + RegisteredUser types. New ADRs: ADR-022 (h3 deferred — same-day superseded by ADR-023 after the post-close audit caught it as effort-disguised-as-blocker per CLAUDE.md §10.6), ADR-023 (h3 via the quinn carve-out following the Ferrum precedent). New workspace deps: quinn 0.11, h3 0.0.8, h3-quinn 0.0.10, axum-server 0.8, tokio-rustls 0.26, percent-encoding 2.3, reqwest 0.12 (dev), rcgen 0.14 (dev), base32 0.5 (epic-control). Quality gate at close: cargo test --workspace --tests --lib → 45 test binaries pass / 0 failed; cargo clippy --workspace --all-targets -- -D warnings → clean; ci.sh --full Stages 0–6 → passes end-to-end; data-plane invariant unchanged (epic-edge carries zero tokio AND zero quinn deps). See sprints/phase-1/sprint-1.1-control-http-and-auth.md.


[0.1.0] — 2026-05-29 (Phase 0 sign-off)

Phase 0 — Foundation. Empty directory → workspace that compiles → full §10 schema migrated → auth library trustworthy → adapter crates wired → observability live → MeterEvent round-tripping through Ferrum. Sealed by the phase-0-final annotated tag against commit 1646cf25 (unsigned annotated per the T-0.6.13 four-field deferral). See sprints/phase-0/sprint-0.6-sign-off.md for the closing audit transcript; the per-sprint contributions are below.

Added

  • Sprint 0.1 — workspace skeleton, /data backup pipeline, hook scripts, ci.sh Stage 0 + Stage 1, ADRs 001–006, governance docs. See sprints/phase-0/sprint-0.1-bootstrap.md.
  • Sprint 0.2 — six foundation crates with zero I/O: epic-core (newtype IDs, enums, time, error taxonomy), epic-crypto (Argon2id/BLAKE3/AES-GCM/Ed25519/Secret<T>), epic-security (ct_eq, validators, audit canonicalisation), epic-protocol (native + OpenAI/Anthropic shim types, EpicExtension, prost messages, 5-target fuzz harness), epic-trait (9 subsystem traits), epic-test-utils (in-memory adapter per trait, builders, TestClock, optional proptest strategies). ADRs 002, 004, 005 → Accepted. Criterion baselines for Argon2id hash and InferRequest JSON parse. See sprints/phase-0/sprint-0.2-foundation-crates.md.
  • Sprint 0.3 — database layer: epic-db (sqlx 0.9 against Postgres 16, RLS-aware Transaction enforcing (UserId, OrgId) GUCs in 3 round-trips with a combined integer-parsed echo + ADR-003 canary fail-stop, hardened against BYPASSRLS regressions), 21 SQL migrations realising every DESIGN.md §10 table with FORCE ROW LEVEL SECURITY on every customer-data table plus the rls_canary sentinel, audit-log chain + append-only triggers, epic-migrate binary (structured tracing, --dry-run, documented exit codes, Secret<String> DSN), schema-invariants (10) and RLS (12) test suites including a fail-stop test that injects a fake-tenant canary row and asserts Transaction::begin aborts, and two criterion benchmarks verifying Sprint 0.3 §5 perf targets (Transaction::begin p50 ≈ 140 µs vs 1 ms target; RLS overhead ≈ 1.8 % vs 10 % target). ci.sh Stage 3 fully exercising the migration round-trip. ADR-003 → Accepted with acceptance evidence. See sprints/phase-0/sprint-0.3-database.md.
  • Sprint 0.4 — authentication library: epic-auth shipping six services — PasswordService (Argon2id + anti-enumeration dummy PHC on a spawn_blocking boundary, gated behind the password cargo feature so epic-edge's data-plane build pulls no tokio), JwtService (custom EdDSA-only JWT, no ring/openssl, kid-indexed rotation, structural algorithm-confusion immunity documented in the module docs alongside the evaluation of every Rust JWT alternative), RefreshService (32-byte OsRng tokens, BLAKE3 hash at rest, family-revocation on reuse-detection per RFC 6819 §5.2.2.3), SessionService (cookie-side lifecycle parameterised over the epic_trait::SessionLookup trait so the in-memory and Postgres adapters share the contract), TotpService (RFC 6238 HMAC-SHA1 with the Appendix B vector check, AES-256-GCM-encrypted secrets, 10 BLAKE3-hashed recovery codes), and ApiKeyService (ADR-006 epic_{live|test}_{sk|pk|org}_{43 Crockford-base32} format with 215 bits of unguessable entropy, BLAKE3 keyed-hash storage, scope + IP + model allow-list checks, daily caps, rotate_with_overlap that schedules the predecessor's expires_at at now + ROTATION_OVERLAP_DAYS so the lifecycle gate retires the old key automatically). CIDR membership uses ipnet (fail-closed on malformed input). Criterion benches verifying every §5 perf target. 7 dedicated integration tests for refresh-token reuse detection (property + multi-threaded race). Anti-enumeration timing-parity integration test (1024 samples, interleaved with warm-up, max/min ≤ 1.5×). cargo llvm-cov reports 92.46 % line / 93.22 % region across the seven service files; every wire-reachable error branch on every verify method has a passing test. ADR-006 → Accepted with the entropy claim corrected (256 bits → 215 bits) and the value sets narrowed to match the §10 migration constraints. T-0.4.8 in-tree half shipped; the GitHub Secret Scanning Partner Program registration is surfaced as a four-field deferral block (sprint §8) reassigned to Phase 1 alongside epic-control's public HTTP surface. See sprints/phase-0/sprint-0.4-auth.md.
  • Sprint 0.5 — adapter & operations crates: epic-vault (FerraVault HTTP client over hyper 1.x + hyper-rustls with aws-lc-rs, AppRole token in Secret<String>, TTL-bounded in-memory cache, long-poll push channel that invalidates the cache on every FerraVault revision bump); epic-config (typed BootConfig + HotConfig sections, three-layer precedence loader TOML < env < CLI, vault:<key> placeholder resolution against epic_trait::KeyVault, arc-swap-backed watcher that polls the TOML mtime + subscribes to epic_vault::Watcher and refuses hot reloads that change boot-only fields); epic-observability (Prometheus registry with the epic_<plane>_<noun>_<unit> naming convention enforced by validate_name, label-cardinality budget pinned by ALLOWED_LABELS, axum-served /metrics + /healthz admin listener, tracing subscriber installer with JSON / pretty output, feature-gated OTel OTLP/gRPC exporter behind the otel feature so epic-edge's data-plane build avoids the tokio runtime in the OTel batch processor); epic-store (Postgres adapter implementing every Store method against the §10 schema; INET / INET[] columns bound and projected as TEXT with explicit ::INET casts so the adapter does not require sqlx's optional ipnetwork feature; Cache trait local to the crate with MemCache, NoopCache, and FerrumCache (Redis/RESP3) implementations; LayeredStore<S, C> composite read-through + write-through over any Store/Cache pair; FerrumMeter implementing epic_trait::Meter via Redis Streams XADD so MeterEvents round-trip through Ferrum exactly as DESIGN.md §10.8 requires); epic-postmark (templated and inline send paths over hyper + rustls; Template enum covering verify_email / password_reset / payment_receipt / slot_certificate / key_created / abuse_action / security_alert; single retry on 5xx, no retry on 4xx; the X-Postmark-Server-Token is sourced from a Secret<String>; InboundEmail::from_json parses the Postmark webhook payload for the Phase-1 [email protected] flow); epic-control minimal test harness (axum on /health + /healthz, graceful shutdown on SIGTERM/Ctrl-C, observability
    • metrics admin endpoint wired through epic-config); integration docker-compose stack (ci/integration/docker-compose.yaml) bringing up Postgres 16, Redis 7 (Ferrum stand-in), and four wiremock services (Ferron, Postmark, Paddle, FerraVault) with healthchecks on every container plus seeded mappings for the vault secret/revision surface, the Postmark /email endpoint, the Ferron /v1/capabilities endpoint, and a Paddle webhook stub; ci.sh Stage 4 now probes the stack via pg_isready / redis-cli ping / wiremock /__admin/mappings and exports EPIC_*_BASE_URL env vars so integration tests dial the running stack; Stage 6 placeholder ci/load/edge-smoke.js parses cleanly under k6 (real edge workload arrives Phase 2). 59 test suites green workspace-wide; workspace clippy -D warnings clean. See sprints/phase-0/sprint-0.5-adapters.md.
  • Sprint 0.5 hardening pass (post-self-review): a candid audit of the initial pass surfaced ten production-readiness gaps. The rework closed every one across nine phases: (A) Store::insert_user / insert_org widened to take typed UserInsert / OrgInsert carrying every NOT-NULL §10 column; the PENDING_PASSWORD_HASH placeholder is gone. Migration 0022 adds orgs.plan (was hardcoded). Migration 0023 adds api_keys.revoked_at. New BillingProvider enum in epic-core. Test fixture helpers user_fixture / org_fixture in epic-test-utils. (B) Cache trait moved from epic-store to epic-trait per CLAUDE.md §4.3. New LayeredSessionLookup<S, C> for the hot-path session cache the §4 spec asked for and that the initial pass missed (the original LayeredStore cached the wrong reads). (C) Postgres live integration suite: 10 testcontainers-backed tests against postgres:16-alpine surfaced and fixed four real SQL bugs — wrong column names (user_id/name vs issued_by/label), missing revoked_at column, INET projection returning host/prefix not bare IP, seat-trigger firing on default seat_assigned=TRUE. Plus a feature- unification bug: rustls CryptoProvider was implicit; explicit aws_lc_rs::default_provider() pin in epic-vault / epic-postmark now stable workspace-wide. (D) Criterion benches verify §5 perf targets: LayeredSessionLookup hit ≈ 718 ns (target < 500 µs; beats by 695×); miss → inner ≈ 1.28 µs (target < 5 ms; 3900×); FerrumMeter record + tail ≈ 117 µs (target < 50 ms; 427×). (E) Testcontainers-backed meter round-trip — no silent skip. (F) Live OTel collector integration test via otel/opentelemetry-collector-contrib:0.113.0; feature-gated behind otel, runs in ci.sh Stage 4. (G) epic-control becomes a lib + bin pair. The library exposes AppState carrying typed handles for PgStore, FerrumCache, FerrumMeter, VaultClient, PostmarkClient. New /ready endpoint probes each configured adapter at orchestrator readiness-probe cadence; integration test boots the full Postgres + Redis stack via testcontainers and asserts /ready returns 200 with store=true and ferrum=true. (H) ci.sh --full actually runs end-to-end. Stages 1–4 all green; real CI debt cleared (tokio-tar advisory ignore + bollard ring whitelist + audit-ignore propagation; integration stack port conflicts 5433→5499, 6380→6399; --features otel on the integration target glob). See sprints/phase-0/sprint-0.5-adapters.md §7 for the per-phase details.
  • Sprint 0.5 hardening pass II (deferred-debt closure): re-asked "is this really production grade?" and surfaced five more shortcuts. Closed every one: (1) Five dead-code stubs + unused imports across epic-control and epic-vault (_config_env, _signal_module_used, _ip_addr_marker, classify_timeout, ArcWatcher, SecretCache::len) and the missing Reason: paragraph on the clippy::too_many_arguments allow in MeterEvent::new — removed, not allowed. (2) RUSTSEC-2025-0111 (tokio-tar) + RUSTSEC-2025-0134 (rustls-pemfile) — closed by upgrading testcontainers 0.23 → 0.27 (moved to maintained astral-tokio-tar); both advisories drop from the lockfile, deny.toml [advisories] ignore is now empty. (3) OTel collector test exercised only the install path (force-flush/shutdown skipped under 0.27's batch-processor stall). Upgraded opentelemetry stack 0.27 → 0.32, rewrote the pipeline against the new SdkTracerProvider + shutdown_with_timeout(5s) API, and the test now drives install + span + flush + shutdown end-to-end against a live collector inside a 30 s deadline. (4) epic-db RLS test silent-skip pattern: let Some(env) = setup().await else { return; } made the 22 RLS + schema- invariant tests claim ✓ without ever connecting to Postgres when DATABASE_URL was unset. New tests/common/mod.rs boots a postgres:16-alpine testcontainer on the fly; tests now always run, lifting epic-db/src/transaction.rs from 0 % → 100 % coverage. Workspace cargo llvm-cov --tests reports 86.95 % line coverage, clearing the CLAUDE.md §5.3 80 % floor. (5) ci.sh Stage 6 k6 soft-skip: the placeholder script slept 10 ms and skipped silently when k6 wasn't installed. Replaced with a Rust-native load smoke (epic-control/tests/load_smoke.rs) — 200 concurrent VUs, 10 s constant-arrival workload against /health, asserts p99 < 50 ms with zero transport errors. Local run reports ≈ 275k req/s, p99 ≈ 1.2 ms. Stage 6 now hard-fails on envelope breach; the k6 placeholder script is removed. ADR-021 records the Phase 0 choice + Phase 2 revisit triggers so the decision isn't forgotten when epic-edge ships. DESIGN.md §18 + §20 updated to reference it.
  • Sprint 0.6 — Phase 0 sign-off (gating). Three audit documents land under docs/audits/ covering code quality (clippy gauntlet clean, 1 unsafe site with // SAFETY:, 15 library-code .expect()/.unwrap() sites all carrying // INVARIANT: comments after an in-audit normalisation, 10 files >500 lines all architecturally justified), security (cargo audit / deny / geiger clean, RLS posture verified 10/10 + 14/14 against testcontainer-spawned Postgres, anti-enumeration timing test ≤ 1.5× ratio, refresh-token reuse-detection 7/7 property + concurrency), and architecture (10/10 design questions pass — dependency direction, no cycles, foundation crates I/O-free, no raw entity IDs in public APIs, two-plane separation honoured, in-memory adapter per epic-trait trait, spawn_blocking discipline on Argon2id, EPIC_DATA_ROOT discipline). 17 criterion baselines saved as phase-0-final across epic-crypto, epic-protocol, epic-auth, epic-store, epic-db — every Phase 0 SLO target met (RLS overhead +8.5 % vs 10 % ceiling, transaction_begin 161 µs vs 1 ms, JWT sign 10 µs / verify 33 µs, BLAKE3 63 ns, layered session cache hit 690 ns). aarch64 cross-compile + 324 unit tests pass under qemu-aarch64-static. cargo llvm-cov --workspace reports 89.31 % line / 89.91 % region coverage; 100 % on epic-db::Transaction and every parser file. 10/10 sampled commits byte-identical in /data/backups/epic.git; DR drill from latest snapshot → ci.sh --quick Stage 0+1 green in 39.91 s. Three in-audit remediations land alongside: missing // Reason: on one clippy allow, Reason: prefix on a crate-level allow, // INVARIANT: comments uniform across the workspace, ci.sh Stage 2 cargo geiger switched from silently-failing virtual-manifest invocation to per-crate isolated-target-dir invocation, and the banned-shortcut allowlist extended to docs/audits/*.md. See sprints/phase-0/sprint-0.6-sign-off.md for the full audit transcript and the phase-0-final tag context.
  • Sprint 0.6 hardening pass (post-self-review, operator-directed "highest quality production-grade long-term solutions even if we have to redo work"): a candid mid-sprint audit surfaced six findings; all six closed in-sprint. (F-1) ci.sh Stage 3 silently failed when the integration stack was already up on port 5499 — added a precheck that detects the collision via ss/lsof and surfaces both the cause and the two fixes. (F-2) epic_trait::UserInsert.password_hash flipped from String to epic_crypto::PasswordHash, whose Debug truncates so the full PHC string can no longer leak through any format!("{insert:?}") callsite. epic-trait gains a workspace dep on epic-crypto (foundation→foundation, no layering violation). epic-test-utils::user_fixture now produces a real fast-Argon2id PHC via a new placeholder_password_hash() helper. (F-3) epic-auth coverage closed from 92.46 % → 98.52 % lines / 93.22 % → 97.59 % regions via 30+ supplementary tests across the seven verify-bearing files; the remaining 1.48 % is formalised in CLAUDE.md §5.3 as four categories of structurally-untestable code, with the literal contract reframed as the "verify-error-branch invariant". (F-4) aarch64 integration tests escalated from lib-only to the full workspace — 428 tests pass under qemu-aarch64-static, including the testcontainer-driven integration suites in epic-db / epic-store / epic-control / epic-vault / epic-postmark / epic-observability. Stronger than Ferrum's Phase 0 floor. (F-5) phase-0-quality-audit.md §2 "spot-check on 20 random pub items" claim was small + hand-picked; redrew via shuf -n 20 and recorded the full 20-item inspection table inline. (F-6) Geiger artefacts went from 4 manually-captured crates to the full 26-crate sweep produced by ci.sh --full against the Stage 2 remediation. After the hardening pass: clippy / fmt / doc / tests all green; 64 test suites + 428 aarch64 tests + 0 failures. See sprint-0.6-sign-off.md "Sprint 0.6 hardening pass" for the per-finding closure record.

Changed

  • ci.sh Stage 5 — fixed sed pattern that mis-extracted the crate name from absolute paths emitted by find, blocking fuzz harness discovery in Sprint 0.2's cargo-fuzz layout.
  • ci.sh Stage 3 — bumped to the cargo sqlx prepare --workspace --check invocation matching sqlx-cli 0.9 (the -p <crate> flag was retired in 0.9).
  • deny.toml — added allow-wildcard-paths = true for workspace path deps; documented why the architectural-wrapper warnings remain expected through Phase 0.
  • DESIGN.md §10.8 — meter aggregates now spec'd with a surrogate BIGSERIAL PK + UNIQUE NULLS NOT DISTINCT constraint (Postgres rejects expressions in a PRIMARY KEY clause; the new form realises the same COALESCE(user_id, 0) intent in valid SQL).
  • Cargo.toml — sqlx workspace pin bumped 0.8 → 0.9 (latest stable per CLAUDE.md §3.7).
  • epic-core::KeyType — variants reshaped Secret/Publishable → Sk/Pk/Org to match the Sprint 0.3 migration constraint and ADR-006. The Sprint 0.2 form would have failed the api_keys.key_type CHECK on the first insert.
  • epic-core::KeyScope — derives Ord/PartialOrd so callers can hold a canonical BTreeSet<KeyScope> for API-key creation.
  • epic-trait::Store — insert_api_key widened to take an ApiKeyInsert struct carrying every §10 column; added find_api_key_by_hash (constant-time lookup contract), get_api_key_by_public_id, touch_api_key, and disable_api_key. The Sprint 0.2 stubs were too thin to issue a real key against.
  • epic-trait::SessionStoreError::AlreadyRotated carries the family id (AlreadyRotated(i64)) so the auth layer can revoke the family on the theft path without a second hash-to-family lookup.
  • epic-trait::session gained SessionLookup, SessionRecord, and SessionLookupError — the cookie-side session-lookup contract that the auth layer parameterises SessionService over and that the Postgres adapter in Sprint 0.5 will implement. The first cut of Sprint 0.4 placed this trait in epic-auth, which would have forced an upward dep on epic-auth from any other adapter; moved to the foundation layer at sprint hardening time.
  • epic-trait::Store gained schedule_api_key_expiry(id, expires_at), used by ApiKeyService::rotate_with_overlap to retire the predecessor without requiring an external scheduler.
  • Cargo.toml — added hmac = "0.12" and sha1 = "0.10" workspace pins for RFC 6238 TOTP. SHA-1 is the universally-supported authenticator-app digest; stronger digests break compatibility with the installed base.
  • Cargo.toml — added ipnet = "2.10" workspace pin. Replaces a hand-rolled CIDR walker in epic-auth::api_key that had a fail-open path on malformed allow-list entries.
  • epic-test-utils gained MemSessionLookup adapter implementing the new SessionLookup trait. The in-memory adapter was initially inlined as a #[cfg(test)] mod tests definition inside epic-auth::session; moved out so every downstream crate (Sprint 0.5 onward) can reuse it without reimplementation.
  • epic-test-utils::MemStore exposes api_key_lookup_count() -> usize so tests can assert structural anti-enumeration properties (the parse-fails-still-probes-the-store contract in epic-auth::api_key::ApiKeyService::verify).
  • epic-auth::JwtService now consumes a typed Kid instead of impl Into<String>. Kid::parse enforces the [A-Z0-9_-]{1,32} format at the type-system boundary; the wire-side kid field is re-parsed through the same validator at verify time so a token with a path-traversal / control-char kid cannot reach the verifier-set lookup map.
  • epic-auth::SessionCookie tuple field is private; construction goes through SessionCookie::generate() (OsRng) or SessionCookie::from_session_id(id) (explicit, for tests and vault rehydration). sid_claim() is a method on SessionCookie now, not a generic-static-fn on SessionService::<L>.
  • epic-auth::ApiKeyService::verify runs symmetric work on every input — BLAKE3 keyed-hash + indexed find_api_key_by_hash lookup execute regardless of parse outcome, so an attacker cannot distinguish "malformed prefix" from "well-formed prefix but unknown key" by wall-clock timing.
  • epic-auth::ApiKeyService::rotate rejects predecessors that are revoked, disabled, or already expired with the corresponding AuthError::Key* variants — rotating a dead key would create a confusing audit trail.
  • epic-auth::SessionService::lookup throttles the last_seen_at write to a 30-second minimum interval. At production scale the per-request write rate against the Postgres sessions table is bounded by 1 / 30 s per active session rather than 1 per request.
  • epic-auth::RefreshService::refresh family-revoke-failed log line uses a closed error_kind enum instead of the backend's Display form; future Postgres-adapter SessionStoreError::Backend payloads can no longer leak bound parameter values into the log.
  • cargo deny check and cargo audit clean on the introduced workspace deps (ipnet, hmac, sha1).