Skip to main content

S4U Methodology — Changelog

All normative changes to the canon are recorded here. The canon had no version identity before 3.0.0 (assessment 2026-06-11, finding PW-5); v2.x history below is reconstructed from downstream pointers.

3.4.0 (2026-08-02)​

Canon self-audit remediation — the methodology's own §7 gate-admission rule, applied to the methodology (epic #23, findings #24 / #25 / #27). Three claims the canon made about itself were false; each is now either true or honestly labelled, and each is backed by a check that fails when the claim stops holding.

  • §14 Core tier split into Core-mechanized and Core-asserted (#24). The Core row listed eight components and justified them with "trustworthy without relying on any individual's discipline" — while five of the eight had no detector anywhere in the kit. That is a claim about control SHAPE presented as a claim about control STATE. Core is now two tables, each row carrying the three fields §7 demands: per-occurrence cost, enforcement mechanism (a resolvable path or repo-config: control, else the literal none (human-asserted)), and retirement condition — the mechanism that would move an asserted row up. The rationale sentence no longer covers both halves uniformly. R1-R3 keep their single home on the operating card (correct under §4.5) and §14 now gives the pointer it was missing. Mechanism: scripts/check-tier-mechanisms.sh — fails if a Core-mechanized row names nothing that resolves, if a Core-asserted row claims a mechanism it does not have, if any field is blank, or if a named mechanism file does not exist.
  • The single-source rule now has a mechanism that implements it (#25). §4.5 named check-canon-consistency.sh as the detector of project-copy drift, and repeated it under "Reproducible" — but that script only ever checked the canon's internal consistency and never opens a project file. New scripts/check-single-source.sh reads a project's CLAUDE.md / AGENTS.md, shingles every declared rule home (operating card, appendix-m, skills/*/SKILL.md), and fails on a verbatim run of ≥12 words carrying no deviation ADR — naming file:line and the copied text. Allowed copies are explicit: an ADR-NNNN reference in the same block, or an <!-- s4u-allow-copy: reason --> marker. Its limit (verbatim only, not paraphrase) is stated in the script header rather than implied by the canon. Run live against the flagship project on its first outing, it found a real violation. appendix-m's "copy the mandatory + default rows into ADR-0001" instruction — which mandated creating the copy §4.5 forbids — is corrected to pointer-plus-deviations.
  • rule-inventory.md is a register again, and is checked (#27). It was incomplete in both directions with no detector: §3.5 (Incident-Response) and all of §15 (Securing the AI Collaborator) had zero rows despite predating its own audited version; four live operating-card rules (the probe-gated flag flip, the public-doc classification gate, the mandatory STATE.md/memory cadence, the Core tier split) had none; row 9 still asserted a blocking doc-staleness gate that v3.1.2 had retired to advisory; the header was pinned at v3.1.2; the stated extraction count (120) disagreed with the table (123); and the size projection cited a ≤20,000-byte target against an enforced cap of 11,500. Twelve rows added (124-135), row 9 rewritten to the advisory form, rows 38/43/123 corrected, counts and the byte section re-derived from measurement. Sections with genuinely no normative rule (§1, §4.4, §11.5) are now declared with a reason instead of silently absent. Mechanism: scripts/check-rule-inventory.sh — six deterministic checks (version pin, leaf-section forward coverage or explicit declaration, dangling §-refs, card mechanisms registered, bolded card rules registered, arithmetic).
  • templates/scripts/flip-flag.sh gets a spine home (§7, "Gate: probe-gated flag flip") — it shipped and was mandated on the card, but appeared nowhere in methodology.md, so no register row could cite it.
  • The four reviewer templates now carry a "Test evidence" checklist block (MOCK APPROVED present, no internal-class mocking, migrated-schema oracle). §8 claimed a bare mock "is flagged at review" while none of the reviewer definitions mentioned mocks; the block is the artifact §8 now names, and §8 states plainly that this is review, not a gate.
  • Harness 117 → 138 checks; three new gates wired into canon-ci.yml. Every new mechanism ships with a mutation-proved fixture: break the rule, the named check fails.

3.3.0 (2026-08-01)​

Two rules the canon was missing, both found by applying the methodology's own §7 gate-admission rule to the methodology (audit epic #23), plus a false-caveat correction:

  • §7 field (d) — a gate must prove it observed something (#26). The gate-admission rule required a mechanism but never required the gate to see anything, so a named, green job that examined nothing was indistinguishable from one that examined everything. Field (d) has two halves. Demonstrated failure (admission time): every shippable gate script declares # failing-case: <suite> "<test name>" (or a stated-reason none exemption), and scripts/check-test-coverage.sh FAILS when the named test is not present in that suite — so the claim cannot drift away from the test it cites (22 shippable scripts examined; 17 name a real firing test, 5 are declared-exempt with a stated reason). Work witness (run time): a gate that enumerates a subject set prints the count it examined and FAILS at zero — implemented in check-adr-register.sh (examined 0 ADR files), check-doc-classification.sh (examined 0 markdown files) and check-test-coverage.sh (examined 0 shippable scripts), each with a fixture pinning the empty-subject failure. All three previously exited 0 over an empty register/corpus/script set. The corollary is now canon text: a guard verified only where it passes is indistinguishable from a guard that cannot fail. Card + PR template + s4u-code-review (name-the-count and name-the-failing-case become review lines; the oracle table gains a work-witness column) + rule-inventory row 136.
  • Mutation discipline (#29) — appendix-a §14, new. The testing standard (995 lines) did not mention mutation testing, and had no answer to "can this test fail?" beyond an unmechanized third-party mental check that PoC mode's tests-after ordering removes the RED phase from. Adds the three-outcome contract (KILLED / SURVIVED / INCONCLUSIVE, where INCONCLUSIVE is never a pass), the rule that DATA files are first-class mutation targets (21 code-side mutations once all killed their tests while the guard they protected was inert, because its subject was a JSON file), and the triage rule that a survivor means suspect the TEST first (nine of nine survivors in one session were defective tests). Mechanism: templates/scripts/mutation-probe.sh asserts five preconditions before scoring — baseline collected >0, baseline green, anchor occurs exactly once, bytes changed, mutated run still collects >0 — and --collect-cmd is required and fail-closed, because without a collection oracle "the test passed" cannot be told from "no test ran". Whole-suite mutation scoring is explicitly recommended (not yet enforced). Card + s4u-testing-standard + §8 principle 5 + rule-inventory row 137.
  • ADR skill §9 caveats were false (#28). Both "matching caveats" were wrong against the current check-adr-register.sh: ADR-NNNN-*.md files are scanned (-name '[0-9]*.md' -o -name 'ADR-[0-9]*.md', lines 14/28/43) and the back-reference match does tolerate bold (\*{0,2}Supersedes\*{0,2}: ?ADR-<n>). Commit 7aaf8b5 fixed the script and wrote the pre-fix prose into the skill in the same commit; two later edits left it standing. The caveats steered agents to rename a register away from the flagship's own working convention — a stale warning that reports a working control as broken suppresses the control. Replaced with execution-verified behaviour plus a standing instruction to re-verify by running the script, never by reading.
  • Harness 117 → 138 checks (mutation-probe contract ×11, work witnesses ×4, meta-gate failing-case + zero-subject ×6). Verified by mutation: six mutations of the new mechanisms, six kills, no survivors.

3.2.3 (2026-06-16)​

A worked-example generator for the §7.5 freshness gate — ADR publishing (canonical → site):

  • templates/scripts/generate-adr-mirror.sh — renders the canonical ADR corpus (docs/adr/ADR-NNNN-<slug>.md, owned by appendix-g) into a Docusaurus docs tree as NNNN-<slug>.md pages (frontmatter id / sidebar_position = ADR number + 1 / title from the H1, body verbatim). Closes the "published ADR mirror lags canon" drift class — observed in the field where a site's ADR section stopped at 0049 while canon reached 0066 (each new ADR needed a manual copy + sidebar edit). Run it as a build/start prestep with the ADR sidebar set to autogenerated, so new ADRs reach both the docs tree and the nav with no manual step.
  • §7.5-conformant by construction: takes its output dir as $1, rewrites only the NNNN-*.md pages it owns (authored siblings cancel out of the diff), and is deterministic (C-locale glob order, title read from file, quote-escaped YAML, no timestamps/randomness) — so it is pinnable by check-generated-fresh.sh (verified: FRESH on a clean tree, FAIL with drift listed on a mutated page; quote-laden titles escape correctly across 65 real ADRs). Full-generate, single source of truth: canon is authoritative; a project with divergent hand-curated published ADR pages reconciles into canon first (appendix-g: accepted ADR bodies are immutable) rather than running a full-generate gate over divergent copies.
  • Documented in appendix-h (new "ADR Publishing (canonical → site)" section, sibling to the architecture index). Tier: Recommended (§14) — adopted where a project publishes ADRs to a living-doc site; the generator is generic (canon dir in, Docusaurus dir out). Fixture-tested in tests/run-checks.sh (deterministic output + the §7.5 gate reporting fresh-vs-stale over the generator).

3.2.2 (2026-06-15)​

New §7.5 blocking tier — generated-artifact freshness (durable fix for the recurring "generated copy lags source" class behind the 3.2.1 hotfix and the Codex findings on PRs #13/#16/#18/#19):

  • templates/scripts/check-generated-fresh.sh — a generic regenerate-and-diff gate: seed a throwaway dir from the committed generated tree, run the generator into it (generator takes its output dir as $1 and rewrites only the files it owns, so authored siblings cancel out of the diff), then diff. Any difference = the source changed without regenerating → block. Ungameable (runs the real generator, not a hand-maintained file map) and self-correcting (the fix is "run the generator and commit"). Fail-closed on missing/!executable generator, missing out dir, generator error. Generic by construction — pins a Docusaurus site, generated protobufs, or an OpenAPI client alike.
  • Blocks immediately, no warm-up flag (unlike the heuristic DOC_SYNC_BLOCKING code↔doc gate): regenerate-and-diff has a structurally near-zero false-positive rate provided the generator is deterministic. website/scripts/build-docs.sh gained an optional output-dir argument (default unchanged) so the gate can target a temp dir without mutating the tree.
  • Compares content and the executable-bit set (Codex P2 on PR #21: diff -q is content-only, so a generated script that lost its +x bit would pass on identical bytes; the gate now diffs find -perm -u+x too, and seeds with cp -Rp so the comparison is faithful). Matters for generated scripts/wrappers, not the markdown site.
  • Wired into .github/workflows/canon-ci.yml (real-tree, authoritative) and the pre-push-gate.sh template (documented optional block §8); fixture-tested in tests/run-checks.sh (a trivial deterministic generator + matching/stale trees + a mode-drift case + fail-closed branches). Harness 102 → 113. Operating card + rule-inventory (row 123) updated.

3.2.1 (2026-06-15)​

Bot-review follow-through (Codex review on PR #18, two findings merged-over — caught in a post-merge sweep):

  • P1 — incident routing contradicted its own cycle. The §3.4 trigger table's Incident-Response entry point read reproduce → root-cause → regression-pinning test, routing responders to diagnosis before service restoration — directly contradicting §3.5's "Mitigate before you diagnose." The entry point now leads with mitigate first (flag-off / revert), matching the cycle's reverse-lever-first posture.
  • P2 — stale generated site page. website/docs/lifecycle/lifecycle-integration.md still showed the pre-v3.2 6-row trigger table (missing the Inception and Incident-Response rows) because PR #18 edited canon after PR #17's site regeneration without re-running build-docs.sh. Regenerated from canon. Underlying class (canon-PR-merges-after-regen-PR) noted for a future same-PR doc-sync gate.

3.2.0 (2026-06-14)​

Two structural additions (design: specs/2026-06-14-v3.2-inception-stage-and-doc-sync-design.md; second-party reviewed across two rounds; deep-research-backed):

  • §3.6 The Inception Cycle — an optional fifth lifecycle for new projects / new bounded contexts: arc42 inception canvas + ≥3 quality-attribute scenarios + risk-storming (security lens) + per-system trust-boundary/STRIDE threat enumeration + ≥1 fitness function; gated on artifact presence (substance via the §3.1 second-party threshold). Templates in templates/; full reference in appendix-o-inception.md. Fixed the stale §3 intro ("three lifecycles" → five) and fanned the cycle into s4u-lifecycle (which had also been missing the §3.5 Incident-Response row).
  • Tiered documentation-sync (§7.5 + §11.4) — doc-sync becomes a tiered fitness-function: scoped code↔doc-pointer drift + link integrity + ADR-register integrity BLOCK (fail-closed, via templates/scripts/check-doc-sync.sh + a doc-pointers manifest + a [skip-docs:] escape hatch logged for the census); blanket 30-day staleness + prose/Diátaxis stay advisory — preserving the v3.1.2 honesty fix (a blanket age-gate surprises contributors). s4u-doc-excellence rewritten to the tiered model; canon-pinned-defects.tsv pins the retired "advisory-only" universal claim, disjoint from the kept blanket-tier text (the blocker the two-round review caught). The §2.8 census now surfaces the skip-docs log + manifest coverage (silent-decay defense). The scoped gate ships behind DOC_SYNC_BLOCKING until a sub-10% false-positive dry-run; probe docs/probes/doc-sync-blocking-2026-06-14.txt reports 5.0% (PASS).
  • Operating card + rule-inventory (rows 121-122) updated to match; byte budget held. The Docusaurus site-format template ships separately (specs/2026-06-14-docusaurus-site-template-design.md).

3.1.2 (2026-06-13)​

Adversarial-review fixes (regressions from the heavy 3.1.x editing + structural hardening):

  • Fan-out regression fixed: the 6→7 Decision-Cost axis change had reached the canon spine but NOT the skills agents load (s4u-lifecycle, s4u-adr) — their Pre-Mortem / Decision-context block formats omitted the Cost: line, so an agent following the skill emitted a 6-axis block. Now seven everywhere; the rule-inventory and CHANGELOG editorial note are aligned too.
  • Fails closed now: check-canon-consistency.sh scans skills/*/SKILL.md + rule-inventory.md (was methodology.md-only — the hole the 6/7 drift fell through), with a pinned-defect on stale axis counts. The meta-gate's "tested" check now requires a non-comment invocation (a false # no-fixture: can no longer hide an untested script).
  • Honesty drift corrected: prose that asserted a blocking doc-staleness gate (§2.5, appendix-h) downgraded to advisory, matching the shipped exit-0 hook; the STATE.md staleness claim is now true — generate-state-md.sh emits a staleness date and check-doc-staleness.sh surfaces a STATE.md >30 days old.
  • Two real script bugs: methodology-health.sh grep -c || echo 0 crash fixed; consolidation-census.sh born-date pickaxe anchored to a word boundary (was matching auth_enabled inside oauth_enabled, inverting the age-rank). Both now invoked by the suite. Harness 92 → 102.
  • New §3.5 Incident-Response Cycle (detect → mitigate via flag-off/revert → root-cause → regression-pin → named-class postmortem → memory+STATE.md) — the missing reverse lever for a production posture. Worked Cost-axis example added.
  • pr-review.yml AI-review re-gated via a needs-output job (the job-level if: secrets.* pattern is unreliable); bootstrap now installs the doc-staleness hook; product-scale-planning skill gains a §5.6 crosswalk.

3.1.1 (2026-06-13)​

Audit-remainder polish (no rule changes; consistency, coverage, and enforcement):

  • Canon links: repointed 18 dead/circular links into the redirect-stub appendices (b/c/f/i/k/n) to their real homes (§5, §13, the relevant skills, showcase.md). §7.1 reframed "Three-Layer Defense" → "Three Core Gates + Optional Security Layer" (count now consistent). §5.2 column rekeyed Model → Capability Tier (tier is the contract; model names are point-in-time bindings). Operating-card budget stated byte-cap-primary (the enforced unit).
  • Skills now carry their mapped procedures: reviewer-dispatch mapping (s4u-code-review); R1–R3 silent-failure discipline + the 90%-in-PoC core floor (s4u-testing-standard); stale-test classification + convergent-design signal + a Superpowers prerequisite line (s4u-lifecycle); always-on subagent-dispatch hygiene + the four-status protocol (s4u-loop-dispatch); single-source pointers fixed to survive a bootstrapped project.
  • Gates that now gate: expect_err test helper (asserts the diagnostic, not just the exit code) + fixtures for previously-untested high-stakes branches; a shipped advisory templates/hooks/check-doc-staleness.sh; CI (canon-ci.yml) now runs the operating-card version header + the adr-register / doc-classification / kit-vs-project integrity gates it previously skipped; canon-consistency gains a redirect-stub-link flag; consolidation-census age-ranks flags by born-date + documents the manual retirement gate (honest over a flaky stateful auto-fail). Harness 77 → 92 checks.
  • Adoption/site: bootstrap writes a .s4u-kit-version stamp (upgrade story)
    • points adopters at .claude/agents/*.md; reviewer-agent frontmatter notes the capability tier above each model: line; the site gains local search (@easyops-cn/docusaurus-search-local) and a navbar version indicator.
  • Mandated STATE.md + memory cadence (explicit): the operating card now carries a crisp rule — every finished branch regenerates STATE.md and updates the relevant memory files in the same commit; memory also updates on any decision/correction/new-pattern mid-session; STATE.md stale >30 days is a defect (flagged by the doc-staleness hook). §3.1 Finish & Merge and the §11.4 documentation-commit pattern (now a 5th required artifact) state it as non-optional, confirmed at review.

3.1.0 (2026-06-13)​

  • §15 "Securing the AI Collaborator" (NEW spine section) — a threat model of the collaborator: 15.1 untrusted-input-is-data (the behavioral rule marked recommended (not yet enforced), with §15.4's permission gate as the enforced backstop), 15.2 secret hygiene (→ check-doc-classification.sh), 15.3 tool/MCP least-privilege + provenance (→ advisory scripts/check-tool-provenance.sh), 15.4 permission-mode tied to blast-radius. Safety sign-off: Tsunami-max, 2026-06-13 — trigger 6 cleared (see specs/2026-06-13-methodology-hardening-design.md).
  • Decision-Cost Rubric → seven axes: added Cost (compute/token spend) to §2.7, the §3.1 Pre-Mortem Block format, templates/spec-template.md, and the rubric diagram. Closes the audit's "no economic axis for subagent-heavy work".
  • Machinery: self-policing meta-gate scripts/check-test-coverage.sh (every shippable script must be tested or carry # no-fixture:); fixtures added for the 5 previously-untested scripts (harness 53→71); new project-agnostic scripts/methodology-health.sh (effectiveness trend instrument, no borrowed metrics) + scripts/adoption-smoke.sh (stack-agnosticism probe). The probe caught and we fixed a real external-validity gap: templates/project-claude.md hard-coded Python/Postgres commands outside the {{...}} placeholders.
  • SETUP-GUIDE.md retired to docs/archive/ (it taught three killed patterns); inbound pointers repointed to bootstrap.sh. setup/ADOPTION-TRIAL.md added (external-validity protocol for outside adopters).

Unreleased — editorial (2026-06-13)​

  • Non-normative: added six Mermaid diagrams to the spine, visualizing existing prose (no rule changes): the documentation knowledge-flywheel (§2.5), the the Decision-Cost Rubric diagram (§2.7), the consolidation/subtraction loop (§2.8), the four memory types feeding session context (§6.2), and — in §3.4 — the skill-chaining pipeline and the subagent orchestration model. Authored for the standalone methodology documentation site; they propagate to all sync targets.
  • Non-normative: de-projecting + readability pass across the canon. Removed all references to specific real projects and their metrics/incidents (named projects, frameworks, and incident anecdotes). Every **Evidence:** paragraph that cited a project metric was replaced with a generic, mechanism-based statement (name the enforcing hook/script/gate; reproducible). Each section gained an inverted-pyramid lead (**In one line:** / **Do this:**); body prose tightened. No rules changed; all four canon gates stay green. Rationale: the methodology should read as a reusable, engaging, actionable standard — not a project case-study novel. Evidence is now demonstrated by runnable machinery, not borrowed metrics.
  • Audit-driven finishing pass (multi-agent audit, 57 verified findings). Gates: fixed the BP-5 defect in the blocking pre-push-gate.sh (was ruff check app/ --quiet; now full ruff check . + ruff format --check .), corrected its block exit code (1→2, the only code Claude Code PreToolUse blocks on), added python3 interpreter guards; broadened check-canon-consistency.sh so it actually validates s4u-* skill refs (the old regex excluded the digit 4); boundary-anchored check-adr-register.sh (was a substring match); extended check-doc-classification.sh to catch IPv6 / URL-embedded creds / cloud keys (AKIA/ghp_/glpat-/sk-/AIza), tuned to not false-match ISO-8601 timestamps; re-budgeted context-budgets.tsv onto the actually-loaded files. Test harness grew 40→53 (negative fixtures for each new branch). De-projection completed beyond docs/: README, root CLAUDE.md, the five canon-mirroring skills, and appendix-l/j/a are now project-free (the earlier "removed all references" claim is now true repo-wide); front doors repointed from the superseded SETUP-GUIDE to bootstrap.sh. Spine: removed phantom /designing,/planning skill names, de-duplicated §13, fixed the §4.3 Python-floor contradiction. Site: removed dead Docusaurus scaffold + fixed the broken social-card ref.

3.0.0 (2026-06-12)​

  • Added machine-readable version header + this changelog (PW-5).

  • §2.8 Consolidation Review written as a mechanism (census script + retirement mandate); was a dangling "(planned)" reference; cadence set to monthly (AF-1).

  • Canon consistency check shipped (the "mechanical greps" §5.6/appendix-l promised): phantom skill names fixed (/designing, /planning, /worktree, /finish, /code-review → installed Superpowers names, incl. both mermaid diagrams); freezegun three-way contradiction resolved (§8 defers to §4.5 default); §5.2 model-naming self-contradiction removed; §2.7 case-study timeline de-inflated ("month three"/"for months" → days, matching its own facts); /writing-skills added to the §4.4 table (was listed as 14, contained 13).

  • §3.1: SIXTH Brainstorm-Gate trigger — changes to safety policy / guard / refusal behavior gate hardest and require a literal human sign-off line (assessment meta-pattern C: the jury dosing incident entered through an APPROVED policy relaxation no gate covered).

  • §4.5 rule 1: verbatim-copy mandate REPLACED by single-source-plus-pointers; per-project canon mirrors retired; drift checking is mechanical (CE-5, PW-5).

  • §3.1 lifecycle v3: spec+plan merged into ONE design artifact (templates/spec-template.md); Plan Walkthrough RETIRED (zero recorded completions in 3 months) in favor of a second-party scrutiny threshold (safety path / schema / public API / auth -> fresh-context or human review); §7 gains three wrong-oracle defenses: live contract smoke (I13), migrated- schema oracle (I35), and the name-the-oracle review line (meta-pattern A).

  • §14 rewritten: Core tier = the enforced gates with documented saves (required CI checks, CODEOWNERS safety review, safety-floor evals, R1-R3); named-project specifics demoted to project-specific-with-ADR; permission mode reclassified preference -> security control; NEW §14.1 multi-dev operating model (org repo, deploy lock, generated STATE.md, incident roles) (TA-02/05/06).

  • §7 gate-admission meta-rule: every gate declares cost + enforcement mechanism + retirement condition; enforced via the canon PR template (assessment §6 standing meta-rule).

  • §6 + appendix-d: hub budget restated in BYTES (24,000 — the loader's unit); durable-first section ordering is policy under truncation; advisory memory-budget Stop hook shipped; MEMORY-template reordered (CE-2).

  • Operating card extracted (docs/operating-card.md, ~7.6KB / 42 rules of 119 inventoried in docs/rule-inventory.md) — the only always-loaded surface; methodology.md demoted to reference (CE-1/CE-4/PW-1).

  • Showcase split + appendix dispositions: §12 + appendix-f -> showcase.md; appendix-c merged into §5 (specialists now OPTIONAL — resolves the flagship-forbids-them conflict); appendix-i tables merged into §13; appendices b/k/n retired to skills; appendix-e rewritten — prompt-type blocking Stop hooks REMOVED at all sites (completion-loop failure mode), command-type diff-aware advisory is the standard; appendix-m's false project-stack claim corrected (CE-5); a/d/g/l carry superseded-by notes.

  • STATE.md policy: generated from git/gh or absent, never hand-maintained; generator shipped at templates/scripts/generate-state-md.sh (CE-8, TA-05).

2.3 (2026-05-12) — reconstructed​

  • §2.7 Decision-Cost Rubric; §3.1 Brainstorm Gate (Pre-Mortem Block).

2.2 — reconstructed​

  • §2.6 Investigative Discipline; appendix-n Documentation Excellence Passes.

2.1 / 2.1.x — reconstructed​

  • §4.5 canonical tech stack; §5.5 /loop pattern; §5.6 product-scale planning; appendices k/l/m.