Skip to main content

Evidence Over Claims

In one line: "It should work now" means "I have not verified it." Nothing is complete until fresh output proves it.

Do this: Before marking a task done, report the current subject/revision, executed checks, results and limits, with sanitized evidence links. Include applicable coverage/lint evidence; do not invent those measures for prose-only work or paste private logs into public reports.

What: No implementation is complete until fresh verification output (test results, coverage, runtime behavior) confirms the claim.

Why: LLMs produce plausible, confident output regardless of correctness — a structural property, not a bug. Code with correct syntax, type annotations, and coherent docstrings looks right to a reviewer even when it hides a subtle logic error, an off-by-one, or wrong API usage.

The dangerous failure mode is not obviously broken code (caught immediately) but code that passes visual review and fails at runtime: an async function called without await, a query using the wrong column that happens to match another column's type, an exception silently swallowed where it should propagate.

Human and model reviewers can miss defects. Automated checks give reproducible answers to the predicates they actually test, not to every property the system should satisfy. Use complementary oracles, adverse cases and explicit coverage limits.

The methodology therefore mandates verification at multiple levels:

  1. Tests must run and pass. Not "the tests exist." Not "the tests should pass." The actual test output must be present in the session before a task is marked complete.
  2. Coverage must meet the adopted profile. The reference profile keeps a 90% business-logic/state-machine floor, including PoC, and a 70% floor for other PoC layers; production uses its stricter gates. Verify scope and executed checks. A starter hook does not configure coverage automatically.
  3. Linting must pass. Configure and verify the selected local/CI checks. A post-edit hook is feedback, not proof that every changed path or every actor was checked.

Mechanism: templates/hooks/pre-push-gate.sh runs lint, format and a configured test command for recognized pushes. The Claude PreToolUse adapter uses exit 2 to block; missing root or command blocks rather than guessing a runner. Its command parser is bounded, not a shell security sandbox. Coverage needs explicit project configuration, and a successful command must still establish the required selection and evidence. The former optional test placeholder is historical; verify the installed adapter and effective server controls independently.

The local adapters and server controls (§7.1) contribute different evidence:

  • Layer 1 (post-edit): A registered adapter runs its configured, scoped lint check on supported events. Verify coverage of intended paths; not every save or actor necessarily triggers it.
  • Layer 2 (stop verification): The supplied Stop hook is an advisory reminder. It does not collect test receipts or prove completion.
  • Layer 3 (pre-push): A configured adapter blocks recognized agent pushes on command failure. Human pushes, alternate invocation paths and web merges need server controls; coverage is not implicit.
  • Layer 4 (merge): Verify effective required checks, owner review and bypass policy on the target branch. An observer can report a bypass after the event; it does not prevent it.

How: The Superpowers verification-before-completion skill governs the completion gate. Before marking any task complete, the skill requires:

  1. Exact current subject and executed expected cases, with actual results
  2. Applicable coverage scope, exclusions and adopted thresholds
  3. Configured lint/static-check outcomes, including unassessed paths
  4. For UI changes, actual visual/interaction checks and their viewport/state limits

The supplied Stop hook is advisory. The completion claim still needs fresh, scoped evidence and review; the reminder itself is not an enforcement receipt.

For the full testing standard, see appendix-a-testing.md. For quality gate architecture, see appendix-e-hooks.md.