Skip to main content

Universal release validation

Validate the meaning of actual outputs against approved expectations and authoritative evidence. This protocol applies to UI, APIs, CLI tools, batch artifacts and events; it requires no domain addendum, browser for a headless product, cloud or agent host. Binding authority and non-waivable controls precede project profiles and exceptions. It extends methodology §2.11, §7 and §8 without supplying release authorization.

Contents

Scope and expectations

The adopter's reviewed profile inventories surfaces with stable id, kind, mandatory, demand_tier and blocking declarations. Mandatory validation, demand tier and blocking policy are separate. Inventory actual consumers, not only the surfaces convenient to test; every inventoried surface needs a receipt entry.

Before candidate evaluation, pin an independently reviewed expected-result corpus. Exercise ordinary successful/empty and adverse/error behavior plus applicable missing, unknown, pending and not-assessed states. A benign and adverse case are a starting minimum, not comprehensive coverage. A completed empty count can correctly yield zero; a timed-out warehouse cannot become zero, success or clearance. Partial assessment needs an explicitly supported partial state, not an overall complete claim.

Combine controlled synthetic boundary fixtures with approved sanitized goldens. Record source, expected-result author/reviewer and actual independence, corpus version, redaction transformations and known coverage limits in adopter-owned evidence. Captured output records what happened, not automatically what should happen. Pending captures and unverified historical outputs stay explicit gaps. Never commit private production records or credentials to the kit or public receipts.

Compare actual outputs

For each case and surface, trace authoritative record/state → transported payload → actual externally observable output. Bind the candidate revision/artifact identity, profile/corpus version, execution environment and observation to the same case.

SurfaceAppropriate actual-output evidence
UIAuthorized browser/render observation of text, state and visual cues, with payload/state references; label simulated backends
APIActual response/status/serialization at the declared API boundary
CLIActual stdout/stderr, exit and invocation/environment
Batch artifactActual generated file, content/count/completion state and applicable visual inspection
EventActual emitted message, identity/schema/order and delivery boundary where claimed

An API capture is not browser proof. Missing access, evidence or a failed adapter is cannot-assess, not passed. Do not invent browser output to fill the receipt. Evidence from one surface cannot silently stand for another; surface agreement alone also does not prove the underlying business meaning.

Check five failure shapes:

  1. Absence asserted as a result, or a verified result represented as absence.
  2. Inconsistent representations of the same fact.
  3. Unsupported status/severity cues.
  4. Wording that exceeds supporting evidence.
  5. Stale or mismatched revision data.

Use the contract's time meaning: event time comes from an event, not render time; a correctly labelled page-render timestamp is legitimate. Record expected versus observed behavior and disposition per surface, with sanitized evidence references. Explaining the root cause does not clear a defect; clearance needs a verified fix or explicitly authorized permissible acceptance. Non-waivable duties remain blocking.

The scoped comparison evaluates meaning; the structural checker validates the record's shape/accounting. Neither a JSON pass nor a diagram approves release.

Independent oracles

Assign expectation/contract-test authorship separately from implementation where feasible and record any limits. Fresh context may reduce anchoring but is not proof of independent judgment. Pin approved expectations before implementation and retain their provenance. The implementer may propose a correction, with discrepancy, reason, contract/history evidence and impact; an independent reviewer must approve the new pinned expectation. Business meaning still requires the accountable owner's approval. Do not silently edit both implementation and oracle to obtain green tests.

For new guards retain baseline passing execution, proof the intended mutation reached the actual guard, intended assertion failure, safe restoration and final passing execution. Unapplied mutations, unrelated errors and empty selections are inconclusive. See Appendix A §14. Registry/caller tests discover omitted in-scope consumers independently of the existing register; declare roots, access forms, exclusions and dynamic/external gaps. Keep separate behavior tests exercising the real resolver and callers on missing and ordinary inputs.

Alternate paths and exceptions

When recovery, retry, resend or another route reaches an existing effect, inventory the primary-path controls and verify equivalent required protection on each route. Include applicable authorization, lifecycle-state checks, idempotency, validation, privacy and audit behavior. Legitimate path differences need a supported explanation. Prefer an owning resolver for genuine shared invariants; similar labels in unrelated bounded contexts do not justify forced centralization or identical semantics.

Exceptions name owner, actual authority reference, scope, reason and review/expiry condition. Add a machine-checkable predicate when possible and verify it with an adopter-owned mechanism. Otherwise explicitly label the exception human-assessed and retain dated human review. Predicate text in a receipt is not executed; a review reference alone does not authenticate the decision. A complete record does not clear a failed outcome or waive a non-waivable control.

Review and coordination

Before initial review of reinterpreted inputs/state, the readers manifest records input owner, authoritative contract/revision, producers/readers and affected/unaffected dispositions, discovery witness and gaps. Include the input-owning subsystem within authorized discovery. Report inaccessible/dynamic/external consumers and escalate cross-boundary work rather than editing without authority. R170–R172 retain second same-shape discovery and round-four accountable reassessment per predicate family. Authorized scoped incident containment can proceed while unresolved structural work remains open; a counter does not authorize centralization or waive blockers.

Dispatch handoffs identify ownership, source/revision, expected corpus/surfaces, checks, remaining budget, unresolved evidence, active jobs and contingency trigger/ permitted action/decision owner. A Plan B is not permission to execute it. Preserve forward-only history for mid-flight commit-affecting guidance; no implied amend/rebase permission. Stop/fence old writers before transferring ownership.

Choose bounded worker lives, reuse/fresh handover and WIP checkpoints from actual host behavior, review capacity and approved profile budgets. Three PRs and a 50% budget checkpoint are optional profile examples, not universal caps/stops. Aggregate budget and unresolved state across replacements; reserve required verification and shutdown/reporting allowance before dispatch. Exhaustion stops new effects, not the record of missing checks. Consolidate observation through an available supported event/wait or non-model observer; the kit supplies no watcher or zero-token guarantee. See s4u-loop-dispatch for stopping, wakeups and reconciliation.

Structural checker contract

The detailed exact v1 fields, limits and invocation are in the receipt contract, with profile and receipt templates. Copy these into adopter-owned files and use independently approved exact candidate/profile/corpus identities. Do not invent schema extensions: provenance can live in referenced evidence or allowed descriptive objects. The unexecuted receipt template correctly reports cannot-assess.

python3 -B scripts/release_validation.py receipt.json \
--profile profile.json --candidate artifact:synthetic-candidate-v1 \
--profile-version synthetic-profile-v1 --corpus-version synthetic-corpus-v1 \
--evidence-root ./owned-evidence

Statuses are exactly passed, failed, cannot-assess, inapplicable; the last two require a reason. Each inventoried surface appears exactly once, including optional surfaces. A reported passed identifies evidence. Local references must be ordinary readable files confined to the explicitly supplied trusted root under the contract's path/read limits. External references are never fetched and remain unverified. No semantic adapter, credential retrieval, command execution, author authentication or approval verification is performed.

The CLI emits JSON on stdout and a concise structural summary on stderr. Retain both the report and actual exit: 0 means structurally valid even with failed/unassessed producer outcomes; 1 means known record violations; 2 means inability to evaluate. record_status is distinct from reported_outcomes; semantic_verification is always not-performed. evidence_verification.checked_local and unverified_external count examined references; examined_surfaces is the visited record count, not a completeness/meaning certificate. Read diagnostics and retain limits. Advisory invocation must not swallow an exit or print semantic pass.

Gate admission: adopter declarations name demand tier, blocking status, mandatory validation, cost, actual mechanism, retirement, known failing case, scope/limits and work witness. The checker is bounded linear structural work; mechanism scripts/release_validation.py, known firing case tests/test_release_validation.py::test_wrong_expected_candidate_is_rejected, witnesses examined_surfaces and evidence counts. Retire it when an adopted stronger validator covers this contract. Promotion to blocking needs a named human decision, recorded telemetry and verified project integration; no installer/host registration or required-check policy change is supplied here.

Release decision

Use RV1–RV6 for the final candidate. Presence checking of checklist slugs is not semantic enforcement. Under an adopted demand-only cadence, complete one final-candidate full run of gates, complete suite and applicable configured security tiers; missing mandatory tiers fail closed. [skip ci] PR heads require explicit dispatch, and default-branch workflow availability must be checked. A later substantive candidate change needs new applicable evidence.

Report structural record status, producer outcomes, verified/unverified evidence, mandatory gaps, exceptions and accountable release disposition separately. Required validation left cannot-assess remains a blocker under existing policy. Changing cadence changes when logic runs; removing server-required contexts changes enforcement and needs separate owner authorization. No continuous drift observation is implied by demand-only CI, and a human final-run obligation is not equivalent server enforcement.

The optional decision-support addendum is an adopter-selected illustration, not a core dependency. This protocol does not certify legal compliance, reference-product compatibility, production coverage or autonomous release readiness.