Skip to main content

The GitHub Setup Nobody Runs For You

In one line: three of the four Core-mechanized rules are GitHub repository settings; setup/bootstrap.sh makes zero API calls, so until a human performs these four steps those rules exist only in the table that claims them.

This section exists because the omission was measured, not imagined. Asked whether the canon documented the manual GitHub setup, the answers were: Codex/bot review — absent from this document; "GitHub Actions" — zero occurrences; manual setup steps — nowhere in docs/, setup/ or templates/. And setup/bootstrap.sh makes zero GitHub API calls. It writes files. It cannot create a ruleset.

At that earlier audit, this repository had no rulesets or CODEOWNERS while §14 described those controls as mechanized. That is historical evidence, not a statement about current settings. Use an explicit reviewed project policy with scripts/check-repo-config.sh <owner/repo> --policy <policy.json> to assess present enforcement.

Step 1 — Create the ruleset. Settings → Rules → Rulesets → New branch ruleset. Target the default branch, set Enforcement: Active (a ruleset in Evaluate mode reports and blocks nothing), and enable Require status checks to pass.

Step 2 — Add each actual reported check context, and its expected integration where required. Confirm the name against a real workflow result; job IDs, explicit names, reusable workflows and matrices can affect it. A required context that never reports can block every PR, including the PR intended to add the check. Introduce the producing workflow before requiring its verified context.

Add only jobs that should genuinely refuse a merge. A reporting job (SARIF upload, advisory audit) belongs outside the required set, and naming a job honestly — — blocks… versus — reports… — costs nothing and prevents the confusion above.

Step 3 — Configure an approved review integration if selected. Scope repository/source access and credentials before installation. Integrations differ: some post reviews, some emit check runs, and blocking depends on effective settings. Do not infer enforcement from a bot comment or silence. Two traps:

  • GitHub re-anchors older review comments onto newer commits. commit_id says where a comment now points; original_commit_id says what it was written against. Filter on the latter or you will re-litigate findings you already fixed — 18 of 24 comments in one review round were stale re-anchors.
  • A silent bot is not an approving bot. Rate-limited, stalled and clean look identical from outside.

Step 4 — CODEOWNERS, and something that requires it. Add .github/CODEOWNERS mapping safety paths to reviewers, then — in the same ruleset — enable Require a pull request before merging with Require review from Code Owners. A CODEOWNERS file that nothing enforces assigns reviewers and blocks nobody; both halves are the mechanism.

Then verify, because a setting you believe in is not a setting you have:

scripts/check-repo-config.sh <owner/repo> --policy path/to/reviewed-repo-policy.json

The checker first verifies that §14 still names repo-config mechanisms, then evaluates the explicit project's required contexts, safety-floor applicability and concrete safety paths. templates/repo-policy.example.json is a shape example, not an adopted policy. Select real context names and optional integration_id values. A policy decision is a configuration reference, not proof that the tool authenticated its approver.

The implementation reads active effective rules for the default branch, including inherited rules, plus classic protection. It uses the pinned base revision's CODEOWNERS (GitHub precedence), syntax diagnostics and supported path/owner eligibility checks; it never substitutes an unmerged local file or counts a team as one person. The supported pattern subset is *, /exact/file and /directory/; complex patterns and email owners remain UNASSESSED, not guessed. A one-owner repository does not automatically become exempt. Explicit not_applicable scope needs a reason and decision and is printed distinctly from a pass.

Exit 0 means a scoped read-only assessment, 1 means an assessed missing control, and 2 means incomplete assessment. Hidden/failed API reads, truncated trees or omitted bypass fields cannot mean “none.” A classic-protection 404 is conservatively unassessed because missing resources and insufficient visibility can be ambiguous. Visible bypasses are limits on their respective layer, not proof an actor bypasses every protection. These sequential API reads are not an atomic snapshot or a merge-time authorization receipt.

Evidence and deployment: offline process fixtures exercise branch scope, named checks, provider identity, pagination, inherited/classic rules, owner eligibility and unavailable visibility. PR CI runs those fixtures using no real GitHub credentials. The separate manual repo-config-assessment.yml accepts a reviewed policy in the selected revision and uses separately approved read access; the limited default token may correctly return UNASSESSED. Do not broaden untrusted PR credentials to make the result green. No installer, fixture run or workflow file establishes that an adopter's actual controls are configured.