Skip to main content

Investigative Discipline

In one line: Investigate every symptom against current production code before declaring it stale, transient, or unrelated.

Do this: When a test fails, inspect the selected implementation, test oracle, environment and recent changes. Git chronology is an investigation lead, not proof that either the implementation or test is wrong.

What: When a symptom appears (failing test, error log, user-reported bug), the default first move is to investigate the symptom against current production code before classifying it. Three rules govern the investigation:

  1. A failing result requires investigation. A pre-existing failure can reveal a real defect, an obsolete expectation or an environmental problem. Establish which applies before changing the test or classifying it as unrelated.

  2. Misattributed-cause is technical debt. When a defensive guard is added with a comment that guesses at the cause (rather than documenting the cause that was actually verified), future debuggers will waste time chasing the wrong hypothesis. Comments must distinguish "verified" from "hypothesized" and be updated when the actual cause is later identified.

  3. Convergent design intuition is a hypothesis, not verification. People and agents can share assumptions, training influences and missing context. Record their agreement as proposal history, then challenge the premises and validate the design against independent observations or acceptance cases.

Why: Three distinct failure modes are addressed by these rules.

The "test is stale" mis-classification is the most common: a test fails on master, the developer runs git stash and observes the failure existed before their change, then declares "pre-existing — therefore stale" and moves on. The "pre-existing" half of that inference is correct; the "therefore stale" half is the wrong leap. A pre-existing failing test could be a pre-existing real bug — and frequently is. The shortcut bypasses the step that would have caught the regression.

The "misattributed cause" failure happens when a developer (or AI agent) hits a symptom they don't have time to fully diagnose. They write a defensive workaround, then write a comment guessing at the cause. The workaround works (because it covers the real cause too, even though the guess was wrong). Months later, another developer encounters a similar symptom, reads the comment, and chases the wrong hypothesis for hours before realizing the original attribution was speculation. The cost compounds across the codebase as defensive guards accumulate.

Agreement can help a team choose which idea to investigate, but repeating a claim does not independently support it. A test only adds evidence for the actual behavior it observes; a test derived from the same mistaken assumption may agree with the implementation and still miss the business requirement.

Evidence: Each rule has a concrete failure it prevents:

  • Stale-misclassification: a failing test gets tagged "stale" after a git stash shows the failure is pre-existing — but "pre-existing" is not "wrong." A canary test that was the bug's only detector gets silenced, and the bug ships. The rule forces a prod-vs-test git log comparison before any stale verdict.
  • Misattributed cause: a defensive guard ships with a comment guessing at the cause. The guard works (it covers the real cause too), the guess is wrong, and a later debugger chases the wrong hypothesis for hours. The rule forces comments to mark causes verified vs. hypothesized.
  • Convergent intuition: a shared assumption can survive several reviews. Record what independent observation challenges it and which acceptance case would fail if it were wrong.

Use a controlled failing case to demonstrate the investigation path. A history comparison alone cannot establish the cause or justify a changed business expectation.

How: Three operational mechanisms encode these rules.

For failure investigation, compare the histories of the implementation and test, reproduce against a pinned candidate in an authorized disposable environment, and check the oracle against approved intent. A deliberate code change does not itself authorize a changed expectation. Correct an obsolete test only with evidence of the intended behavior and the required owner's approval for any amendment. Live effects require separate authority; do not use production as an implicit diagnostic fixture.

For "misattributed-cause is technical debt," code comments on defensive guards must distinguish verified from hypothesized causes. The format is structural: Verified <date> via <method>: <cause> for known causes, Hypothesized <date>: <cause> (untested; defensive guard works regardless) for guesses. When the actual cause is later identified, the comment is updated and any cross-references in memory entries are corrected. Memory entries that documented disproven hypotheses retain the original framing at the bottom for historical fidelity but lead with the corrected diagnosis.

For convergent intuition, name the shared premise, the observation that could contradict it and the result of that check. A second reviewer should inspect the premise, not merely endorse the first review. Agreement is neither business approval nor evidence that the candidate behaves correctly.

For a documentation-modernization-pass project shape that exercises all three rules at scale, see skill:s4u-doc-excellence.