Skip to content

/review: verification probes mutate the shared worktree while reverse auditors read it #9207

Description

@wenshao

What happened?

Step-4 verification agents run mutation probes directly inside the shared review worktree. While a round-5 reverse auditor ran concurrently, the worktree carried an un-reverted probe mutation (compose-review.ts modified with the probe's mutant + a leftover __probe__.test.ts), and the auditor read the mutated tree before noticing.

The auditor nearly filed a false Critical over the probe residue; it recovered by improvising a fallback the briefs never mention — verifying against git show HEAD: instead of the working tree. Two other agents in the same run also reported the residue ("worktree contamination … act before further verification"). Nothing corrupted the final review, but only because concurrent agents happened to notice and improvise correctly.

What did you expect to happen?

Either:

  • verification probes run on a private scratch copy (e.g. a throwaway git worktree add torn down after the probe), leaving the review worktree pristine at all times; or
  • reverse-auditor and chunk-agent briefs explicitly state that the working tree may be transiently mutated by concurrent verifiers, and all code evidence must be taken from git show HEAD: (the commit under review), never the working tree.

Client information

Anything else we need to know?

The verifier briefs already require "leave the tree exactly as found", and verifiers did restore the tree at the end of their run — the exposure window is during a verifier's probes, while auditors of the pipelined reverse-audit round are reading the same tree. The pipelined design (verifiers riding with the next audit round) is what makes the window structural rather than incidental.

中文

发生了什么?

Step-4 验证 agent 直接在共享审查工作树上运行突变探针。在第 5 轮反向审计员并发运行期间,工作树上带着一个未还原的探针突变(compose-review.ts 被改为探针突变体 + 遗留的 __probe__.test.ts),审计员在察觉之前读到了被突变的树。

该审计员差点把探针残留当成真问题报出假 Critical;它靠一个 brief 从未提及的临场办法恢复——以 git show HEAD: 而非工作树取证。同轮还有两个 agent 报告了残留("worktree contamination … act before further verification")。最终审查未被污染,仅因并发 agent 恰好察觉并正确应对。

期望的行为?

二选一:

  • 验证探针在私有 scratch 副本上执行(如用后即拆的临时 git worktree add),审查工作树始终保持干净;或
  • 在反向审计员/分块 agent 的 brief 中明确声明:工作树可能被并发验证员临时突变,一切代码证据必须取自 git show HEAD:(被审提交),而非工作树。

其他

验证员 brief 已要求"树保持原样离开",验证员也确实会在其运行结束时还原——暴露窗口在验证员施加探针"期间",而流水线化反审轮的审计员正在读同一棵树。流水线设计(验证与下一轮审计同批执行)使该窗口成为结构性而非偶发。

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    category/developmentDevelopment experiencepriority/P2Medium - Moderately impactful, noticeable problemscope/testingTest frameworks and casestype/bugSomething isn't working as expected

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions