Skip to content

fix(ci): surface blocked autofix takeover admission - #8410

Merged
wenshao merged 12 commits into
QwenLM:mainfrom
qqqys:codex/issue-8409-autofix-takeover-admission
Aug 6, 2026
Merged

wenshao merged 12 commits into
QwenLM:mainfrom
qqqys:codex/issue-8409-autofix-takeover-admission

Conversation

@qqqys

@qqqys qqqys commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

What this PR does

This change makes forced AutoFix review admission explicit and fail-closed. Live pull-request metadata and fork-author permissions are validated and retried up to three times, each rejection receives a stable reason code, and takeover pull requests update their existing AutoFix status to a bilingual blocked state when admission cannot safely continue. Scheduled fork discovery reuses the same permission reader.

The regression coverage executes the admission classifier, transient and terminal lookup paths, the GitHub status-comment update, its failure path, and the complete review-scan shell syntax.

Why it's needed

A trusted Critical review on a managed takeover pull request could route into AutoFix but then be silently rejected by the forced scan after a transient metadata or permission lookup. The prior success status remained visible even though the feedback was not addressed. This keeps review-scan authoritative while making transient failures recoverable and terminal blockers visible without advancing the feedback watermark.

Reviewer Test Plan

How to verify

Confirm that a takeover-labelled fork remains eligible when live metadata and write permission are available after a transient API failure. Confirm that malformed or repeatedly unavailable metadata and permission responses stop admission after three attempts. For an identifiable takeover PR with a terminal permission blocker, confirm that the existing AutoFix status comment is updated with the specific reason and run link; if that comment cannot be read or updated, the scan must fail visibly.

Evidence (Before & After)

Before: forced-scan lookup failures collapsed into one aggregate rejection and could leave the previous successful takeover status visible.

After: admission emits stable reason codes, retries bounded live reads, and behaviorally verifies the blocked-status PATCH and its failure semantics.

Tested on

OS Status
🍏 macOS ✅
🪟 Windows N/A
🐧 Linux N/A

Environment (optional)

Node.js 22 workspace. The focused new regression passes independently; all 110 workflow assertions passed in the complete local run, although the local Vitest worker reported an RPC timeout after publishing the passing results. Type checking, formatting, shell syntax, and diff checks pass.

Risk & Scope

  • Main risk or tradeoff: a forced takeover scan now fails loudly when it cannot make a terminal blocker visible, which may turn a previously silent no-op into a red workflow run.
  • Not validated / out of scope: event-route admission and the independent final review-address write-safety recheck are intentionally unchanged; hosted GitHub Actions CI will provide Linux coverage.
  • Breaking changes / migration notes: none.

Linked Issues

Fixes #8409

中文说明

本 PR 的改动

本次修改让强制触发的 AutoFix 评审准入具备明确原因并保持写入侧 fail-closed。PR 实时元数据和 fork 作者权限都会校验响应结构并最多重试三次;每种拒绝都有稳定原因码;takeover PR 无法安全继续时,会把现有 AutoFix 状态更新为中英双语的 blocked 状态。定时扫描 fork 候选也复用同一权限读取逻辑。

回归测试会实际执行准入分类、临时和终态查询失败、GitHub 状态评论更新及其失败路径,并检查完整 review-scan shell 语法。

为什么需要

受信任的 Critical 评审可以把已接管 PR 路由进 AutoFix,但强制扫描可能因为一次临时元数据或权限查询失败而静默拒绝,旧的成功状态仍然保留,看起来像反馈已经被处理。本修改继续以 review-scan 作为权威实时准入,同时让临时错误可恢复、终态阻塞可见,并且不推进反馈水位。

Reviewer Test Plan

如何验证

确认带 takeover 标签的 fork PR 在元数据和 write 权限首次查询临时失败、后续成功时仍可准入。确认元数据或权限响应畸形、连续不可用时会在三次后停止。对于元数据可识别但权限查询最终失败的 takeover PR,确认现有 AutoFix 状态会更新为包含明确原因和运行链接的 blocked 状态;如果状态评论无法读取或更新,扫描必须明确失败。

证据(修改前后)

修改前:强制扫描的查询失败会合并成一个笼统拒绝,并可能继续显示上一轮成功的 takeover 状态。

修改后:准入输出稳定原因码,实时查询做有界重试,并通过行为测试验证 blocked 状态 PATCH 及其失败语义。

测试平台

OS 状态
🍏 macOS ✅
🪟 Windows N/A
🐧 Linux N/A

环境(可选)

Node.js 22 工作区。新增聚焦回归可独立通过;完整本地运行的 110 个工作流断言全部通过,但本地 Vitest worker 在发布通过结果后报告了一次 RPC 超时。类型检查、格式检查、shell 语法和 diff 检查均通过。

风险与范围

  • 主要风险或取舍:强制 takeover 扫描无法公开终态阻塞时现在会明确失败,过去的静默空跑会变成红色工作流。
  • 未验证或不在范围内:事件路由准入和 review-address 独立的最终写安全复核保持不变;由 GitHub Actions CI 提供 Linux 覆盖。
  • 破坏性变更或迁移说明:无。

关联 Issue

Fixes #8409

@github-actions github-actions Bot added the review/self-reported The linked issue was opened by the PR author (self-reported) label Aug 3, 2026
@qqqys
qqqys dismissed a stale review via ea34938 August 3, 2026 06:09
@wenshao

wenshao commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Code review

Reviewed ea34938 (gh pr diff 8410) against the PR head of .github/workflows/qwen-autofix.yml. I replayed the new classifier and the retry helpers locally rather than reading them only.

What I verified independently

  • forced_admission_reason is behaviourally equivalent to the old OK predicate. I extracted it verbatim and ran it: skip at label index 0 → skip_label (jq treats 0 as truthy, so the old index($skip) | not was already correct and the new != null matches), bot in-repo → eligible, missing isCrossRepository → cross_repo_state_missing, fork without allow-edits → maintainer_edits_disabled. No admission widening.
  • The whole review-scan run block is syntactically valid — extracted and dedented, bash -n is clean.
  • The [[ cond ]] && sleep idiom in both retry loops is safe under -eo pipefail (defaults.run.shell: bash). I confirmed the final iteration's non-zero &&-list does not trip errexit and the loop falls through to return 1.
  • FORCED_PR never reaches this step without a PAT — the fork pull_request:labeled path is already guarded at the route (L516-L526), so the new exit 1 cannot produce the "red run for a label that never engaged anything" failure that comment warns about.

Nice touches: the explicit status_lookup_ok flag is strictly better than the existing status writer at L3466, which conflates "lookup failed" with "no comment exists" via || STATUS_ID=''. And the two new fleet_row calls in the scheduled fork loop close a genuine visibility hole.


Suggestions

1. The blocked comment tells maintainers to wait, for blockers that will never clear (L1779-L1785)

Only maintainer_edits_disabled gets specific guidance. author_permission_read / author_permission_triage and cross_repo_state_missing fall into the else branch and render "A later scheduled scan will retry without advancing the feedback watermark." But the scheduled scan applies the same write+ gate (L1923-L1933), so it will skip the PR on every tick forever. The comment then sits on the PR indefinitely promising a retry that structurally cannot succeed — which is the same class of misleading-status problem this PR sets out to fix.

Worth a third branch for author_permission_*: grant the fork author write+, or drop autofix/takeover.

2. The bot's own fork PRs get no blocked status at all (L1769)

[[ "$(jq -r --arg take "${TAKEOVER_LABEL}" '[.labels[]?.name] | index($take) != null' <<< "${META}")" == 'true' ]] || return 0

The workflow deliberately manages the bot's own fork PRs without a label (header L24-L29: "The bot's OWN fork PRs … are auto-managed WITHOUT a label when allow-edits is on"), and bot-prs.json feeds the fork loop at L1917 for exactly that reason. So when a bot-authored fork PR has allow-edits turned off, or its permission read fails, admission still stops silently — the original bug, unfixed for that class. Gating on "managed" (author == AUTOFIX_BOT or takeover label) rather than the label alone would cover both, and matches how forced_admission_reason itself defines unmanaged_author.

3. Terminal API answers are retried and then reported as transient (L1740-L1752, L1717-L1738)

read_live_permission treats every outcome outside admin|maintain|write|triage|read as retryable. That folds in answers that are terminal, not transient:

  • "permission": "none" — returned for a user with no access (private repos, blocked users). Verified: on this public repo a non-collaborator returns read, so the common case is fine, but none is a documented value of that field.
  • HTTP 404 — verified live: gh api repos/QwenLM/qwen-code/collaborators/<gone>/permission → "… is not a user" (HTTP 404), non-zero exit. A deleted or renamed fork author hits this.

Either produces 3 sleeps, ADMISSION_REASON=permission_lookup_failed, the "will retry" comment from #1 above, and exit 1 (red run) — where the old code produced a clean green terminal rejection. read_forced_pr_meta's schema gate has the same shape: a successful response that fails the structure check (e.g. author: null for a ghost author) is not transient, yet it retries three times and hard-exits.

Splitting "the call failed" (retry) from "the call answered and the answer is terminal" (classify, exit 0) would keep the fail-loud property where it belongs.

4. Both readers discard the API error (2> /dev/null, L1721/L1730/L1743)

The stated goal is making blockers visible, but the only trace left is attempt N/3 — a maintainer debugging a red forced scan cannot tell 404 from 403 from a secondary-rate-limit. This file already has the better pattern twice, in the route job (L470-L490): capture stderr to a temp file and echo it in the warning (::warning::Permission API call failed for ${SENDER_LOGIN}: ${api_error}). Copying it here costs three lines.

5. cross_repo_state_missing is unreachable in production (L1761)

read_forced_pr_meta already rejects a non-boolean .isCrossRepository (L1728), so a forced dispatch with that field missing dies as metadata_fetch_failed before the classifier runs. The new test asserts the branch, which reads as coverage of the documented jq //-trap but exercises a path the workflow can no longer reach. Either drop the field from the schema check (so malformed metadata classifies cleanly instead of erroring) or drop the branch — keeping both makes the fail-closed guarantee look like it lives somewhere it doesn't.

6. jq program interpolation deviates from this file's own convention (L1791)

--jq ".[] | select(.user.login == \"${AUTOFIX_BOT}\") | select(.body | contains(\"<!-- autofix-status -->\")) | .id"

The identical lookup at L3466-L3470 uses --arg ab "${AUTOFIX_BOT}". AUTOFIX_BOT is vars.AUTOFIX_BOT_LOGIN || 'qwen-code-dev-bot', so this is repo-var controlled and low risk, but a " or \ in that var silently rewrites the filter. --arg is free.

7. Dead ${FPERM:-none} fallbacks (L1861, L1931)

After read_live_permission, FPERM is one of the five regex values on success and the empty case has already exited/continued, so author_permission_${FPERM:-none} can never render none at either call site. The reason vocabulary is effectively author_permission_read|author_permission_triage. Harmless, but it reads as if the none case were handled — see #3.

8. Outputs written immediately before exit 1 (L1827-L1828, L1852-L1853, L1869/L1874)

targets=[] / has_targets=false are appended and then the step exits non-zero. review-address requires has_targets == 'true' and the issue-phase job requires needs.review-scan.result == 'success', so neither consumer can observe them on a failed job. Not a bug — but if this is deliberate belt-and-braces it's worth one line of comment, otherwise it's noise in three places.


Test coverage

The behavioural harness is a real step up from string assertions, and neutralising sleep in the fixtures is the right call. Gaps worth closing, given the PR leans on "behaviorally verifies":

  • No end-to-end case for a successful-but-below-write permission. The author_permission_* branch (L1860-L1863) and the blocked-comment text it produces — the most likely real rejection — are never exercised. This is also the case that would surface pre-release: fix ci #1.
  • No case for permission: "none" or a 404 from the permission endpoint (see 如何自定义密钥文件 .env可能与其他文件冲突 #3).
  • The reporter's two early return 0 paths are untested: non-takeover-label (the silent path, Where is the config saved? #2) and a non-matching reason. A regression there re-introduces silence with green tests.
  • actor != AUTOFIX_BOT → return 1 is untested, though it's one of only two ways the terminal path turns red.
  • The PATCH-failure path is untested — the fixture only fails the lookup (FAIL_STATUS_LOOKUP); Failed to update blocked takeover status never runs.
  • expect(reviewScanJob).toContain('exit 1') is vacuous — exit 1 appears throughout a 1000-line script. Either anchor it to the surrounding block or drop it.
  • The fake gh interpolates successOutput into a single-quoted bash string; this only works because the fixture JSON happens to contain no '. A heredoc or an env var would remove the trap for the next person editing the fixture.

Summary

No Critical. The classifier rewrite is sound and I could not find an admission-widening path. The substantive concerns are all on the new reporting surface rather than the gate: #1 and #2 mean the "make terminal blockers visible" promise is only partly delivered (wrong guidance for permission blockers, no coverage for the bot's own fork PRs), and #3/#4 mean some terminal conditions are laundered into transient-looking red runs with no diagnostic in the log. Those three plus the author_permission_* test case are what I'd fix before landing; the rest are nits.

@QwenLM QwenLM deleted a comment Aug 5, 2026
@wenshao

wenshao commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Review: fix(ci): surface blocked autofix takeover admission

Overview

The forced (FORCED_PR) branch of review-scan is refactored from a single boolean OK predicate into three shell helpers plus a jq classifier:

  • read_forced_pr_meta — gh pr view with response-shape validation and 3 bounded retries (replaces || echo '{}').
  • read_live_permission — collaborator permission read with the same retry shape; also reused by the scheduled fork-candidate loop.
  • forced_admission_reason — replaces the boolean with stable reason codes (not_open, wrong_base, skip_label, unmanaged_author, maintainer_edits_disabled, cross_repo_state_missing, eligible).
  • report_forced_takeover_blocked — PATCHes (or posts) the <!-- autofix-status --> comment with a bilingual ⛔ blocked notice for terminal reasons on managed PRs.

Plus a shared qwen-pr-head-write-<pr> concurrency group with review-address, a fleet_row for blocked fork candidates, and ~310 lines of behavioural regression coverage.

The direction is right, and the test approach — extracting the shell VERBATIM and replaying it against a stub gh on PATH — is genuinely strong. Three things I'd want addressed before merge.


🔴 1. permission: "none" is misclassified as a transient lookup failure

read_live_permission accepts only admin|maintain|write|triage|read:

&& [[ "${permission}" =~ ^(admin|maintain|write|triage|read)$ ]]; then

But GET /repos/{owner}/{repo}/collaborators/{username}/permission returns 200 with permission: "none" for a user who is not a collaborator — which is precisely the common case for a fork author on a autofix/takeover PR. "none" fails the regex, so the helper retries 3× and returns 1.

Confirmed by replaying the extracted function against a stub gh that prints none:

::warning::Permission lookup failed for outsider (attempt 1/3)
::warning::Permission lookup failed for outsider (attempt 2/3)
::warning::Permission lookup failed for outsider (attempt 3/3)
RC=1

Consequences, all regressions against main:

  • Forced path (L1874-1882): a fork author with no write access now yields permission_lookup_failed → exit 1 (red run), and the posted notice says "A later scheduled scan will retry without advancing the feedback watermark" — wrong guidance, the condition is terminal. On main this printed permission='none' below write and exited 0.
  • The author_permission_${FPERM:-none} branch at L1888 and its correct "Grant the fork author write access, or remove the autofix/takeover label" message become unreachable — FPERM can now only be triage or read when that case is reached.
  • Scheduled path (L1946): same misclassification, so every non-privileged takeover fork reports blocked / permission_lookup_failed instead of author_permission_none, and costs 3 API calls + 3s of sleep on every scan tick. ${FPERM:-none} at L1958 is likewise dead.

Suggested fix — separate "the call failed" from "the call answered":

if permission="$(gh api "repos/${REPO}/collaborators/${login}/permission" --jq '.permission // ""' 2> /dev/null)" \
  && [[ -n "${permission}" ]]; then
  printf '%s' "${permission}"
  return 0
fi

Any non-empty documented value (including none) is a real answer; only a non-zero gh exit or an empty/malformed payload is transient. Please add a none case to the regression test — its absence is exactly why this slipped through.


🟠 2. Sharing the head-write lock can queue or silently cancel a forced scan

concurrency:
  group: 'qwen-pr-head-write-${{ needs.route.outputs.pr_number || github.run_id }}'
  cancel-in-progress: false

review-address holds this same group with timeout-minutes: 300 and max-parallel legs. Two consequences:

  • review-scan's own timeout-minutes: 15 does not cover queue time, so a forced dispatch on a PR with an address run in flight can now sit pending for up to 5 hours instead of running immediately. On main it ran, hit the busy check, and emitted fleet_row … 'busy' 'address run in flight' — a fast, visible answer.
  • GitHub keeps at most one pending entry per concurrency group; a newer entrant cancels the previously-pending one even with cancel-in-progress: false. So a second forced dispatch (or the next review-address matrix leg entering the group) can cancel the queued review-scan → job cancelled, review-address skipped, no comment, no error. That is the same silent no-op failure class this PR exists to eliminate.

The race being defended against is narrow: the blocked-notice PATCH is the only new write, and it only happens on terminal admission failure. Options worth considering instead of a job-wide lock:

  1. Accept the race and make the write best-effort (consistent with review-address's own status writes — see 如何自定义密钥文件 .env可能与其他文件冲突 #3), or
  2. Re-read the comment immediately before PATCHing and skip if it now carries an in-flight 🔄 AutoFix is working marker from a newer run.

If the job-level group is kept, please at least note in the comment that the scan may queue behind a 300-minute address job.


🟠 3. Fail-closed severity is inconsistent with the rest of the status pipeline

review-address's status writes are explicitly best-effort — the existing comment says "a status post that fails warns and continues — it must never cost the round" — and every failure path there is || echo "::warning::…; continuing.".

The new reporter is the opposite: any failure to publish the notice fails the whole scan. The sharpest edge is the identity gate:

if [[ "${actor}" != "${AUTOFIX_BOT}" ]]; then
  echo "::warning::Blocked takeover status skipped: PAT authenticates as '${actor:-unknown}'"
  return 1
fi

If CI_DEV_BOT_PAT is unset, rotated, or unavailable (fork-originated runs), gh api user fails 3× → actor='' → return 1 → the caller exit 1s. Every terminal takeover admission becomes a red run with no comment written and only a ::warning:: explaining why. Forks and secret rotation are the exact situations where this fires, and there's nothing an operator can do about it from the PR side.

Suggestion: treat "cannot authenticate as the bot" as not applicable (warn, return 0) rather than failed to publish, and reserve return 1 for the cases where the bot identity is confirmed but the write itself failed. At minimum give it a distinct reason code so the red run is diagnosable from the summary.


🟡 Smaller notes

  • cross_repo_state_missing is dead code. read_forced_pr_meta now validates (.isCrossRepository | type == "boolean") before returning, and GraphQL types both isCrossRepository and maintainerCanModify as Boolean! — so META can never reach the classifier with the field missing. Harmless defence-in-depth, but the unit test drives the classifier directly, so it reads as live coverage of a guard that can't fire. Worth a one-line comment saying the branch is unreachable-by-construction and kept deliberately.
  • Warning streams are inconsistent. read_forced_pr_meta / read_live_permission send warnings to >&2 (correct — they're inside command substitution); report_forced_takeover_blocked sends its four warnings to stdout. Not a bug today, but it's a trap if the reporter is ever wrapped in $( ). Make them all >&2.
  • New-comment fallback isn't in the PR description. When no <!-- autofix-status --> comment exists, the reporter posts a fresh one via gh pr comment. The description says takeover PRs "update their existing AutoFix status"; the create path deserves a mention. (Also: review-address posts via gh api "repos/${REPO}/issues/${PR}/comments" — same effect, but the two writers now use different mechanisms for the same comment.)
  • ✅ -f body= (not -F) is the right choice for a body that could contain a leading @; tail -1 matches review-address's | last; schema-validating gh pr view output is a real improvement over || echo '{}'.

Test coverage

The replay harness is good. Gaps, roughly in priority order:

  1. No permission: "none" case — see pre-release: fix ci #1.
  2. No test that a non-managed META returns 0 with zero gh calls. That META gate is the guard preventing AutoFix from commenting on unrelated PRs; it's the one branch worth asserting on call count, and it's untested.
  3. No test for actor != AUTOFIX_BOT → return 1. The stub always answers qwen-code-dev-bot, so the riskiest new failure path (如何自定义密钥文件 .env可能与其他文件冲突 #3) is uncovered.
  4. expect(reviewScanJob).toContain('exit 1') asserts nothing — a 900-line job contains exit 1. Either drop it or anchor it to the surrounding line.
  5. reviewAddressJob = workflow.match(/\n {2}review-address:[\s\S]*$/) is greedy to EOF. It works only because review-address is currently last; a job appended after it makes both toContain assertions silently match the wrong job. Use the (?=\n {2}# ==========) lookahead the other extractors use.
  6. The extraction regexes (\n {10}\}) and .replace(/\n {10}/g, '\n') hard-code YAML indentation. toBeTruthy() catches a total miss, but a partial reindent would de-indent into subtly different shell. A shared helper (extractShellFn(job, name)) with an explicit assertion on the closing brace would be sturdier.
  7. The new it() is ~230 lines covering six distinct behaviours (transient meta, transient perm, terminal perm, terminal meta, six reporter scenarios, bash syntax). Splitting into focused cases would localize failures.

Verdict

Fix #1 (correctness regression, confirmed by repro) before merge. #2 and #3 are design calls where I'd like the author's reasoning — both trade a previously-fast visible no-op for a new way to be slow or red. The rest are polish.

中文说明

总体:把强制路径的布尔 OK 拆成三个 shell helper + jq 分类器,方向正确;测试用"逐字提取 shell 再用桩 gh 真实回放"的做法很扎实。合并前有三点建议。

🔴 1. permission: "none" 被误判为临时失败:read_live_permission 的正则只接受 admin|maintain|write|triage|read,但 GitHub 对非协作者返回 200 + "none"——正是 fork 作者最常见的情况。实测该函数会重试 3 次并返回 1。后果:强制路径把终态的"无写权限"变成 permission_lookup_failed → exit 1(红),且提示语说"后续扫描会重试"(错误引导);author_permission_none 分支和正确的"授予 write 权限"文案变成死代码;定时扫描每个无权限 fork 每轮多 3 次 API + 3 秒 sleep。建议改为"gh 退出码非零或输出为空才算临时失败",并补一个 none 的用例。

🟠 2. 与 review-address 共享 head-write 锁有风险:review-address 的 timeout-minutes: 300 会长时间占用同一 group;review-scan 的 15 分钟超时不覆盖排队时间。且同一 group 只保留一个 pending 任务,新进入者会取消先前 pending 的 review-scan——任务被取消、下游跳过、无评论无报错,正是本 PR 要消灭的静默空跑。原先强制扫描会立刻跑完并输出 busy 行。建议改为写入时乐观复核,或接受竞态按尽力而为处理。

🟠 3. fail-closed 的力度与现有状态流水线不一致:review-address 的状态写入明确是尽力而为("must never cost the round"),新 reporter 则任何失败都让整个 scan 失败。最尖锐的是身份门:CI_DEV_BOT_PAT 未配置/已轮换/fork 触发时,actor 为空 → return 1 → 调用方 exit 1,红色运行且没有任何评论。建议"无法以 bot 身份认证"当作不适用(warn + return 0),return 1 只留给确认身份后写入仍失败的情况。

🟡 其他:cross_repo_state_missing 因为 read_forced_pr_meta 已校验布尔类型(GraphQL 是 Boolean!)而不可达,建议注释说明;reporter 的 warning 走 stdout 而 reader 走 stderr,建议统一 >&2;无状态评论时会新建评论,PR 描述未提及。

测试:缺 none 用例;缺"非托管 META 应零 gh 调用"的断言(这是防止在无关 PR 上评论的关键防线);缺身份不匹配路径;toContain('exit 1') 等于没断言;reviewAddressJob 贪婪匹配到文件末尾,只是碰巧 review-address 是最后一个 job;缩进硬编码的提取正则建议抽成公共 helper;230 行单个 it() 建议拆分。

结论:#1 是确证的正确性回归,合并前修复;#2/#3 想听作者的取舍理由。

Two review blockers on the forced takeover admission gate.

R7-1: `read_live_permission` whitelisted only admin/maintain/write/triage/
read, so GitHub's definitive answers for "holds nothing here" never matched.
Bot-type logins (dependabot[bot], github-actions[bot], renovate[bot]) and org
logins return HTTP 200 with permission 'none'; nonexistent or empty logins
return 404. Both burned three API calls plus back-off and then returned
`permission_lookup_failed`, so the forced path exited 1 (a red run) instead of
the routine `author_permission_none` rejection, the blocked comment promised
"a later scheduled scan will retry" — a retry that can never succeed — and the
actionable "grant the fork author write access" guidance behind
author_permission_* was unreachable. `author_permission_${FPERM:-none}` could
never render `none`. The scheduled loop re-paid the same cost per candidate
per tick, permanently.

Accept 'none', answer HTTP 404 terminally, skip the call for an empty login,
and keep the retry budget for genuinely transient answers (5xx, network, auth)
so a legitimate write-holder is never silently rejected. gh's own stderr now
rides along in the warning instead of going to /dev/null — a rate limit, an
expired PAT and a 5xx were indistinguishable before.

R6-1: the new blocked-status comment lookup is the 14th `--paginate` code
site, but the deliberate site-count pin still asserted 13, failing
`Test (ubuntu-latest, Node 22.x)` deterministically. Bump it to 14 and record
why this site stays out of the `jq -s 'add // []'` normalizer: it consumes the
page stream inline via `--jq ... | .id` into `tail -1` and never lands in a
WORKDIR json file, so wrapping it in an array would break the tail-1 consumer.

R6-2: pin the forced-admission wiring (reader -> classifier -> live-permission
gate -> reporter) end to end, plus fixtures for none, 404, empty login, every
grant level, and a transient 5xx.
@wenshao

wenshao commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Review round 2 — e49aadef

Re-reviewed against the current head (e49aadef, "treat terminal permission answers as a routine rejection"), diffing the forced-admission block and scripts/tests/qwen-autofix-workflow.test.js rather than re-reading the PR description.

✅ Round-1 #1 is fixed and I verified it. read_live_permission now accepts none in the regex (L1770) and short-circuits HTTP 404 and the empty-login case to a terminal none (L1763-1776). The comment at L1748-1759 states the invariant precisely — "none and HTTP 404 are DEFINITIVE answers, not lookup failures" — and the author_permission_* branch it used to strand is reachable again. Good fix, and the reasoning left in the comment is the right thing to leave behind.

Round-1 #2 and #3 are unchanged, and I think both are still worth resolving. One new issue landed with this commit.


🟠 1. Shared qwen-pr-head-write-<pr> lock — still open, and the failure mode is worse than I described last time

# .github/workflows/qwen-autofix.yml:1696-1698
concurrency:
  group: 'qwen-pr-head-write-${{ needs.route.outputs.pr_number || github.run_id }}'
  cancel-in-progress: false

review-address holds the same group (L2910) with timeout-minutes: 300 (L2870). Scheduled scans are unaffected — pr_number is empty, so the group falls back to github.run_id and is run-unique — so this only bites the forced/real-time path, which narrows it. But on that path:

  • The 15-minute budget does not cover queue time. A review event on PR N while an address run for N is in flight now sits pending for up to 5 hours. The pre-existing BUSY_PRS guard already handled this in seconds, and I confirmed a forced PR reaches it: CANDIDATES="${FORCED_PR}" (L1931) flows into for PR in ${CANDIDATES} and hits the busy skip at L2130-2133 → fleet_row … 'busy' 'address run in flight'.
  • The single pending slot is now contended by two different jobs. GitHub keeps at most one pending entry per concurrency group and cancels the previously-pending one when a newer entrant arrives — independent of cancel-in-progress: false. Before this PR only review-address legs were in the group. Now a queued review-scan and a queued review-address leg for the same PR compete for that one slot, in either order: a second review event on PR N can cancel a pending address leg (the round never runs, no report comment), and a newly-queued address leg can cancel a pending scan (the review event is dropped silently). build-cli and review-address both needs: review-scan with no always(), so a cancelled scan takes the whole run with it.

That silent-drop class is exactly what this PR exists to eliminate, so trading it in here feels like the wrong direction. The write being protected is one comment PATCH on a rarely-taken terminal path — scoping the lock to that write (or re-reading the status comment immediately before PATCHing and skipping if it carries a newer 🔄 AutoFix is working marker) buys the same safety without putting a 15-minute job behind a 300-minute one. If the job-level group stays, please at least say so in the concurrency comment — "share its per-PR lock so neither writer can erase the other's state" reads as free.

🟠 2. PAT-identity mismatch still reds the run — and it is the only site in this step that does

# .github/workflows/qwen-autofix.yml:1818-1821
if [[ "${actor}" != "${AUTOFIX_BOT}" ]]; then
  echo "::warning::Blocked takeover status skipped: PAT authenticates as '${actor:-unknown}'"
  return 1
fi

The caller turns that into exit 1 (L1920-1924). The same condition is handled four times in this same step — L2370, L2474, L2520, L2590 — and every one of them warns and continues green, using a memoized SCAN_BOT_ACTOR with || echo 'unknown':

if [[ -z "${SCAN_BOT_ACTOR:-}" ]]; then
  SCAN_BOT_ACTOR="$(gh api user --jq '.login' 2> /dev/null || echo 'unknown')"
fi
if [[ "${SCAN_BOT_ACTOR}" != "${AUTOFIX_BOT}" ]]; then
  echo "::warning::cap-refused notice skipped: PAT authenticates as '${SCAN_BOT_ACTOR}', expected ${AUTOFIX_BOT}"

So a PAT rotation, or AUTOFIX_BOT_LOGIN being set to something the PAT does not match, turns every review event on every managed fork PR into a red run, while the four neighbouring writers in the same shell stay green. "The PAT is not the bot" is a not-applicable condition, not a failed write — I'd warn and return 0, and reserve return 1 for "identity confirmed, write failed". Reusing SCAN_BOT_ACTOR instead of a fresh local actor also drops the extra gh api user calls (up to 3 more per run) for free.

🟠 3. NEW — metadata_fetch_failed reds a run that main exited 0 on, and is the one blocked reason with no PR-visible explanation

# .github/workflows/qwen-autofix.yml:1878-1882
if ! META="$(read_forced_pr_meta)"; then
  echo "::error::Forced PR #${FORCED_PR} admission blocked: metadata_fetch_failed"
  echo "targets=[]" >> "${GITHUB_OUTPUT}"
  echo "has_targets=false" >> "${GITHUB_OUTPUT}"
  exit 1
fi

On main this was META="$(gh pr view … || echo '{}')" → OK=false → a descriptive ❌ #N is not an open main-targeting PR … line → exit 0. Now any gh pr view that fails three times costs 3 calls plus 3 seconds and reds the run. That includes ordinary operator error — a mistyped pr_number on a workflow_dispatch, a number that is an issue rather than a PR, a deleted or transferred PR — none of which is a system fault.

report_forced_takeover_blocked is deliberately not called here (META doesn't exist yet, so the managed-PR gate at L1803-1805 can't run), which is understandable, but it makes metadata_fetch_failed the only blocked reason whose sole trace is a log line inside a red run. That inverts the PR's stated goal. Two options: keep exit 1 but only for a genuinely transient signature, and take exit 0 + the old descriptive line for a 404; or fall back to gh pr view --json number alone and, if that succeeds, post the blocked notice.

While here: the targets=[] / has_targets=false writes immediately before exit 1 (L1880-1881 and again L1905-1906) are dead — a failed job's outputs are never read, and build-cli/review-address gate on needs.review-scan.outputs.has_targets with no always(), so both are skipped outright. Dropping them removes the implication that there is a fall-through contract.

🟡 4. The status-lookup jq diverges from the sibling upsert it is duplicating

# .github/workflows/qwen-autofix.yml:1836-1837
if status_ids="$(gh api "repos/${REPO}/issues/${FORCED_PR}/comments" --paginate \
  --jq ".[] | select(.user.login == \"${AUTOFIX_BOT}\") | select(.body | contains(\"<!-- autofix-status -->\")) | .id" 2> /dev/null)"; then

versus the 'Post autofix status comment' step (~L3691), which does the same read on the same comment:

jq -rs --arg m "${MARKER}" --arg ab "${AUTOFIX_BOT}" \
  '[ .[][] | select((.user.login // "") == $ab) | select((.body // "") | contains($m)) ] | last | .id // empty'

Three deliberate differences in the sibling are dropped here:

  • No // "" guards. A null body aborts the whole filter — real jq, not hypothetical:
    jq: error (at <stdin>:1): null (null) and string ("<!-- autof...) cannot have their containment checked
    rc=5
    
    gh then exits non-zero, all three attempts fail, status_lookup_ok stays false, return 1 → exit 1. A red run that never posts the blocked status it exists to post.
  • ${AUTOFIX_BOT} interpolated into the jq program text instead of --arg. It comes from vars.AUTOFIX_BOT_LOGIN (L97), so it is repo-configurable — a " or \ in that var makes every lookup a jq parse error rather than a mismatch.
  • gh pr comment for the create path where the sibling uses gh api …/comments -f body. Same effect, but two writers on one comment with two mechanisms.

Extracting a single upsert_status_comment helper would collapse this and remove three independent chances to drift.

🟡 Smaller notes

  • cross_repo_state_missing is now provably unreachable, not just probably. read_forced_pr_meta validates (.isCrossRepository | type == "boolean") (L1736) before returning, so the classifier's (((.isCrossRepository == true) or (.isCrossRepository == false)) | not) (L1795) is always false. Only the unit test can produce that string, by feeding META directly and bypassing the reader — so expect(reason(missing)).toBe('cross_repo_state_missing') certifies a branch that cannot fire, while the real malformed-metadata case takes metadata_fetch_failed → exit 1 with no comment. Either drop the branch or comment it as unreachable-by-construction.
  • Hardcoded https://github.com/… in the blocked body (L1832). Every other status writer passes RUN_URL: '${{ github.server_url }}/…' (L3675, L4937) and the report step uses ${GITHUB_SERVER_URL} (L4900). On a proxied/GHES host, the one link this message exists to surface is the broken one, sitting next to correctly-resolved links in the same comment.
  • Warning streams still split. read_forced_pr_meta / read_live_permission write to >&2; all four warnings in report_forced_takeover_blocked (L1815, L1841, L1854, L1862) go to stdout. Raised last round; still a trap if the reporter is ever wrapped in $( ).
  • The status-lookup failure discards gh's stderr (2> /dev/null, L1836) — directly against the rule this same diff establishes 60 lines above: "Surface gh's own diagnosis instead of discarding it: a rate limit, an expired PAT and a 5xx all look identical otherwise." (L1778-1779). On a red run the operator sees only attempt 3/3.
  • Back-off is well below GitHub's. Secondary rate-limit responses carry Retry-After, typically 60s; the sleep 1 / sleep 2 ladder neither reads nor approximates it, so a rate-limited scan burns its three attempts inside 3 seconds and gives up.
  • Six copies of the same retry skeleton landed in one diff (read_forced_pr_meta, read_live_permission, and four loops in the reporter), each for attempt in 1 2 3 / warn / [[ "${attempt}" -lt 3 ]] && sleep "${attempt}". One retry3 "<what>" <cmd…> helper collapses all six and makes a Retry-After change a one-line edit.
  • ${FPERM:-none} (L1914, and L1984 in the scheduled loop) can no longer expand to anything but ${FPERM} — the new reader either fails (handled above) or returns a regex-matched value. Harmless, but it hides the invariant the reader now guarantees.
  • The rejection log lost the author. 🧭 forced fork PR #N rejected: author_permission_read no longer names the login, and neither does the blocked comment; read_live_permission prints it only on the failure path. A maintainer told to "Grant the fork author write access" has to go find who that is, and can't distinguish an unexpected FORK_AUTHOR (bot login, transferred account) from a genuine permission gap. The scheduled loop still logs author ${FAUTHOR}=${FPERM}, so the two paths now disagree.

Tests

The verbatim-extract + stub-gh approach remains the right one, and the reporter stub covering transient actor / PATCH / comment failures is good coverage of the retry paths.

  1. The harnesses run under the wrong shell. defaults.run.shell: 'bash' (L89) makes GitHub invoke bash --noprofile --norc -eo pipefail {0}; every new harness (runReader, runReporter, and the failing-gh blocks) spawns bash -c with no flags. I don't believe anything currently misbehaves under errexit — each helper is called from an if ! or || context, which disables it — but that is a property of the call sites, not of the functions, and the tests are what would catch it changing. set -eo pipefail as the first line of each harness script costs nothing and makes the suite reproduce production semantics.
  2. Still no test for actor != AUTOFIX_BOT → return 1. The stub answers qwen-code-dev-bot and the harness sets AUTOFIX_BOT: 'qwen-code-dev-bot', so the branch in Where is the config saved? #2 above — the riskiest new failure path — has no coverage in either direction.
  3. No null body case for the status lookup, which is Are you interested in AI Terminal? #4 above; the stub returns a bare 123, so the filter itself is never exercised against a realistic comments payload.
  4. Round-1 notes 4-7 (toContain('exit 1') asserting nothing, the greedy reviewAddressJob regex, hard-coded indentation in the extractors, the ~230-line single it()) still stand.

Verdict

Nothing here is a correctness regression on the scale of round-1 #1 — that one is properly fixed. What's left is #1 and #2, where I'd like your reasoning before merge because both convert a previously fast-and-green outcome into a slow or red one, and #3, which is new and cheap to fix. The rest is polish.

中文说明

针对当前 head e49aadef 重新审阅。

✅ 第一轮 #1 已修复且我已验证:read_live_permission 的正则现已接受 none(L1770),并把 HTTP 404 与空 login 直接短路为终态 none(L1763-1776),author_permission_* 分支恢复可达。L1748-1759 的注释把不变式说得很准确,留下这段推理是对的。

🟠 1. 共享 qwen-pr-head-write-<pr> 锁(未处理,且比我上轮描述的更糟):定时扫描不受影响(pr_number 为空 → group 退化为 run 唯一),但强制路径上:15 分钟超时不覆盖排队时间,而已有的 BUSY_PRS 守卫本来几秒就给出答案(我确认强制 PR 确实会走到 L2130 的 busy 跳过);更关键的是同一 group 只保留一个 pending 名额,而现在 review-scan 与 review-address 分片都在这个 group 里争抢:新来的 scan 可能取消 pending 的 address 分片(该轮永不执行、无报告评论),新来的 address 分片也可能取消 pending 的 scan(评审事件被静默丢弃)。下游 build-cli/review-address 没有 always(),scan 被取消整条 run 一起没。建议把锁收敛到那一次评论 PATCH,或写入前复核状态评论上是否已有更新的 🔄 AutoFix is working 标记。

🟠 2. PAT 身份不匹配仍然让整条 run 变红:L1818-1821 return 1 → 调用方 exit 1。同一 step 内 L2370/L2474/L2520/L2586 四处相同条件全部是 warning + 继续(且复用了 memo 的 SCAN_BOT_ACTOR)。PAT 轮换或 AUTOFIX_BOT_LOGIN 配置不符时,所有托管 fork PR 的每个评审事件都会变红。"PAT 不是 bot" 属于"不适用"而非"写入失败",建议 warn + return 0,并复用 SCAN_BOT_ACTOR 省掉最多 3 次 gh api user。

🟠 3. 新增问题 —— metadata_fetch_failed 把 main 上绿色退出的情况变成红色,且是唯一没有 PR 可见说明的阻塞原因:main 是 || echo '{}' → OK=false → 描述性提示 → exit 0;现在 gh pr view 三次失败即 exit 1。dispatch 输入的 PR 号打错、传的是 issue、PR 已删除/迁移都会命中。此处无法调用 report_forced_takeover_blocked(META 尚不存在,托管门槛过不了),于是它成了唯一只在红色 run 的日志里留痕的阻塞原因,与本 PR 的目标相反。另:exit 1 前的 targets=[]/has_targets=false(L1880-1881、L1905-1906)是死代码。

🟡 4. 状态查找的 jq 与被它复制的兄弟实现出现分歧(L1836-1837 vs ~L3691):缺 // "" 空值守卫(实测 body: null 会让整个 filter 以 rc=5 失败 → 三次全败 → return 1 → 红色 run 且没发出那条本该发的阻塞状态);${AUTOFIX_BOT} 被拼进 jq 程序文本而非用 --arg(该值来自 vars.AUTOFIX_BOT_LOGIN,可配置,含 " 即解析错误);新建评论走 gh pr comment 而兄弟走 gh api -f body。建议抽出统一的 upsert_status_comment。

🟡 其他:cross_repo_state_missing 现在是可证明不可达(reader 已校验布尔类型),只有绕过 reader 的单测能造出它;L1832 硬编码 github.com,而其他状态写入都用 github.server_url/GITHUB_SERVER_URL;reporter 的四条 warning 仍走 stdout;状态查找用 2> /dev/null 丢弃 gh 诊断,恰好违反同一 diff 在 L1778-1779 立下的规则;1s/2s 退避远低于 GitHub 的 Retry-After(约 60s);同一 diff 里出现 6 份完全相同的重试骨架,建议抽 retry3;${FPERM:-none} 已不可能为空;拒绝日志不再打印 fork 作者,而定时循环仍打印,两条路径措辞不一致。

测试:所有新 harness 用 bash -c,而生产是 bash --noprofile --norc -eo pipefail(L89)——目前没有实际错行为(各 helper 都在 if !/|| 上下文里,errexit 被禁用),但那是调用点的性质而非函数本身的性质,加一行 set -eo pipefail 零成本;仍缺 actor != AUTOFIX_BOT → return 1 的用例;缺 body: null 的用例;第一轮的 4-7 条(toContain('exit 1') 等于没断言、reviewAddressJob 贪婪匹配、缩进硬编码、230 行单个 it())依旧成立。

结论:这轮没有第一轮 #1 那种级别的正确性回归。剩下 #1、#2 想听你的取舍理由(两者都把原本"快且绿"的结果换成了"慢或红"),#3 是新引入且修起来便宜,其余为打磨项。

Addresses review round 2 on e49aade.

- Status-comment lookup now uses the same jq filter as the sibling upsert in
  'Post autofix status comment': `// ""` guards so one comment with a null
  body cannot abort the program (verified: the old filter exits 5, so all
  three attempts fail and the run reds out without posting the very status
  it exists to post), and --arg for AUTOFIX_BOT so a repo-configured login
  containing " or \ is a mismatch rather than a jq parse error.
- The lookup no longer sends gh's stderr to /dev/null, matching the rule
  read_live_permission states 60 lines above.
- The blocked body resolves its run link from GITHUB_SERVER_URL like every
  other status writer, instead of hardcoding github.com.
- All four reporter warnings go to stderr like the two reader helpers, so
  the reporter stays safe to wrap in $( ).

Tests: the reporter and reader harnesses now run under production shell
options (`set -eo pipefail`) and call through `|| exit $?` so errexit is
suspended inside the helper exactly as the `if !` call sites do. New cases
cover the null-bodied comment page, the PAT-identity mismatch branch, the
in-repo pass-through (no collaborator call), and the permission_lookup_failed
red exit. Each kills a probe-verified mutant that previously survived:
dropping the `// ""` guard, dropping the `isCrossRepository == true`
conjunct, and flipping that `exit 1` to `exit 0`.

Verified: npm run test:scripts 946 passed / 9 skipped, 46 files;
prettier --check clean; eslint clean; git diff --check clean.
@qqqys

qqqys commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

已修复(commit 5d45d9f32),针对 review round 2 的 #4 与部分 smaller notes;#1 / #2 / #3 未改动,等你的取舍。

Fixed in 5d45d9f32 — round-2 item #4 and three of the smaller notes. #1, #2 and #3 are deliberately untouched, see below.

Fixed

  • Are you interested in AI Terminal? #4 — the status-lookup jq diverged from the sibling upsert. The lookup now runs the same filter as Post autofix status comment: // "" guards on both .user.login and .body, and --arg for the bot login and the marker instead of interpolating them into the program text. Reproduced your rc=5 first — the old filter against a page whose first bot comment has body: null:
    jq: error (at <stdin>:0): null (null) and string ("<!-- autof...) cannot have their containment checked
    rc=5
    
    The new filter returns 123 on the same input, 456 on two concatenated pages (the pre-2.31 gh shape), and exits 0 on an empty stream. Kept as an inline id stream into tail -1, so the WORKDIR page normalizer still does not apply and the toBe(9) pin is unchanged; --paginate site count stays 14.
  • Hardcoded https://github.com/… in the blocked body → ${GITHUB_SERVER_URL}, matching every other status writer.
  • 2> /dev/null on the status lookup → gh's own stderr now rides along in the per-attempt warning, the rule stated 60 lines above for read_live_permission.
  • Split warning streams → all four reporter warnings now go to >&2 like the two reader helpers.

Tests

  • Harnesses now run under production shell options. Every harness (runReader, runReporter, runGate, and the failing-gh blocks) starts with set -eo pipefail and calls through || exit $?, so errexit is suspended inside the helper exactly as the if ! / || call sites suspend it — both halves of the production semantics, not just one.
  • actor != AUTOFIX_BOT → return 1 now has a case in both directions (status 1, warning names the actor, no comment written).
  • The reporter stub returns a realistic comments page — objects rather than a bare 123, including a null-bodied bot comment — so the filter itself is exercised.
  • runGate gained the two missing scenarios: in-repo pass-through (admitted by label alone, asserting no collaborator call) and a 502-on-every-attempt permission lookup (status 1, blocked comment posted first).

Each new case kills a mutant that previously survived — I re-ran the suite against all three:

Mutant Before After
drop the // "" body guard green ✗ expected { status: 1 } to deeply equal { status: 0 }
drop the isCrossRepository == true conjunct green (your R8-3 probe) ✗ expected '🧭 … rejected: aut…' to contain 'ADMITTED:eligible'
flip permission_lookup_failed's exit 1 to exit 0 green (your R8-3 probe) ✗ expected +0 to be 1

Verification

  • npm run test:scripts — 946 passed, 9 skipped, 46 files, 0 failed (qwen-autofix-workflow.test.js 116/116).
  • npx prettier --check on both changed files — clean.
  • npx eslint scripts/tests/qwen-autofix-workflow.test.js — clean.
  • git diff --check — clean.

Not changed, on purpose

#1 (shared qwen-pr-head-write-<pr> lock), #2 (PAT-identity mismatch reds the run) and #3 (metadata_fetch_failed reds a run main exited 0 on) all change which outcome a run reports, and you asked for reasoning before merge on #1 and #2 while #3 offers two designs. I have not picked one unilaterally — leaving those to the author/maintainer decision rather than guessing. The remaining polish notes (the retry3 extraction, cross_repo_state_missing reachability, Retry-After-aware backoff, the dead targets=[] writes before exit 1, the rejection log dropping the author) are untouched this round.

@wenshao

wenshao commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

@qwen-code /triage

…enLM#8410)

Review round 9 raised two Suggestion-level findings on the forced
admission path; both are addressed here.

R9-1 (.github/workflows/qwen-autofix.yml:1698) — the review-scan
concurrency group was broader than the job's own `if:`. GitHub
evaluates concurrency before the job condition, and `route` emits
`pr_number` unconditionally from the dispatch input, so a
`workflow_dispatch` with `phase: issue` + `pr_number: N` resolved the
group to `qwen-pr-head-write-N` even though `do_review` is false and
the job only skips. Because `issue-autofix` declares
`needs: ['route', 'review-scan']`, the skipped leg could queue behind
an in-flight address round for PR N (`timeout-minutes: 300`,
`cancel-in-progress: false`) and stall the issue phase the operator
actually dispatched, with only a "queued" badge to explain it. The
group now carries the same `do_review == 'true'` conjunct, so a run
that will skip falls to the per-run `github.run_id` group. The forced
path's serialization against review-address is unchanged.

R9-2 (scripts/tests/qwen-autofix-workflow.test.js:4034) — the pin
asserted both jobs' groups as independent literals and never compared
the two prefixes, so renaming one side while updating its literal in
the same block would ship green with the lost-update race on the
status comment silently reopened. The test now extracts both prefixes
and asserts equality, mirroring `groupOf` in
qwen-resolve-workflow.test.js, and a new case pins the group predicate
against the job's own `if:` block.

Verified: qwen-autofix / qwen-resolve / qwen-triage workflow suites
266/266 pass. Both new assertions were mutation-checked — reverting
the group to the broad predicate fails 2 tests, and renaming only
review-scan's prefix fails the equality assertion. Prettier and ESLint
clean (the group is double-quoted because the expression embeds
'true', which Prettier will not leave escaped).
@qqqys

qqqys commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

已修复 round 9 的两条 Suggestion(commit bf6d79656)。三条 Critical 仍按上轮说明保留,等待维护者取舍。

Fixed both round-9 suggestions in bf6d79656. The three [Critical] items remain deliberately unchanged — as the review itself notes, they are pending maintainer judgment calls, not code defects.

R9-1 — .github/workflows/qwen-autofix.yml:1698. Confirmed the mechanism: route computes ROUTE_PR from the dispatch input and emits pr_number unconditionally (line 641), while phase: issue leaves DO_REVIEW=false (the forced-routing block that would flip it only applies in auto/empty phase). So phase: issue + pr_number: N did resolve the group to qwen-pr-head-write-N for a job that only skips, and issue-autofix (needs: ['route', 'review-scan']) idled behind it. The group now carries the same do_review == 'true' conjunct, so a skipping run falls to github.run_id. Serialization against review-address on the real forced path is unchanged.

R9-2 — scripts/tests/qwen-autofix-workflow.test.js. The pin now extracts both prefixes and asserts equality, mirroring groupOf in qwen-resolve-workflow.test.js, plus a new case pinning the group predicate against the job's own if: block.

Verification

Check Result
qwen-autofix + qwen-resolve + qwen-triage workflow suites 266/266 passed
Mutation: revert group to the broad predicate 2 tests fail ✅
Mutation: rename only review-scan's prefix equality assertion fails ('qwen-pr-scan-write' to be 'qwen-pr-head-write') ✅
prettier --check, eslint on both changed files clean

Note on style: the group is now double-quoted. Its expression embeds 'true' and contains no double quotes, so Prettier rewrites the escaped-'' single-quoted form; the sibling runs-on lines keep '' only because they also contain ".

@wenshao

wenshao commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

@qwen-code /triage

wenshao added a commit that referenced this pull request Aug 6, 2026
wenshao added a commit that referenced this pull request Aug 6, 2026
@wenshao

wenshao commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Maintainer verification — local real-stack run of review-scan

I rebuilt this PR's admission path locally and ran it for real rather than reading the diff. Summary: the behaviour this PR claims is real and verified, and the harness also turned up one genuine defect in the new blocked-status writer, plus two smaller notes worth a look before merge.

How it was verified (harness)

The point was to avoid stubbing anything above the network:

  • The Scan for PRs with new feedback step's run: block is extracted byte-for-byte from .github/workflows/qwen-autofix.yml (no rewriting, only the block indent removed) and executed by bash.
  • It runs in a Linux container (runs-on: ubuntu-latest parity — GNU date -d, bash 5.2), with the real gh CLI v2.96.0 and jq 1.7.1.
  • api.github.com is mapped to 127.0.0.1 in the container and served by a local origin over genuine TLS (self-signed cert installed in the container trust store). So gh's own argument parsing, --paginate Link-header following, --jq filtering, GraphQL encoding and error handling are all exercised unmodified — only the origin's answers are ours.
  • The workflow-level env: map is extracted from the YAML too, so no constant is hand-copied.
  • Every HTTP request is written to a ledger, so assertions are on what actually went over the wire, not on log text.

31 scenarios × {main base, PR head} = 64 full step executions, 303 real HTTP requests.
Verified head bf6d7965 (its review-scan run: block is byte-identical to 5d45d9f3); base = merge-base 89b3d5ea.

Confirmed working

Injected condition main (before) PR #8410 (after)
Fork permission read: 500 → write rejected, dispatch lost admitted after 1 retry, exit 0
Fork permission read: 500 ×3 silent reject, exit 0, stale ✅ status stays permission_lookup_failed, existing status comment PATCHed to the blocked body, exit 1
Permission = HTTP 404 rejected (leaks {"message":"Not Found"} into the log) author_permission_none, exactly 1 API call, routine exit 0
Permission = none / read (200) same aggregate message author_permission_none / author_permission_read, 1 call
Maintainer edits off generic rejection maintainer_edits_disabled + "Re-enable maintainer edits…" guidance
gh pr view 500 → 200 || echo '{}' → silent green no-op retried, admitted
gh pr view 500 ×3 silent green no-op metadata_fetch_failed, exit 1
Status PATCH 500 ×3 n/a 3 bounded attempts, ::error::blocked status update failed, exit 1
Status comment absent n/a posts a new one (addComment) with the same marker
PAT authenticates as a human n/a refuses to write, ::error::, exit 1
DRY_RUN=true n/a no write at all (ledger confirms), exit 1
not_open / wrong_base / skip_label / unmanaged_author one aggregate message distinct reason codes, no status write (label/reason gate holds)
Scheduled scan, fork permission 500 ×3 skipped … below write, no fleet row blocked / permission_lookup_failed fleet row in the run summary
Scheduled scan, permission read / 404 no fleet row blocked / author_permission_read / author_permission_none

Edge probes that also passed: a null-body comment in the list does not abort the jq filter (the // "" guard holds); an AUTOFIX_BOT_LOGIN containing " and \ is a mismatch, not a jq parse error (the --arg guard holds); a status comment living on page 2 is found correctly through --paginate + .[][]; the run link is built from GITHUB_SERVER_URL, and the bilingual body renders as intended (screenshot 1).

The PR's own suite: 117/117 pass on the head commit.

verification


🐞 Defect found — connection-level failure on the status read is neither retried nor surfaced

report_forced_takeover_blocked reads the status comments through a pipeline:

if status_ids="$(gh api ".../comments" --paginate 2> "${err}" |
  jq -rs --arg ab "${AUTOFIX_BOT}" --arg m '<!-- autofix-status -->' '…')"; then
  status_lookup_ok=true

The review-scan step does not set pipefail (the sibling Post autofix status comment step does — it starts with set -uo pipefail; this one only mentions pipefail in a comment). So the if tests jq's exit status, not gh's. When gh fails with an HTTP status it prints the error body to stdout and jq errors out, so the retry works — that path is fine. But when gh fails at the connection level (TCP reset / TLS abort / DNS blip) stdout is empty, jq -rs happily produces nothing and exits 0.

Verified end-to-end (scenario: terminal author_permission_none, status read answered with a TCP RST):

  • 1 status read instead of 3 — the documented bounded retry never happens
  • status_lookup_ok=true although nothing was read
  • a duplicate ⛔ AutoFix blocked comment is posted while the stale ✅ round 3 pushed comment stays — the exact two-status state this PR exists to remove
  • the run exits 0, green

Re-running the identical scenario with the one-line change below gives 3 bounded retries, gh's real diagnosis in the warning, Failed to read takeover status comments, and exit 1 — i.e. the contract the PR description states:

if status_ids="$(set -o pipefail; gh api ".../comments" --paginate 2> "${err}" |
  jq -rs …)"; then

defect


Smaller notes (non-blocking)

  1. npm run test:scripts exits 1 on macOS. On this head, scripts/tests/qwen-autofix-workflow.test.js reports 117/117 passed but the process still exits 1 on an unhandled [vitest-worker]: Timeout calling "onTaskUpdate". Deterministic here — 3/3 runs on macOS (file runtime 62–86 s); the same file on the merge-base is exit 0, 112 passed, 55 s. Linux CI is green, and the Test (macos-latest) job was skipped in that run, so this is currently unobserved by CI rather than proven safe there. Worth a look, since the file is not excluded on darwin (only on win32).

  2. A 404 on the forced PR lookup is a terminal answer treated as transient. read_forced_pr_meta retries any gh pr view failure three times, so a mistyped pr_number in a manual dispatch burns 3 calls + back-off and then reds out with metadata_fetch_failed — a transient-sounding code for a permanent condition. This is the same distinction the PR already draws for permissions (e49aade); mirroring it here (grep -q 'HTTP 404' → pr_not_found) would be consistent.

  3. The jq -e schema gate on the metadata is effectively unreachable through real gh. gh pr view --json always emits every requested field, so the "malformed metadata" branch cannot fire in practice — a deleted-account author comes through as login: "app/" (still a string), and the run then correctly ends at author_permission_none. Harmless belt-and-braces, but the transient-vs-malformed coverage it implies is theoretical.

Not covered

  • The concurrency: group change in bf6d7965 is a job-level expression, evaluated by the Actions engine before the step runs — it is outside this harness. I only reviewed it by reading; its semantics (do_review != 'true' → github.run_id) look right, but it is untested here.
  • Everything downstream of admission (review-address, the push path, the agent round itself).
  • Real GitHub rate-limit / secondary-limit behaviour.

Verdict: the admission changes do what the description says, before/after is clearly demonstrable, and the reason codes + status write behave correctly under every failure mode I could inject. I'd like the pipefail fix in before merge; the three notes can follow up.

中文版本

维护者验证 —— 在本地对 review-scan 做真实环境运行

我没有只读 diff,而是把这个 PR 的准入路径在本地真实跑了一遍。结论:PR 声称的行为属实且已验证;同时harness 也发现了新版 blocked-status 写入逻辑里的一个真实缺陷,外加两三个合并前值得看一眼的小问题。

验证方法(harness)

目标是:网络层以上的东西一律不打桩。

  • Scan for PRs with new feedback 这一步的 run: 脚本块从 .github/workflows/qwen-autofix.yml 里逐字节抽出(不做任何改写,只去掉块缩进),交给 bash 执行。
  • 运行在 Linux 容器里(对齐 runs-on: ubuntu-latest:GNU date -d、bash 5.2),使用真实 gh CLI v2.96.0 与 jq 1.7.1。
  • 容器内把 api.github.com 指向 127.0.0.1,由本地服务通过真实 TLS 提供(自签证书装进容器信任库)。因此 gh 自己的参数解析、--paginate 的 Link 头翻页、--jq 过滤、GraphQL 编码与错误处理全部原样执行 —— 只有源站的应答是我们控制的。
  • workflow 顶层 env: 也是从 YAML 抽取的,没有任何常量是手抄的。
  • 每一次 HTTP 请求都记入 ledger,所以断言基于真正发出去的请求,而不是日志文本。

31 个场景 × {main 基线, PR head} = 64 次完整步骤执行,303 次真实 HTTP 请求。
验证的 head 为 bf6d7965(其 review-scan 的 run: 块与 5d45d9f3 逐字节相同);基线为 merge-base 89b3d5ea。

确认有效的部分

注入条件 main(修改前) PR #8410(修改后)
fork 权限查询:500 → write 拒绝,本次调度丢失 重试一次后准入,exit 0
fork 权限查询:连续 3 次 500 静默拒绝、exit 0,旧的 ✅ 状态仍在 permission_lookup_failed,现有状态评论被 PATCH 成 blocked 内容,exit 1
权限返回 HTTP 404 拒绝(且把 {"message":"Not Found"} 泄进日志) author_permission_none,只调用 1 次 API,常规拒绝 exit 0
权限为 none / read(200) 同一句笼统信息 author_permission_none / author_permission_read,1 次调用
未允许 maintainer edits 笼统拒绝 maintainer_edits_disabled + “请重新允许 maintainer edits” 指引
gh pr view 500 → 200 || echo '{}' → 静默绿色空跑 重试后准入
gh pr view 连续 3 次 500 静默绿色空跑 metadata_fetch_failed,exit 1
状态 PATCH 连续 3 次 500 无此路径 3 次有界重试,::error::blocked status update failed,exit 1
不存在状态评论 无此路径 以相同 marker 新建一条(addComment)
PAT 身份是人类账号 无此路径 拒绝写入,::error::,exit 1
DRY_RUN=true 无此路径 完全不写(ledger 可证),exit 1
not_open / wrong_base / skip_label / unmanaged_author 一句笼统信息 各自独立原因码,且不写状态评论(标签/原因门控生效)
定时扫描,fork 权限连续 3 次 500 skipped … below write,无 fleet 行 run summary 出现 blocked / permission_lookup_failed fleet 行
定时扫描,权限 read / 404 无 fleet 行 blocked / author_permission_read / author_permission_none

另外通过的边界探针:评论列表里存在 body 为 null 的评论不会让 jq 中断(// "" 护栏有效);AUTOFIX_BOT_LOGIN 含 " 和 \ 时是不匹配而非 jq 解析错误(--arg 护栏有效);状态评论位于第 2 页时经 --paginate + .[][] 也能正确定位;run 链接来自 GITHUB_SERVER_URL,中英双语正文渲染符合预期(见截图 1)。

PR 自带测试:head 提交上 117/117 通过。


🐞 发现的缺陷 —— 连接层失败时,状态评论读取既不重试也不暴露

report_forced_takeover_blocked 通过管道读取状态评论:

if status_ids="$(gh api ".../comments" --paginate 2> "${err}" |
  jq -rs --arg ab "${AUTOFIX_BOT}" --arg m '<!-- autofix-status -->' '…')"; then
  status_lookup_ok=true

review-scan 这一步没有 pipefail(兄弟步骤 Post autofix status comment 有 —— 它以 set -uo pipefail 开头;本步骤里 pipefail 只出现在注释中)。因此 if 判断的是 jq 的退出码而非 gh 的。当 gh 因 HTTP 状态码失败时,它会把错误体打到 stdout,jq 随之报错,所以那条路径没问题。但当 gh 在连接层失败(TCP reset / TLS 中断 / DNS 抖动)时,stdout 是空的,jq -rs 什么都不输出并以 0 退出。

端到端验证结果(场景:终态 author_permission_none,状态评论读取被 TCP RST 打断):

  • 状态读取只发生 1 次而非 3 次 —— 文档承诺的有界重试根本没发生
  • 明明什么都没读到,status_lookup_ok 却为 true
  • 新发了一条重复的 ⛔ AutoFix blocked 评论,而旧的 ✅ round 3 pushed 仍然留在 PR 上 —— 正是本 PR 想消除的“两条状态并存”状态
  • 本次运行 exit 0,绿色

把同一场景换成下面这一行改动后重跑:3 次有界重试、warning 里带出 gh 的真实诊断、Failed to read takeover status comments、exit 1 —— 即 PR 描述所声明的契约:

if status_ids="$(set -o pipefail; gh api ".../comments" --paginate 2> "${err}" |
  jq -rs …)"; then

其他较小的问题(不阻塞)

  1. npm run test:scripts 在 macOS 上 exit 1。 在本 head 上,scripts/tests/qwen-autofix-workflow.test.js 报告 117/117 通过,但进程仍以 1 退出,原因是未处理的 [vitest-worker]: Timeout calling "onTaskUpdate"。本机可稳定复现 —— macOS 上 3/3(该文件耗时 62–86 秒);同一文件在 merge-base 上是 exit 0、112 通过、55 秒。Linux CI 是绿的,且那次绿色运行里 Test (macos-latest) 是 skipped,所以这属于“CI 目前没观察到”,而不是“已证明在 macOS 上没事”。该文件只在 win32 被排除、darwin 并未排除,值得看一下。

  2. 强制 PR 元数据查询遇到 404 是终态,却被当作临时错误。 read_forced_pr_meta 对任何 gh pr view 失败都重试三次,因此手动 dispatch 时写错 pr_number 会白白消耗 3 次调用加退避,最后以 metadata_fetch_failed 变红 —— 用一个听起来像“临时”的原因码描述一个永久状态。这正是本 PR 已经为权限查询做出的区分(e49aade);在这里同样处理(grep -q 'HTTP 404' → pr_not_found)会更一致。

  3. 元数据上的 jq -e 结构校验,在真实 gh 下实际上不可达。 gh pr view --json 总会输出所请求的全部字段,所以“畸形元数据”分支在实践中不会触发 —— 账号已注销的作者会以 login: "app/" 出现(仍是字符串),随后流程正确地停在 author_permission_none。作为兜底无害,但它所暗示的“临时失败 vs 结构畸形”的覆盖只是理论上的。

未覆盖范围

  • bf6d7965 里的 concurrency: 改动属于 job 级表达式,由 Actions 引擎在步骤运行前求值,不在本 harness 范围内。我只是读过:其语义(do_review != 'true' → github.run_id)看起来正确,但这里没有实测。
  • 准入之后的一切(review-address、推送路径、agent 轮次本身)。
  • GitHub 真实的 rate limit / secondary limit 行为。

结论: 准入相关改动确实做到了描述所声称的事情,前后对比清晰可证,且在我能注入的每种失败模式下,原因码与状态写入行为都正确。希望合并前先修掉 pipefail 这一处;其余三点可以后续跟进。

…LM#8410)

Maintainer verification of the forced-admission path found the one `if`
in `report_forced_takeover_blocked` that tests a PIPELINE rather than a
single command:

    if status_ids="$(gh api .../comments --paginate 2> "${err}" | jq -rs ...)"

A gh failure carrying an HTTP status prints the error body to stdout, so
jq chokes on it and the bounded retry fires. A CONNECTION-level failure
(TCP reset, TLS abort, DNS blip) leaves stdout empty — `jq -rs` then
prints nothing and exits 0. Absent pipefail that reads as success on
nothing read: the loop breaks on attempt 1, `status_lookup_ok` goes true,
and the empty id routes the writer to its "no status comment yet" branch,
posting a DUPLICATE blocked comment beside the stale one, run green.

Set the option locally on that command substitution. `defaults.run.shell:
bash` expands to `bash --noprofile --norc -eo pipefail`, so the step is
already pipefail on a real runner and this is redundant today; it is the
only guard that survives that default changing or the helper moving to a
step that sets its own options.

Tests: replay the same connection-level failure with the ambient pipefail
dropped and pin 3 bounded reads, exit 1, and no comment written; add the
HTTP-status half so the two failure shapes stay distinguishable and the
local option cannot be "simplified" away as carried by jq; pin both the
`set -o pipefail` and `defaults.run.shell: 'bash'` textually, since the
ambient half is what every other gh|jq writer in this file relies on
without saying so.

Both mutations verified to fail the suite.
@qqqys

qqqys commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

已修复(commit d8d66b5b0):report_forced_takeover_blocked 的状态评论读取现在自带 set -o pipefail。三条小问题本轮未动。

Fixed the blocking pipefail item in d8d66b5b0. The three smaller notes are untouched this round.

The change

.github/workflows/qwen-autofix.yml — exactly the form you proposed:

if status_ids="$(set -o pipefail; gh api "repos/${REPO}/issues/${FORCED_PR}/comments" --paginate 2> "${err}" |
  jq -rs --arg ab "${AUTOFIX_BOT}" --arg m '<!-- autofix-status -->' \
    '.[][] | select((.user.login // "") == $ab) | select((.body // "") | contains($m)) | .id')"; then

One factual note on severity, since it changes what the fix is for rather than whether to take it: this file sets defaults.run.shell: 'bash' (line 87), which GitHub expands to bash --noprofile --norc -eo pipefail {0} — so review-scan does run with pipefail on a real runner, and the duplicate-comment path your harness reproduced needs the harness's plain bash to appear. Your reading of the mechanism is exactly right either way, and the option is now carried locally, so it holds regardless of that default or of the helper moving to a step that sets its own options. The # pipefail comment you spotted in this step now says which of the two it means.

Verification

Both halves of the failure are now replayed against the extracted helper, and I checked each new assertion bites by mutating the source:

Scenario (gh stub) Shell Reads Exit Status comment written
connection-level: stdout empty, diagnosis on stderr, rc=1 set -eo pipefail 3 1 none
connection-level (same) set -e only — ambient pipefail dropped 3 1 none
HTTP-level: error body on stdout, rc=1 set -e only 3 1 —

The second row is the new one: it is the case that fails without the local option. Mutation checks —

  • remove set -o pipefail; → that row reports exit 0 instead of 1 (1 failed | 116 passed), i.e. exactly the silent-green duplicate you demonstrated
  • remove the defaults: block → the new textual pin fails (1 failed | 116 passed)

The third row is there so the jq-chokes-on-the-body path stays pinned separately and the local option cannot later be read as redundant with it.

Commands run on d8d66b5b0:

  • npx vitest run --config scripts/tests/vitest.config.ts scripts/tests/qwen-autofix-workflow.test.js scripts/tests/qwen-resolve-workflow.test.js scripts/tests/qwen-fleet-shepherd-workflow.test.js scripts/tests/package-scripts.test.js → 173/173 pass (autofix suite 117/117)
  • npm run build → ok; npm run typecheck → ok
  • npx prettier --check + npx eslint on both changed files → clean
  • YAML re-parsed after the edit (8 jobs, defaults.run.shell: bash), and the suite's own bash -n over the whole review-scan run block still passes

Note 1 (macOS test:scripts exit 1) I could not reproduce — this is a Linux box, where the file is exit 0 at 117/117 in ~7s. Notes 2 and 3 I've left as follow-ups per your "non-blocking".

Picks up main so `Test (ubuntu-latest, Node 22.x)` stops failing. CI checks
out `refs/pull/8410/head` but runs the base branch's `ci.yml`, so main's
"Check voice guard mirror sync" step ran `npm run check:voice-guard-sync`
against this branch's older `package.json`, which predates that script
(added in QwenLM#8350) -> `npm error Missing script`, exit 1. Nothing in this PR
caused it; the branch was simply 17 commits behind.

Merge resolution: both sides had added a `const reviewAddressJob` in
`scripts/tests/qwen-autofix-workflow.test.js`, in different hunks, so git
merged them textually into a duplicate `const` (SyntaxError at import,
whole suite unloadable). Kept main's bounded slice
`(?=\n {2}[a-z][a-z0-9-]*:\n|$)` and dropped this branch's older
unbounded `[\s\S]*$` form — main's is strictly more general (it still
allows EOF, so it keeps working while review-address is the last job, and
it shrinks correctly once a job is appended after it).
@qqqys

qqqys commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

已修复(merge commit dfc6ec551):沙箱验证报告的那条 Critical(reviewAddressJob 重复声明)以及 Test (ubuntu-latest, Node 22.x) 的红灯,根因都是本分支落后 main 17 个 commit。

Fixed in merge commit dfc6ec551. Both the sandboxed verification's Critical and the red Test (ubuntu-latest, Node 22.x) had the same root cause — this branch was 17 commits behind main. Neither was caused by a change in this PR.

1. Test (ubuntu-latest, Node 22.x) — npm error Missing script: "check:voice-guard-sync"

job 92688890346 died at the Check voice guard mirror sync step, exit 1.

The asymmetry is the cause: for pull_request, the workflow definition comes from the base branch, but ci.yml:242 checks out refs/pull/8410/head. So main's step run: 'npm run check:voice-guard-sync' (ci.yml:358) ran against this branch's package.json, which predates that script — it landed in main with #8350 (732f4d8a2), after this branch's merge-base 89b3d5ea8. Nothing on the PR side to fix; the branch just had to catch up.

After the merge, locally: npm run check:voice-guard-sync → Voice guard mirror check passed. (exit 0).

2. The sandboxed verification Critical — duplicate const reviewAddressJob

Confirmed and fixed, with one correction to the attribution: the duplicate does not exist on the PR head. On d8d66b5b0 (and on bf6d79656c, the sha you verified) scripts/tests/qwen-autofix-workflow.test.js declares reviewAddressJob exactly once. main also declares it exactly once. The duplicate is created by the merge: both sides added the declaration in different hunks, so git took both textually without raising a conflict. That is why it only shows up on the committed/merged tree — which is precisely the tree that matters for merging, so the finding stands.

Resolution — kept main's slice, dropped this branch's older one:

// dropped (this branch, older — assumes review-address is the last job):
const reviewAddressJob =
  workflow.match(/\n {2}review-address:[\s\S]*$/)?.[0] ?? '';

// kept (main, bounded on the generic next-job shape, still allows EOF):
const reviewAddressJob =
  workflow.match(
    /\n {2}review-address:[\s\S]*?(?=\n {2}[a-z][a-z0-9-]*:\n|$)/,
  )?.[0] ?? '';

main's form is strictly more general: the |$ alternative keeps it correct while review-address is last, and it shrinks correctly once a job is appended after it. No assertion changed.

Verification on the merged tree (dfc6ec551)

  • npm run test:scripts → 47 files, 961 passed / 9 skipped, exit 0 (qwen-autofix-workflow.test.js alone: 120/120, up from 117 — main added 3)
  • npm run build → ok · npm run typecheck → ok
  • npm run check:voice-guard-sync → pass · npm run check:lockfile → pass · npm run check:desktop-isolation → pass
  • npx prettier --check and npx eslint on scripts/tests/qwen-autofix-workflow.test.js → both clean (the no-redeclare you reported is gone)

The merge is a fast-forward on top of d8d66b5b0; the only manual resolution in it is the two-line deletion above (git show dfc6ec551 shows nothing else).

The three smaller notes from the maintainer verification round remain untouched this round, as stated previously.

wenshao added a commit that referenced this pull request Aug 6, 2026
@wenshao

wenshao commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Re-verification after d8d66b5 — and a correction to my own report

Re-ran the whole harness against dfc6ec5 (merge-base now 037b4d9). First, the correction, because it is mine to make.

⚠️ My severity call on the pipefail finding was wrong

d8d66b5's commit message is right and my report was not: this workflow sets defaults.run.shell: 'bash', which GitHub expands to bash --noprofile --norc -eo pipefail. My harness launched the extracted step with plain bash. That single infidelity is what made the status-read failure look reachable in production. It isn't — the ambient pipefail already covered it on every real runner.

I fixed the harness to launch the step exactly as the runner does, and re-ran the scenario as a 2×2:

bf6d7965 (no local pipefail) d8d66b5 ($(set -o pipefail; …))
runner shell -eo pipefail — production ✔ 3/3 reads, nothing written, exit 1 ✔ 3/3 reads, nothing written, exit 1
ambient pipefail dropped — hypothetical ✘ 1/3 reads, duplicate ⛔ comment, exit 0 ✔ 3/3 reads, nothing written, exit 1

The failure lives only in the bottom-left cell. So: not a production bug, and I should not have called it one. The fix is still worth keeping for the reason the commit gives — it is the only guard that does not depend on a workflow-level default staying put or on this helper never moving to a step that sets its own options — and the two new tests (connection-level vs HTTP-level failure, with ambient pipefail dropped) pin exactly the right distinction. Pinning defaults.run.shell: 'bash' textually is a good call too: the other gh … | jq … writers in this file silently depend on it.

Also worth noting for anyone else building a harness against this file: -e is part of that same default, and I re-checked it — it changes nothing in this step, because every call site of the new helpers is an if ! / || context, which suspends errexit inside the call.

re-verification

Full matrix re-run under the runner shell

All 31 scenarios × {merge-base 037b4d9, head dfc6ec5}, now with bash --noprofile --norc -eo pipefail. Every admission outcome, reason code, retry count and status write is byte-identical to my first run — so the whole "Confirmed working" table in my earlier comment stands as written, only now on a faithful shell:

  • transient permission 500 → write: admitted after one retry
  • permission 500 ×3: permission_lookup_failed, existing status comment PATCHed to the blocked body, exit 1
  • permission 404 / none / read: terminal, 1 API call, author_permission_*, routine exit 0
  • maintainer_edits_disabled, metadata_fetch_failed, PAT-identity refusal, DRY_RUN writes nothing, distinct reason codes for not_open / wrong_base / skip_label / unmanaged_author, scheduled-scan blocked fleet rows — all unchanged
  • edge probes unchanged: null comment body, AUTOFIX_BOT_LOGIN containing " and \, status comment on page 2, GITHUB_SERVER_URL-derived run link

The PR's own suite on dfc6ec5: 120/120 pass.

Status of my other three notes

  1. test:scripts exit 1 on macOS — still there, but I have to soften how I described it. On this head it is intermittent: 2 of 6 runs exit 1 with the unhandled [vitest-worker]: Timeout calling "onTaskUpdate", always on the slower runs (~78 s file runtime vs ~58 s when green); tests are 120/120 either way. On the earlier head I saw 3/3, so "deterministic" was an artefact of a small sample. Still worth a look — the file is only excluded on win32, and this is a process-level non-zero exit, not a test failure.
  2. 404 on the forced PR lookup treated as transient — open. read_forced_pr_meta still retries a nonexistent pr_number three times and reports metadata_fetch_failed. Verified again on dfc6ec5: 3 pr_view calls, exit 1.
  3. jq -e metadata schema gate unreachable through real gh — open, and harmless. Informational only.

Verdict

Unchanged apart from the retraction: the admission changes do what the description says, and I could not find a production-reachable failure in the forced or scheduled path across 31 injected conditions. My earlier "I'd like the pipefail fix in before merge" was based on my harness bug, not on the code — d8d66b5 is a good defensive addition rather than a required fix. From my side this is ready to merge; notes 1–3 are follow-up material.

中文版本

d8d66b5 之后的复验 —— 以及对我自己报告的更正

针对 dfc6ec5 重跑了整套 harness(merge-base 现为 037b4d9)。先说更正,因为这是我的问题。

⚠️ 我对 pipefail 那条发现的严重性判断是错的

d8d66b5 的 commit message 是对的,我的报告不对:本 workflow 设置了 defaults.run.shell: 'bash',GitHub 会把它展开成 bash --noprofile --norc -eo pipefail。而我的 harness 是用普通 bash 启动抽出来的步骤的。 正是这一处不忠实,让状态读取的问题看起来在生产环境可达。实际上并不可达 —— 环境里本来就有的 pipefail 在任何真实 runner 上都已经覆盖了它。

我把 harness 改成完全按 runner 的方式启动步骤,并把该场景重跑成一个 2×2:

bf6d7965(无局部 pipefail) d8d66b5($(set -o pipefail; …))
runner shell -eo pipefail —— 生产 ✔ 读 3/3 次,不写入,exit 1 ✔ 读 3/3 次,不写入,exit 1
去掉环境 pipefail —— 假设 ✘ 只读 1/3 次,重复 ⛔ 评论,exit 0 ✔ 读 3/3 次,不写入,exit 1

失败只存在于左下角那一格。所以:这不是生产缺陷,我不该那样定性。 这个修复仍然值得保留,理由就是 commit 里写的 —— 它是唯一不依赖「workflow 级默认值保持不变」或「这个 helper 永远不被挪到自设选项的步骤里」的护栏;而新增的两个测试(连接层失败 vs HTTP 层失败,且都去掉环境 pipefail)正好钉住了这个区分。把 defaults.run.shell: 'bash' 做文本断言也是对的:本文件里其他 gh … | jq … 写入者都在无声地依赖它。

另外提醒任何要为这个文件搭 harness 的人:-e 也是同一个默认值的一部分,我一并复核过 —— 它在这一步没有任何影响,因为新 helper 的每一处调用点都是 if ! / || 上下文,会在调用内部挂起 errexit。

在 runner shell 下重跑完整矩阵

全部 31 个场景 × {merge-base 037b4d9,head dfc6ec5},这次用 bash --noprofile --norc -eo pipefail。所有准入结果、原因码、重试次数、状态写入与我第一轮完全一致 —— 因此前一条评论里那张「确认有效」的表格原样成立,只是这次跑在忠实的 shell 上:

  • 权限查询 500 → write:重试一次后准入
  • 权限连续 3 次 500:permission_lookup_failed,现有状态评论被 PATCH 成 blocked 内容,exit 1
  • 权限 404 / none / read:终态,只调 1 次 API,author_permission_*,常规 exit 0
  • maintainer_edits_disabled、metadata_fetch_failed、PAT 身份拒写、DRY_RUN 不写入、not_open / wrong_base / skip_label / unmanaged_author 各自独立原因码、定时扫描的 blocked fleet 行 —— 全部不变
  • 边界探针不变:body 为 null 的评论、含 " 与 \ 的 AUTOFIX_BOT_LOGIN、位于第 2 页的状态评论、来自 GITHUB_SERVER_URL 的 run 链接

PR 自带测试在 dfc6ec5 上:120/120 通过。

其余三条的状态

  1. macOS 上 test:scripts exit 1 —— 依然存在,但我得修正之前的描述。在这个 head 上是间歇性的:6 次里 2 次 exit 1(未处理的 [vitest-worker]: Timeout calling "onTaskUpdate"),且都发生在较慢的那几次(文件耗时约 78 秒,绿色时约 58 秒);两种情况下测试都是 120/120。之前的 head 上我看到 3/3,所以「稳定复现」是样本太小的假象。仍值得看一下 —— 该文件只在 win32 被排除,而这是进程级的非零退出,不是测试失败。
  2. 强制 PR 查询的 404 被当作临时错误 —— 仍未处理。read_forced_pr_meta 对不存在的 pr_number 仍重试三次并报 metadata_fetch_failed。在 dfc6ec5 上再次验证:3 次 pr_view 调用,exit 1。
  3. 元数据的 jq -e 结构校验在真实 gh 下不可达 —— 仍未处理,且无害,仅供参考。

结论

除了那处撤回之外结论不变:准入相关改动确实做到了描述所声称的事情,在 31 种注入条件下,我没能在强制路径或定时路径中找到生产可达的失败。我之前说「希望合并前先修掉 pipefail」是基于我自己的 harness bug,而不是代码本身 —— d8d66b5 属于良好的防御性补强,而非必需修复。就我这边而言可以合并;1–3 三条作为后续跟进即可。

@wenshao

wenshao commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

@qwen-code /triage

@wenshao
wenshao enabled auto-merge August 6, 2026 22:41
@QwenLM QwenLM deleted a comment Aug 6, 2026
@QwenLM QwenLM deleted a comment Aug 6, 2026
@QwenLM QwenLM deleted a comment Aug 6, 2026
@QwenLM QwenLM deleted a comment Aug 6, 2026
@QwenLM QwenLM deleted a comment Aug 6, 2026
@wenshao
wenshao added this pull request to the merge queue Aug 6, 2026
Merged via the queue into QwenLM:main with commit d5e4770 Aug 6, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review/self-reported The linked issue was opened by the PR author (self-reported)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(autofix): prevent silent takeover admission mismatches

2 participants