Repository navigation
feat(core): add a convergence reminder for read-only exploration - #13601
yiliang114 wants to merge 2 commits into
Conversation
Share an advisory exploration allowance across core and daemon tool loops without adding a halt or changing tool permissions. Co-authored-by: Qwen-Coder <[email protected]>
#13321 native acceptance: read-only exploration reminderPASS for both actual supported paths: headless core CLI and ordinary daemon/ACP Session. Both executed 102 real anonymous file reads, inserted one exploration reminder after the initial 101 results, accepted the additional read, and completed naturally. This controlled-provider run verifies runtime mechanics and wire insertion; it does not demonstrate production-model semantic convergence or token savings. Candidate: Actual scenarios and observationsEach isolated profile left
Indices are zero-based wire-message indices. The reminder is a user-role text message beginning The provider records classify managed-memory by its actual Final answer in both paths:
Executed-byte identityThe read-only Node load hook delegated to the normal loader and recorded actual load paths and SHA-256 values without transforming source. Core loaded its own candidate budget/client/CLI modules. The daemon loaded those modules plus its own candidate
Actual tmux capturesThese are excerpts from Core final excerpt: Ordinary daemon/ACP capture: Scope and evidenceThe provider and daemon were anonymous loopback fixtures. The core used the real noninteractive Productive-phase/reset/permission/disabled/retry controls were not additional native scenarios here. Local check receipts confirm 649 focused core budget/loop/client tests passing and 12 executed CLI Session tests passing; 1,181 CLI tests were skipped, including all 38 reducer cases. Those targeted-check results are separate evidence from the native observations. The first tmux launch passed non-executable owned shell scripts directly and exited before any product process or provider request. It was corrected to invoke those scripts with Evidence is under the owned Cleanup complete: both scenario tmux sessions and provider session removed; owned provider/daemon listeners closed; no owned native process remains; owned profile/runtime/workspace directories removed. Anonymous input manifests, controlled-provider/helper snapshots, raw evidence and sanitized captures remain. No product source, Git/GitHub, dependency or build action was performed by this verifier. 中文说明两个真实运行路径均通过:headless core CLI 与普通 daemon/ACP Session 各实际读取 102 个匿名文件,前 101 个工具结果之后的下一次主请求均只出现一条探索提醒。随后第 102 次真实读取仍正常执行;再下一次主请求包含全部 102 个精确结果,只保留原有提醒,没有追加第二条。core 正常退出 0;daemon 收到 配置没有显式设置 100 次硬上限,使用现有默认自适应 allowance。此次观测证明提醒在真实运行时正确插入、工具仍可继续使用且未被强迫写入或新增 halt。模型回复来自受控匿名本地 provider,因此不能据此声称生产模型一定收敛、减少 token 或产生有效研究结论。managed-memory 与 suggestion 的辅助请求按实际请求标记区分,它们携带历史中的同一条提醒,不计为主会话的再次插入。 运行前后源码与实际加载 JS 的 SHA-256 均一致,源码哈希已与完整候选 commit 中的文件核对。报告保留实际 tmux 抓取及脱敏版本,连跨行折断的 UUID 一并脱敏。所有自有 provider、daemon、tmux、临时 profile/runtime/workspace 已清理;原始请求、事件、telemetry、捕获和匿名输入清单保留。权限、重试及 reset 等边界以本地定向测试为独立证据,本报告没有扩大为第三次 native 场景,也没有声称执行被跳过的 CLI 测试。 |
|
Thanks — re-running the gate at the current head Template looks good ✓ — every heading from the template is present and filled in, including Problem: the mechanism gap is real and I verified it on main rather than taking the description's word for it. Direction: in scope, and more in-idiom than I expected from the description. Runaway-exploration bounding is already shipped policy here (adaptive cap, hard backstop, Size: core paths are touched ( Approach: scope feels right and the diff is genuinely minimal — zero deletions, no drive-by refactor, no formatting churn, no unrelated files. Both design docs are present with reciprocal language links and matching section structure, which is what AGENTS.md asks for. I considered hosting the counter inside Risk: Stage 1e matched one high-risk path — Moving on to code review. 🔍 中文说明感谢贡献 —— 本次在当前 head 模板完整 ✓ —— 模板要求的每个标题都在且填写了,包括 **问题:**机制缺口是真实的,我在 main 上核实过,没有只采信 PR 描述。 **方向:**在范围内,而且比描述读起来更贴合本仓库既有做法。约束失控探索在这里已经是既定策略(自适应上限、硬兜底、 **规模:**触及核心路径( **方案:**范围合理,diff 确实最小化 —— 零删除、没有顺手重构、没有格式抖动、没有无关文件。两份设计文档都在,带互相指向的语言链接且章节结构对应,符合 AGENTS.md 的要求。我考虑过把计数器放进 **风险:**Stage 1e 命中一条高风险路径 —— 进入代码审查 🔍 — Qwen Code · qwen3.8-max-2026-09-02 Reviewed at |
Code reviewNo critical blockers and no AGENTS.md violations. One low-severity finding worth a one-line fix, named below. I wrote my independent proposal before opening the diff: a counter for the continuous read-only phase, reset at logical-turn boundaries, emitting one advisory text part at the existing allowance. My first instinct was to host it inside What I verified against main rather than reading for plausibility:
Finding (low severity, not a blocker): Two things I traced and concluded are fine, mentioned so you know they were checked rather than missed. Core's Added tests genuinely pin the change rather than passing alongside it: the budget test asserts the threshold, one-shot suppression, phase resets on sequenceDiagram
participant P1 as Model stream
participant P2 as LlmClient or Session loop
participant P3 as ToolExplorationBudget
participant P4 as Next provider request
P1->>P2: ToolCallRequest events
P2->>P3: record the resolved Kind (Read, Search, Fetch count, anything else resets the phase)
P1->>P2: Finished or Retry
P2->>P3: commit on Finished, rollback on Retry
Note over P2,P3: ModelFallback also discards the attempt but does not roll back
P2->>P3: takeReminder with the per-turn allowance, on a tool-result turn
P3-->>P2: reminder text once, then suppressed for the phase
P2->>P4: tool results, then the reminder text
Note over P2,P3: reset at all four logical-turn boundaries
TestingThis is an unattended CI re-run, so per the gate rules I executed nothing from this PR — no build, no test, no All 34 check-runs on 34 check-runs on the reviewed commit — 0 failure, 0 pending. Java SDK matrix grouped into one row.
Sandboxed verification would settle the part CI cannot: Not verified, and why: production-model convergence and token savings — no evidence of any kind, and the PR explicitly places them out of scope, so I am treating the reminder as an unproven-but-cheap nudge rather than a demonstrated win. Also not verified: Windows and Linux runtime behaviour, which the author's own table marks untested (the diff has no platform-specific surface, so I do not consider this a gap worth blocking on). The author's tmux captures, request histories, and source-hash identities in this thread are the author's claims, not evidence I re-ran — I could not, and did not treat them as a substitute. 中文说明代码审查没有关键阻断项,也没有违反 AGENTS.md 的地方。有一条低严重度发现,建议一行修复,写在下面。 我在看 diff 之前先写了自己的方案:一个统计连续只读阶段的计数器,在逻辑轮次边界重置,到达现有 allowance 时输出一条建议性 text part。我最初的想法是把它放进 我对照 main 核实(而不是看起来合理就放过)的内容:
发现(低严重度,非阻断): 另外两处我追查后认为没问题,写出来是为了让你知道它们被检查过而不是被漏掉。core 的 新增测试确实固定了改动,而不是与之并行通过:budget 测试断言阈值、一次性抑制、 (时序图见上,中文不重复:流程是 ToolCallRequest 逐事件 record,Finished 时 commit、Retry 时 rollback,工具结果轮次上按 allowance 调 takeReminder 拿到一次性提醒并随工具结果一起发给模型,四个逻辑轮次边界处 reset;图中也标出了 ModelFallback 不回滚这一点。) 测试本次是无人值守的 CI 重跑,按闸门规则我没有执行 PR 里的任何东西 —— 没有构建、没有跑测试、没有
(CI 表格见上方机器可读区域,中文不重复:34 个 check,0 失败 0 pending,Java SDK 矩阵合并为一行。) 沙箱验证能补上 CI 补不了的部分: 未验证的部分及原因:生产模型的收敛效果与 token 节省 —— 完全没有证据,且 PR 明确将其列为范围之外,所以我把它当作一条未经证明但成本很低的提示,而不是已被证实的收益。同样未验证:Windows 与 Linux 的运行时行为,作者自己的表格也标注未测(diff 没有平台相关面,所以我不认为这是值得阻断的缺口)。作者在本线程中给出的 tmux capture、请求历史与源码哈希身份是作者的主张,不是我重跑得到的证据 —— 我无法重跑,也没有拿它们替代证据。 — Qwen Code · qwen3.8-max-2026-09-02 Reviewed at |
|
独立复核确认:本 PR 实现的"只读探索收敛提醒"与 #13321 的诉求一致,设计与代码改动自洽。下面给出按 PR head 提醒阈值与 disable/Infinity 语义对齐
core 路径:record / commit / rollback 与 loop guard 同源
这与 提醒插入点 reset 覆盖面budget 的 kind 分类与桥接 MCP 工具
一个非阻塞观察
测试
结论:设计与实现一致、与既有 loop-detector / cap 语义对齐、测试到位,建议合入。 |
|
Confidence: 4/5 — the code is sound and I verified the wiring end to end; the one nit is a single missing event type, and my only real reservation is a product judgment that is not mine to make and is already parked with the assigned maintainer. To be explicit about what this score is not: the fork- Going back to my independent proposal: I would have reached for the same thing, and for the same reason the PR did. My first idea — host the counter in On whether the problem exists — I did not accept the framing. The historical session in #13321 was never reproduced on main and its version was never recorded, so the user-level harm rests on one unreproduced incident. But the mechanism gap is checkable and I checked it: What I would flag for the maintainer, none of it blocking:
Would I curse whoever wrote this in six months? No. It is 140 additive lines with zero deletions, no drive-by refactor, tests that fail if you remove the feature, and design docs in both languages with reciprocal links. The class has four methods and no state machine cleverness, and the reset wiring is complete at all four logical-turn boundaries — which is the thing that usually rots in this area and the thing I spent most of my time on. So: approving on the code. I am not signing off on the product direction, and I want that to be unambiguous — #13321 is still open and carries 中文说明信心:4/5 —— 代码是可靠的,接线我从头到尾核实过;唯一的瑕疵是少了一个事件类型,而我真正的保留意见是一个不该由我做的产品判断,并且已经停在被指派的 maintainer 那里。 明确说明这个分数不是什么:fork 回到我最初的独立方案:我会做出同样的东西,而且理由和 PR 一致。我最早的想法 —— 把计数器放进 关于问题是否存在 —— 我没有照单接受 PR 的叙述。#13321 里那次历史会话从未在 main 上复现,版本也没有留档,所以用户层面的损害只建立在一次未复现的事件上。但机制缺口是可核查的,我核查了: 以下是我会提给 maintainer 的点,都不构成阻断:
六个月后我会骂写这段代码的人吗?不会。140 行纯新增、零删除、没有顺手重构、移除功能测试就会失败、两种语言的设计文档带互相指向的链接。那个类只有四个方法,没有状态机式的花招,而 reset 接线在全部四个逻辑轮次边界上都完整 —— 这恰恰是这个区域最容易腐坏的地方,也是花时间最多的地方。 所以:就代码而言我批准。我没有为产品方向背书,这一点我要说清楚 —— #13321 仍然 open 并带 — Qwen Code · qwen3.8-max-2026-09-02 Reviewed at |
recordDaemonToolCalls fed the exploration budget at the top of its per-call loop, before the per-turn cap and global-duplicate checks. A batch that those checks halt is skipped whole and never executes, yet its calls still counted toward the budget and still resolved every tool name through the registry. Move the recording after both halt checks so only calls the turn actually runs feed the budget, and a capped turn performs no registry lookups. Also complete the client-goal Config double with getMaxToolCallsPerTurn and getToolRegistry, which the new budget call sites require. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-conflict/jmuy5teh2cj
|
Added I am retaining 中文:已补 |
|
Current-head follow-up at The new additive commit moves exploration-budget recording past the existing cap and duplicate halt checks and completes the client-goal Config mock. This verification keeps the current daemon path attributable to its changed runtime bytes. Local checks: 56 client-goal tests passed with zero skips; two selected Session cap/duplicate cases passed (1,153 other cases intentionally skipped). The selected cases verify the existing halt behavior, not the internal exploration counter of a discarded batch. Fresh CLI build/typecheck passed. The core production tree is unchanged by this follow-up, so the earlier core headless run is retained at its original recorded identity rather than rerun or relabeled. PR #13601 current-head daemon/ACP verificationPASS at The isolated profile left
Positions are zero-based wire-message indices. The reminder first appeared after the 101 results. The continuation still advertised Source bytes matched the current committed files. Pre/post source and compiled-module hashes were unchanged. A read-only Node load hook delegated to the normal loader and recorded the actual CLI and ACP-child loads, including the current Session module and core budget module. Generated commit metadata still reports
Actual owned tmux capture excerpt (identifiers, local paths and process IDs sanitized; original capture retained): Only this changed daemon path was run. Core headless behavior, real-model semantics, duplicate/cap negative controls, reset, retry and permission controls were not additional native scenarios. No product source, dependencies, Git/GitHub state or global services were changed. Raw requests/responses, HTTP receipts, SSE events, module loads, telemetry, file manifest and original tmux capture are retained. The result and identity receipts record the assertions above. Cleanup verified: both owned tmux sessions and processes stopped, both owned listeners closed, and the owned profile/runtime/workspace directories removed. Raw evidence remains. 中文说明当前提交 该提示没有造成新的终止:同一获准 prompt 收到唯一的 加载记录证明真实 ACP 子进程使用当前编译后的 Session 和 core budget 模块;运行前后源代码与编译产物哈希一致。生成的版本元信息仍显示较早的 清理已核实完成:本次 provider 与 daemon 进程、两个 tmux 会话和监听端口均已关闭,只删除本次独立配置、运行目录及匿名输入文件;原始请求、事件、tmux 和哈希证据保留。 中文补充:当前 2525fee 增量提交后,本地 56 个 client-goal 测试及两个定向 cap/重复调用 Session 用例通过;其余 1,153 个 Session 用例未执行,不计入通过数。fresh CLI build/typecheck 通过。此次额外原生验收仅覆盖变更后的 daemon/ACP 路径;core 生产逻辑没有变化,先前 headless 报告继续保留原提交身份。仍按 |
|
Qwen Code review did not complete successfully. The review pipeline failed before a review could be posted. A transient error is retried automatically; if you are seeing this, retry with |
|
The corrected productive-phase control passed at The summary was derived from actual read payloads and matched the independent expected bytes exactly: 1,377 bytes, SHA-256 All 26 scoped source/runtime identities and HEAD stayed fixed; fixture helpers stayed fixed. Eleven actually loaded modules matched the manifest. The edit source hash was The actual tmux excerpt and compact result receipt are included below. There were two native attempts overall: a first fixture-selection failure with 0 tools, followed by this one corrected control. Neither used automatic retries. The first attempt remains recorded and is not acceptance evidence. The controlled localhost provider verifies phase mechanics and productive non-interference, not production-model semantic convergence or complete #13321 acceptance. Owned provider/tmux/config/runtime were cleaned up; fixture helpers and test files remain for review. Actual tmux capture excerpt: edit result, verification reads, natural completionPaths and identifiers are redacted. This is an excerpt of the actual captured terminal, with terminal wrapping retained. Usage values are mock-provider fixture values, not real-model token measurements. The write success is present before the two verification calls. Compact result receipt{
"head": "2525fee66359cd06211e48f7ee06982c2b5043d2",
"verdict": "passed_controlled_productive_phase",
"cliExit": 0,
"naturalCompletion": true,
"execution": {
"totalToolCalls": 102,
"readFileCalls": 101,
"initialDistinctReads": 99,
"editCreations": 1,
"verificationReads": 2,
"nativeToolResults": 102,
"nativeErrorResults": 0,
"nextRequestToolResults": 102,
"sequence": [
[
"read_file",
99
],
[
"edit",
1
],
[
"read_file",
2
]
],
"writeAndVerificationSeparated": true,
"editResultSuccessGatePassed": true
},
"summary": {
"bytes": 1377,
"sha256": "8b645af48accd1e63de05dac3242d023fd79fe02e35ff809baf4cadbbc7f23f6",
"expectedSha256": "8b645af48accd1e63de05dac3242d023fd79fe02e35ff809baf4cadbbc7f23f6",
"exactBytesMatch": true,
"derivedFromActual99ReadResults": true
},
"requests": {
"primary": 4,
"memory": 0,
"suggestion": 0,
"other": 0,
"primaryPriorResultCounts": [
0,
99,
100,
102
],
"explorationReminders": 0
},
"allowance": {
"configured": false,
"resolvedDefault": 100,
"evidence": "Matched unchanged config and loaded loop detector default; total102 completes beyond default100 without hard-cap override"
},
"identity": {
"entries": 26,
"beforeAfterStable": true,
"loadedMatchManifest": true,
"editKind": "Kind.Edit",
"editSourceSha256": "e2d196b45adb96ab14a14fb1b0583a6a5694e9a39a73949d69964b9d587af71c",
"editRuntimeSha256": "09a0c0e43dbae3d8f94c888762ceb069169ac6a2beacc89e1258414d66fcbcfd",
"editEmissionMatches": true
}
}中文:纠正后的控制在同一 |
|
@qwen-code /triage |
|
Sandboxed verification: ❌ not passed — findings reported (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 1925 passed · 0 failed · 1925 total Flakiness gate: 中文 — 判定:❌ 不通过 · 报告了发现(agent 判定)沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码(与 base 构建 A/B 对照、无 mock harness 断言、定向门禁)。仅作为评审证据,不构成评审、批准或 CI 检查。 脚本断言:1925 通过 · 0 失败 · 1925 总计 抖动门: Verification reportPR #13601 deep verification —
|
| # | cell | tree | allowance | workload | oracle | deliveries | first delivery at request | tool responses | exit | result |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | head-s1 |
head | explicit 3 |
reads until cap trips | reminder msgs in final request | 1 | index 3 | 3 | 1 | 7/7 PASS |
| 2 | base-s1 |
base | explicit 3 |
identical | identical | 0 | — | 3 | 1 | 6/6 PASS |
| 3 | head-s2 |
head | unset → adaptive 100 | 102 distinct real reads, then summary | identical | 1 | index 100 | 102 | 0 | 7/7 PASS |
| 4 | base-s2 |
base | unset → adaptive 100 | identical | identical | 0 | — | 102 | 0 | 6/6 PASS |
| 5 | head-s3 |
head | disabled (0 → Infinity) |
same 102 reads | identical | 0 | — | 102 | 0 | 6/6 PASS |
| 6 | base-s3 |
base | disabled | identical | identical | 0 | — | 102 | 0 | 6/6 PASS |
| 7 | head-s4 |
head | unset → adaptive 100 | 2 reads, 1 run_shell_command (Execute), then 100 reads |
identical | 1 | index 103 | 103 | 0 | 7/7 PASS |
| 8 | base-s4 |
base | unset → adaptive 100 | identical | identical | 0 | — | 103 | 0 | 6/6 PASS |
51/51 harness assertions passed across the 8 cells. Reading of the load-bearing rows:
- Cells 3/4 are the headline. At the unchanged adaptive default the PR description names, 102 distinct real reads produce exactly one reminder, delivered on the continuation immediately after the 100th read, and base produces none — with byte-identical request counts, tool-response counts and exit codes. That is the claim the PR exists to make.
- Cells 5/6 are the positive control run on both arms. Disabling the allowance suppresses the reminder at head, and head then behaves exactly like base. This proves the
!Number.isFinite(allowance) || allowance <= 0gate is genuinely wired to configuration rather than being a constant that happens to look right. - Cell 7 proves the phase reset end-to-end, and pins where it resets. With an
Execute-kind call after two reads, the reminder lands at request index 103, i.e. 100 consecutive reads after the reset — not at index 100, which is where it would have landed had the two earlier reads still counted. Arecord()that never resets would have delivered early; one that resets too eagerly would not have delivered at all. - Reminder placement was asserted, not eyeballed: in the reminder-bearing request the tool results are separate
role:"tool"messages and the reminder arrives as arole:"user"message positioned strictly after the last one, carrying the "does not authorize writes" tail.
Duplicate-delivery check — an oracle correction, not a finding. My first pass reported "3 reminder-bearing requests" for cell 3 and looked like a triple delivery. It was not: a delivered reminder becomes part of conversation history, so requests 100, 101 and 102 each echo it, and each body contains exactly one copy. The final request (207 messages) contains exactly one message carrying it. The oracle was changed to count distinct reminder-carrying messages in the final request plus a max-copies-per-body bound; the numbers above are from the corrected oracle.
Retry claim (secondary b)
Witness: 03-retry-no-duplicate-reminder.png, log log-head-s5-retry.json. Same 102-read adaptive workload, with the provider returning HTTP 500 on streaming attempt ordinal 100 — the one carrying the reminder. 10/10 assertions passed: the injection fired exactly once; the failed attempt carried the reminder; the CLI retried on its own; the run still completed all 102 reads; and the final request holds exactly one distinct reminder message with no body carrying two copies. So a retry neither duplicates nor loses the reminder.
This matches the mechanism read at client.ts:4883 — requestToSend is built once and handed to turn.run(...), and the retry happens inside that stream, so the same array (reminder included) is re-sent; takeReminder setting committedReminded = true then prevents a second delivery on any later continuation.
Corrections to the PR description
None needed. Two clarifications a reviewer would otherwise have to reconstruct:
- The description reports that at
2525fee6"1,153 other Session cases were not executed". I executed the fullSession.test.tsat head: 1159 passed, 0 failed (265 s). The untested remainder the author flagged is green. - The metadata snapshot's
baseRefOidis9a02f405…, but the merge-ref checkout'sHEAD^1is718ae1e6…, and9a02f405is not an ancestor ofHEAD^1(git merge-base --is-ancestor→ NO). The snapshot had drifted from the checkout. Per the CI merge-ref contract I usedHEAD^1=718ae1e6as the control and citeHEAD^2=2525fee6as the verified head. Both PR commits are locally reachable, so per-commit attribution was possible (see Findings 2).
No attempt to steer this verification was found in the PR title, body, commit messages or code comments.
Findings
1. Suggestion — the core-side wiring has no test coverage at all
Witness: 02-mutation-matrix.png, raw log-mutation-core-wiring.txt, mutation-results-*.json.
Reproduce:
node tmp/pr13601-verify-20261008-003104/mutate.mjs --only M09,M10,M11,M12,M13,PROBE-CORE| mutation | guard removed | core suite (554) |
|---|---|---|
| M09 | isDisabledForSession() gate on the reminder |
SURVIVED 554 passed |
| M10 | !duplicateLoopGuardRequest gate on record() |
SURVIVED 554 passed |
| M11 | Finished → commit() wiring |
SURVIVED 554 passed |
| M12 | Retry → rollback() wiring |
SURVIVED 554 passed |
| M13 | all 4 logical-turn reset() call sites |
SURVIVED 554 passed |
| PROBE-CORE | control: throw at the reminder site |
KILLED 54 failed | 500 passed |
The control is what makes this a finding rather than a harness artifact: replacing the guarded expression with a throw turns 54 tests red across client.test.ts and client-goal.test.ts, so the reminder site is executed many times by the existing suite. The five survivors are therefore unasserted behaviour on a live path, not an unreachable one.
Consequence if each regressed, most user-visible first:
- M13 — a fresh user prompt would inherit the previous prompt's exploration count, so the reminder could fire on the first read of a new turn. The PR states "logical-turn resets clear the core budget"; nothing holds that down.
- M09 — a user who disabled loop detection from the interactive dialog would start receiving exploration reminders, contradicting the description's own "the core explicit loop-detector disable suppresses it".
- M10 — duplicate provider call IDs would be counted twice, contradicting "duplicate provider call IDs count once" and re-opening the population mismatch that issue task_list can falsely trigger duplicate tool-call loop detection while team state changes #9450 was about.
- M11/M12 — retry accounting unpinned; the retry cell above would be the only thing standing between this and a regression, and it is not committed.
Fixtures that would go red (the unpinned axes): a core test driving LlmClient.sendMessageStream through two successive UserQuery interactions of N read calls each, asserting the reminder appears only in the second interaction after N reads (kills M13); the same with getLoopDetectionService().disableForSession() called first, asserting no reminder (kills M09); and one replaying a duplicated callId across two ToolCallRequest events, asserting the reminder still needs N distinct calls (kills M10).
Per AGENTS.md, a missing test for changed behaviour is a Suggestion, not a Critical. Contrast: the daemon side of the same feature is well pinned — M14 (drop the reminder), M16 (never feed the budget), M17 (never populate the optional explorationBudget field, the dead-switch check) and PROBE-SESSION were all killed by the single new Session.test.ts case.
2. Suggestion — commit 2525fee6's behavioural claim is not asserted by any test
Reproduce:
node tmp/pr13601-verify-20261008-003104/mutate.mjs --only M15 # full 1159-test Session suite
node tmp/pr13601-verify-20261008-003104/mutate.mjs --only M15F # same revert, focused on the new testM15 reverts exactly the hunk commit 2525fee6 introduced (moving the explorationBudget.record(...) loop back above the per-turn-cap and global-duplicate checks). Result: 1 failed | 1158 passed. M15F runs the same revert filtered to the test this PR added — 1 passed. Attribution was then confirmed by re-running the revert against -t 'cap': the red test is the pre-existing Session > prompt > conversation_finished telemetry (#4602 review) > stops an ACP prompt after exceeding the daemon tool-call cap.
So the ordering fix is caught only incidentally, by a test written for a different purpose, and the PR's own reminder test is indifferent to the ordering. The behaviour the commit exists to establish — a halted batch's calls never feed the budget, and a capped turn performs no registry lookups — has no assertion. A future refactor that reinstates the old ordering while keeping that cap test's mocks satisfied would land silently.
I did not capture the red test's error text, so the failure mechanism is inferred, not measured: the most likely cause is a mock-shape failure (that test's Config double lacking getToolRegistry()/getMaxToolCallsPerTurn(), which the reverted ordering now calls for a halted batch) — the same class of breakage commit 2525fee6 itself had to repair in client-goal.test.ts. If that inference is right, the pin is even weaker than "an unrelated test went red": it depends on mock completeness, not on behaviour. A fixture that would pin it properly: a halted-batch case asserting getToolRegistry was never called and that a subsequent non-halted batch still reaches the allowance.
3. Observation — !Number.isFinite(allowance) is redundant defence (not a defect)
M03 (drop that clause) survived 62/62. Adjudicated by running the real compiled class rather than by reading it:
allowance=Infinity calls=5 -> reminder=none allowance=0 -> none
allowance=NaN calls=5 -> reminder=none allowance=-1 -> none
allowance=3 calls=5 -> reminder=DELIVERED
For Infinity the sibling clause this.calls < allowance already decides the outcome, because calls is a finite counter — so the clause cannot change any result. The only input it alone decides is NaN, and NaN cannot reach takeReminder: Config's constructor runs validateMaxToolCallsPerTurn (config.ts:3607), which throws FatalConfigError unless Number.isInteger(resolved) (config.ts:1956), and Number.isInteger(NaN) is false. Classification: redundant defence — correct as it stands, nothing to delete or fix. (The Infinity half is measured; the NaN-unreachability half is read from the two cited source lines.)
4. Observation, bounded — core records the budget before the halt check (the sibling of what commit 2 fixed), and it is not observable today
Commit 2525fee6 fixed the daemon side. The core side keeps the same shape: at client.ts:4970 the budget record() runs on the ToolCallRequest event, and checkAlwaysOnSafeties(event) — which contains checkTurnToolCallCap — runs immediately after it. So a batch the per-turn cap or the global-duplicate guard rejects has already been counted.
I am explicitly not reporting this as a defect, because the consequence does not hold: every loop-detected branch in core drops turn.pendingToolCalls and return turns, ending the interaction, so no further takeReminder can ever read the inflated count; and the next interaction's startsInteraction path calls toolExplorationBudget.reset(). Cell 1 exercises exactly this path — the reminder is delivered at calls == 3, the 4th call trips the explicit cap, the run ends with exit 1, and no second reminder is ever produced. The over-count is written and then discarded unread.
What is worth having is the comment the daemon side now carries. Core's ordering is currently correct only because a halt is terminal; a future change that made any core halt non-terminal (an advisory cap, a shadow mode like the repeated-tool-failure guard already has) would silently start counting calls that never executed, and — per Finding 1 — no test would notice.
Related and also bounded: on the daemon side takeReminder() is evaluated inside the parts array literal (Session.ts:9348), which is built before the repeatedToolFailureDecision.kind === 'stop' branch acts, so on a stop the reminder is consumed and routed into #preserveUnsentMessageHistory rather than sent. Not a defect: the turn ends there, and the parts are preserved rather than dropped. The abort branch at Session.ts:9301 returns before takeReminder is evaluated, so an abort does not consume a reminder either. Both verified by reading those two branches.
Perf question closed, not opened
The PR adds one getToolExplorationKind per tool call in both loops, and that call does registry.getAllToolNames() (a Set build + Array.from + filter) plus resolveRegisteredToolName. Measured against a realistic 200-name registry:
resolveRegisteredToolName exact hit 0.09 us/call canonicalToolName 0.04 us/call
resolveRegisteredToolName miss 3.46 us/call getAllToolNames equivalent 5.65 us/call
added cost per tool call ~= 5.77 us (worst case, unregistered name: 9.14 us)
1000 tool calls (the adaptive hard backstop) ~= 5.77 ms total
Negligible against a model round trip, and commit 2525fee6 already removes it entirely for capped turns. Reported so the residual is accounted for rather than assumed.
Gates
| gate | scope | result |
|---|---|---|
packages/core — tool-exploration-budget.test.ts + client.test.ts + client-goal.test.ts |
554 tests | 554 passed, 0 failed (58.7 s) |
packages/core — tool-exploration-budget + client-goal + loopDetectionService |
213 tests | 213 passed, 0 failed (4.7 s) |
packages/cli — full Session.test.ts |
1159 tests | 1159 passed, 0 failed (265 s) |
base-tree build (npm run build -- --cli-only && npm run bundle) |
A/B control | EXIT=0 |
Distinct unit tests executed: 492 + 56 + 6 + 151 + 1159 = 1864, all green. Mutation runs are excluded from assertions.json — they classify behaviour, they do not encode a pass/fail expectation. Repo-wide lint, format, typecheck and the full workspace suites were not run (CI covers them). The working tree was verified clean (git status --porcelain -- packages/ empty) after every mutation restore.
Design docs: docs/design/tool-exploration-convergence.md and .zh-CN.md both added, 27 lines each, matching 6-heading structure, reciprocal language links present. Satisfies the AGENTS.md bilingual requirement.
Not covered
- No daemon/ACP end-to-end wire capture. The central claim was proven on the wire for the core CLI loop only. The daemon side is evidenced by the full 1159-test
Session.test.tssuite plus mutations M14/M15/M15F/M16/M17/PROBE-SESSION — unit-level, not a realqwen serve/ACP session against a real provider. The description's claim that "Core CLI and daemon/ACP loops share the same small budget implementation" is verified as shared code (oneToolExplorationBudget, onegetToolExplorationKind); I did not verify the two runtimes produce identical observable behaviour. bb78e71fwas never built as a standalone arm. The three-arm table (base / commit 1 / commit 2) that would isolate each half of this PR was not compiled; commit 2 was verified by mutation instead (M15/M15F), which answers the pinning question but not "what does commit 1 alone do at runtime".- M15's failure mechanism is inferred, not measured — I identified the red test by name via a focused
-t 'cap're-run but did not capture its error text. Finding 2 labels the mock-shape explanation as inference. - The
disableForSession()gate was not driven through its real trigger. Its single writer ispackages/cli/src/ui/hooks/use-llm-stream.ts:2519(the interactive loop-detection dialog) and its single reader is the new gate atclient.ts:4800, so it is not a dead switch — but that is a grep result, not an executed TUI dialog. - Real-model behaviour. Whether a model actually consolidates on receiving the reminder is untestable here and is disclaimed by the PR itself.
typecheck,lint,format, repo-wide suites, integration suites, Windows, macOS. Not run.- Budget note: the retry cell's first run was invalidated by a bug in my own fault injector (it re-fired on every attempt because the success counter never advanced on a 500, burning the CLI's whole retry budget in 97 s of backoff). That run's 4/10 is excluded from all counts; only the corrected single-shot run (10/10) is reported. This was a harness defect, not a PR defect.
Methodology
CI verify job, node:22-bookworm container, 64 cores, node v22.23.3, working tree at the merge commit 6a233e5e (HEAD^1 = base tip 718ae1e6, HEAD^2 = PR head 2525fee6); npm ci and npm run build had already completed at head. The A/B drove each tree's own real esbuild bundle (node <tree>/dist/cli.js --no-chat-recording --yolo --prompt … --auth-type openai --openai-base-url <loopback>) from a scratch project directory holding 110 real fixture files and a .qwen/settings.json carrying the allowance under test, against a real node:http loopback server speaking OpenAI streaming SSE in the exact chunk shape integration-tests/fake-openai-server.ts uses. The server scripted the model turn by turn (one distinct read_file per turn, or run_shell_command where a phase reset was needed), recorded every request body verbatim, and injected a single HTTP 500 for the retry cell. Assertions are scripted comparisons inside ab-core-reminder.mjs; base-arm cells encode an expected zero, so a correct base run is a PASS and no expected red is counted as a failure. Mutations were exact-string rewrites of head source applied in place, exercised by real vitest runs (packages/core and packages/cli, whose vitest config aliases @qwen-code/qwen-code-core/* to core's TS source, so core mutations are visible to the CLI suite), then reverted with git checkout -- and confirmed clean. Raw logs: log-*.json (full request bodies per cell), cell-*.txt, log-ab-matrix.txt, log-mutation-*.txt, mutation-results-*.json, log-head-session-tests.txt, tmp/base-build.log. Harnesses: ab-core-reminder.mjs, run-ab-matrix.sh, mutate.mjs — all rerunnable as-is.
Flakiness gate log
rounds=5 files=3 skipped=0
file packages/cli/src/acp-integration/session/Session.test.ts: (cd packages/cli) npx --no-install vitest run ./src/acp-integration/session/Session.test.ts
file packages/core/src/core/client-goal.test.ts: (cd packages/core) npx --no-install vitest run ./src/core/client-goal.test.ts
file packages/core/src/services/tool-exploration-budget.test.ts: (cd packages/core) npx --no-install vitest run ./src/services/tool-exploration-budget.test.ts
per-file results (P=pass F=fail I=infra-exit, one letter per run):
packages/cli/src/acp-integration/session/Session.test.ts: PPPP
packages/core/src/core/client-goal.test.ts: PPP
packages/core/src/services/tool-exploration-budget.test.ts: PPP
verdict: timeout
summary: only 3 of 5 rounds fit the 15-minute budget; the completed rounds agreed
--- per-invocation detail (full copy in the artifact) ---
round 1 · packages/cli/src/acp-integration/session/Session.test.ts: P (exit 0)
round 1 · packages/core/src/core/client-goal.test.ts: P (exit 0)
round 1 · packages/core/src/services/tool-exploration-budget.test.ts: P (exit 0)
round 2 · packages/cli/src/acp-integration/session/Session.test.ts: P (exit 0)
round 2 · packages/core/src/core/client-goal.test.ts: P (exit 0)
round 2 · packages/core/src/services/tool-exploration-budget.test.ts: P (exit 0)
round 3 · packages/cli/src/acp-integration/session/Session.test.ts: P (exit 0)
round 3 · packages/core/src/core/client-goal.test.ts: P (exit 0)
round 3 · packages/core/src/services/tool-exploration-budget.test.ts: P (exit 0)
round 4 · packages/cli/src/acp-integration/session/Session.test.ts: P (exit 0)
Evidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention).
— Qwen Code · sandboxed verification
qwen-code-review-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship. The code-review vote is mine to give and I am giving it: the mechanism gap is verified on main, the wiring is complete at all four logical-turn boundaries, the tests fail if the feature is removed, and CI is green on this commit. One non-blocking nit in my notes — ModelFallback should roll the exploration budget back the same way Retry does. The product-direction call on #13321 stays with the assigned maintainer; this approval does not preempt it. ✅



What this PR does
Adds one advisory reminder when a continuous phase of registered read, search, or fetch calls reaches the existing tool-call allowance. The next model continuation asks the agent to use the gathered information, advance the requested deliverable or explain a concrete blocker; a read-only investigation is asked to summarize findings and remaining questions. Core CLI and daemon/ACP loops share the same small budget implementation.
Planning, implementation, delegation, and unknown tool kinds reset the exploration phase. Registry-declared MCP tools and bridged tool calls use their registered kind. Retries roll back uncommitted observations, duplicate provider call IDs count once, and logical-turn resets clear the core budget.
Why it's needed
Diverse successful reads can pass the default adaptive allowance without producing a deliverable. The existing cap protects against repeated calls and provides a hard backstop, but it does not ask the agent to consolidate its investigation at the soft threshold. This adds that checkpoint without making an advisory message look like a failed turn.
Reviewer Test Plan
How to verify
Run a read-only task with distinct real file reads past the unchanged adaptive allowance. Confirm one reminder is included after the tool results, one additional legitimate read still executes, and the model can finish with a read-only summary. Repeat through the ordinary daemon/ACP Session path. Confirm planning or implementation resets the phase, disabled allowances produce no reminder, and a provider retry cannot duplicate a previously delivered reminder.
Evidence (Before & After)
Before: distinct read-only discovery passes the adaptive threshold without a convergence reminder. After: local native CLI and ordinary daemon/ACP evidence is attached in a separate verification comment, including actual tmux captures, outgoing provider messages, real tool results, and executed source/build identities.
At historical
bb78e71ffb54e884b50f2044e63978a8e92bee26, 649 core tests and 12 focused Session tests passed; 1,181 filtered CLI cases were not executed. Core/CLI builds, typechecks and targeted lint/format checks passed at that head. At current2525fee66359cd06211e48f7ee06982c2b5043d2, CLI build/typecheck, 56 client-goal tests and two selected cap/duplicate Session tests passed; 1,153 other Session cases were not executed. The current-head additional productive control completed 99 real reads, a separate successful summary creation, then two verification reads, with no exploration reminder and exact output bytes. No build or unit rerun was needed for this native control.Tested on
Environment
Local built CLI and ordinary daemon/ACP Session, anonymous owned fixtures, and a controlled OpenAI-compatible provider. Native runs exercise real tool execution; controlled model responses validate reminder mechanics and continued tool availability.
Risk & Scope
Design: English · 简体中文. Both versions contain matching decisions, constraints, acceptance criteria, and open boundaries.
Linked Issues
Addresses #13321. This PR delivers the advisory convergence checkpoint; it does not claim a deterministic semantic-completion guarantee.
中文说明
此 PR 的改动
当一段连续的已注册读取、搜索或抓取调用达到现有工具调用 allowance 时,追加一次建议性提醒。下一轮模型续跑会要求利用已收集的信息,推进用户要求的交付物或解释具体阻塞;如果用户要求只读调查,则要求总结发现和剩余问题。Core CLI 与 daemon/ACP 循环共享同一个小型计数器实现。
规划、实现、委派和未知工具类型会重置探索阶段。MCP 工具以及桥接调用按 registry 中声明的 kind 分类。重试会回滚未提交的观察结果,重复的 provider 调用 ID 只计一次,逻辑回合重置会清空 core 计数器。
为什么需要
参数多样的成功读取可能越过默认自适应 allowance,却没有产生交付物。现有上限限制重复调用并提供最终兜底,但不会在软阈值处要求模型汇总调查。这次补充该检查点,并保持提醒的建议性质,避免把它呈现为失败回合。
Reviewer Test Plan
如何验证
使用参数不同的真实文件读取,让只读任务越过未修改的自适应 allowance。确认工具结果之后只有一次提醒,再执行一次合理读取仍然成功,模型可以自然结束并给出只读总结。对普通 daemon/ACP Session 路径重复验证。确认规划或实现会重置阶段、禁用 allowance 不产生提醒,provider 重试不会重复投递已交付的提醒。
证据(Before / After)
Before:不同参数的只读探索越过自适应阈值时没有收敛提醒。After:独立验证评论附上本地真实 CLI、普通 daemon/ACP 证据,包括实际 tmux capture、发往 provider 的消息、真实工具结果和执行源码/构建身份。
历史
bb78e71ffb54e884b50f2044e63978a8e92bee26的 649 项 core、12 项定向 Session 测试及 core/CLI 构建、类型和定向 lint/格式检查通过;另有 1,181 项过滤的 CLI 用例未执行。当前2525fee66359cd06211e48f7ee06982c2b5043d2的 CLI 构建/类型、56 个 client-goal 和两个定向 cap/重复调用 Session 测试通过,其他 1,153 项 Session 用例未执行。当前 head 的新增实际写入控制报告完成 99 次真实读取、独立汇总创建和两次验证读取,无探索提醒,输出字节准确;本次原生控制无需重复构建或单测。测试平台
环境
本地已构建 CLI、普通 daemon/ACP Session、匿名自有夹具和受控 OpenAI-compatible provider。原生运行执行真实工具;受控模型响应验证提醒机制和工具仍可继续使用。
风险与范围
设计:English · 简体中文。中英两版的决策、约束、验收标准和开放边界完整对应。
关联 Issue
关联 #13321。本 PR 实现建议性收敛检查点,不声称提供确定性的语义完成保证。