Repository navigation
fix(cli): prefer per-session cached token count in /context - #12066
Conversation
In a serve daemon, /context already reads the prompt total from the active chat but still subtracted the process-global cached-content count, so one session's cache hit could collapse another session's messages category. Mirror cached tokens onto GeminiChat and prefer that value, matching the QwenLM#5763 total-token fix. Fixes QwenLM#12047
|
Local merge-reference verification: PASS
I ran the same temporary integration test on the merge base and PR head. Each run started the exact-revision bundle, created two distinct
The failing operand is identifiable from the numbers, rather than only from the red test: This reproduces the issue through the full provider usage → per-session Regression pin: identical harness SHA-1 Narrow head gates also passed:
Before — the result matches B's cached value, not A's: After — the result matches A's own cached value: The local fake provider makes the usage values deterministic; this did not exercise an external provider's cache implementation. It did exercise the real bundled daemon, two real ACP sessions, the production write side, and the production context-usage read side. |
chiga0
left a comment
There was a problem hiding this comment.
Review — approved
核对基线:head dc5a7bc6(base 9dd689f048,3 个文件,+68/-11)。
Scope
llm-chat.ts:新增lastCachedContentTokenCount字段,始终镜像 usage metadata(含零值),新增getLastCachedContentTokenCount()方法,retain/restore 路径同步contextCommand.ts:优先读activeChat.getLastCachedContentTokenCount(),仅在无 chat 时 fallback 到全局 singletoncontextCommand.test.ts:回归测试——foreign global cache 64,653 不能把 65,267 token 会话的 messages 压到 ~614
结论:无阻塞问题
- 与 #5763 的修复模式一致:
lastPromptTokenCount已做了相同的 per-chat 镜像,本次扩展到cachedContentTokenCount - 零值镜像正确:旧代码
if (cachedContentTokenCount && ...)跳过零值,导致会话 B 保留会话 A 的缓存计数。新代码无条件赋值,零值也是有效数据 - telemetry 仍同步更新:
telemetryService?.setLastCachedContentTokenCount(...)保持全局镜像(其他消费者仍可用),但/context不再依赖它 - 可选链
?.保护测试 mock:activeChat?.getLastCachedContentTokenCount?.()兼容不提供此方法的 mock 对象,生产环境 LlmChat 始终有此方法 - retain/restore 路径覆盖:
adoptTokenCountsForRoute恢复retained.cachedContentTokenCount,路由切换不会丢 per-session 计数
yiliang114
left a comment
There was a problem hiding this comment.
Code LGTM — verified the per-session cached-count fix end to end:
lastCachedContentTokenCountis now mirrored on the chat on every usage report (including zero, deliberately), so a later /context in this session cannot inherit a foreign session's cache hit.- Route retain/restore/reset all carry the per-chat value; the previous snapshot code read the process-global singleton, which was itself a cross-session leak — good catch fixing that at the same time.
contextCommandfalls back to the global only when no chat exists, with optional chaining for older chat objects.- The test's behavioral assertion (foreign 64_653 must not collapse messages toward ~614) proves the fix at the observable level rather than mocking internals.
Note: the open CHANGES_REQUESTED from qwen-code-ci-bot is the PR-body template gate (stage=1a), not a code issue — it stays until the body is restructured per the template (## What this PR does / ## Why / ## Reviewer Test Plan / ## Risk & Scope / ## Linked Issues + Chinese details). Once the body matches, the bot will run its full gate. My approval covers the code only.
qqqys
left a comment
There was a problem hiding this comment.
Critical-only review at dc5a7bc6, base 9dd689f0.
Verdict: APPROVE — no merge-blocking correctness, security, data-loss, regression, or compatibility defect found.
Historical blocking issues
The outstanding CHANGES_REQUESTED on this PR is a PR-template/description request, not a code defect, and the body now carries the template's sections. No code-level blocking issue has ever been raised against this head: the follow-up review at this same commit records "no blockers" with all four of its findings at Suggestion severity, and there are no review threads claiming a correctness, security, data-loss, regression, or compatibility defect. The four open threads are all Suggestions, which are out of scope for this pass and do not gate approval.
What I verified against the head code
The value written into the new slot can never be undefined or non-finite. cachedContentTokenCount at the usage site comes from coerceUsageCount, which returns a number on every path — it accepts only a finite value >= 0 and coerces anything else to 0 with a warning. So the change from a truthiness-guarded telemetry write to an unconditional this.lastCachedContentTokenCount = cachedContentTokenCount cannot introduce undefined or NaN into a number-typed field, and the retained record and usage record both declare the field as number.
The unconditional mirror is the actual fix, and it is safe. Writing zero as well as nonzero is what stops a later /context in this session from subtracting another session's cache hit. On the consumer side apiCachedTokens is only used under if (apiCachedTokens > 0), so a legitimate zero now falls through to the scaledOverhead branch instead of applying a foreign cache figure.
No negative or inverted tier can result. The single subtraction is messagesTokens = Math.max(0, totalTokens - apiCachedTokens), clamped at zero. Even in the stale-count case the open Suggestion thread describes — where a non-API writer rewrites the prompt count while the cached count still describes an earlier prompt — the worst outcome is a messages figure floored at 0, not a negative or NaN value. The headline total and tier derive from apiTotalTokens, which this PR does not change. That same skew already existed at base, where the global cached count was mixed with a per-chat total; sourcing both from the same chat is what restores the cached <= prompt invariant the API guarantees.
The new getter adopts the route correctly when called with no argument. collectContextData calls activeChat?.getLastCachedContentTokenCount?.(), and adoptTokenCountsForRoute resolves an absent key via targetRouteKey ??= this.currentRouteKey() after its zero-count fast path. The call therefore behaves like the sibling prompt-count read directly above it rather than defaulting to a wrong route.
Slot lifecycle is complete across all three sites. The retain path records cachedContentTokenCount: this.lastCachedContentTokenCount, the restore path assigns it back from the retained record, and the route-reset path zeroes it alongside lastPromptTokenCount and lastOutputTokenCount. There is no path that restores or resets the prompt count while leaving the new slot pointing at a foreign route's value.
The optional-call fallback preserves the previous behaviour when no chat exists. activeChat?.getLastCachedContentTokenCount?.() ?? uiTelemetryService.getLastCachedContentTokenCount() falls back to the process-global singleton only when there is no active chat, which is the pre-existing read, and ?? rather than || means a legitimate per-session 0 is not discarded in favour of the global value.
CI on this head is green across all 205 checks with no failures, including the ubuntu test lane, lint and static checks, and the no-AK integration lane.
|
Thanks @chiga0 and @yiliang114 for the approvals and the end-to-end verification. Merge is still BLOCKED on an earlier I cannot dismiss that stale bot review from a fork. If a maintainer can dismiss it (or merge over it), this should be ready — happy to push anything else needed. |
|
Thanks @kabishou11 — this is the right fix, and it unblocks item 3 of #12033 (the I read the diff against current
Two small asks before the gate runs:
中文说明感谢 @kabishou11,修复方向正确,而且它是 #12033 第 3 条的前置条件( 我对着当前
进入门禁前的两点小建议:
|
|
Thanks @yiliang114 — really appreciate the careful read against I'll add the Thanks again for the LGTM and the clear asks. |
|
Thanks @qqqys as well for the Critical-only pass and for calling out that the outstanding bot |
Add regression where chat cached=0 and global holds 64_653 so messages use total − scaledOverhead (??), not the foreign cache (|| leak).
d49dcae
|
@yiliang114 Added the chat=
Manual two-session Web Shell Evidence still pending on this box (no serve UI run here); Evidence section notes that. Happy to paste B's before/after |
yiliang114
left a comment
There was a problem hiding this comment.
Reviewed the current head d49dcae2b3 (base 9dd689f048). The second commit only adds the zero-cache regression case; the two production files are byte-identical to dc5a7bc6. The new test is useful and correctly pins ?? rather than ||.
I found no additional diff-level defect beyond the four existing open threads, but I would not treat all of them as cosmetic:
- R1-3 is reproducible on this exact head with a real
LlmChat. After a streamed usage report writesprompt=65,267 / cached=64,653,setLastPromptTokenCount(10,000)leaves cached at64,653, so the/contextformula yieldsmessages = 0. Repeating with/compress-fastproducedtotal=62,767 / cached=64,653, again yieldingmessages = 0. This existed in the single-session baseline, but the new per-chat source makes the stale value stable instead of allowing another daemon session to overwrite it accidentally. The new cached slot should be invalidated alongside non-API prompt-count writes, and the hard-rescue snapshot/rollback should carry it deliberately. - R1-1 therefore remains important: the current CLI tests mock the getter and do not exercise the new core slot, zero reset, or route retain/restore lifecycle.
- R1-2 is still accurate; the retain-path comments describe the removed telemetry read.
- R1-5 is still unaddressed. More fundamentally, a published deferred/provisional serve session can have no initialized chat, and that path still reads the process-global prompt and cached counts. That is pre-existing behavior, not a regression from this PR, but it means the session-ownership fix is incomplete during bootstrap.
All completed CI checks on this head are green. The remaining CHANGES_REQUESTED review is the earlier PR-template gate; the body now follows the template, so that review state appears operationally stale rather than a current code objection.
The PR body now follows the repository template, includes the reviewer test plan, evidence, tested-on table, risk/scope, linked issue, and Chinese translation. This request is stale after the body update.
|
Thanks @yiliang114 for landing I'll treat bot Suggestions that still cite |
doudouOUC
left a comment
There was a problem hiding this comment.
LGTM. Per-chat lastCachedContentTokenCount plus ?? (not ||) on the /context read stops a foreign session's cache hit from collapsing this session's messages. The 0 vs global-64653 cases are load-bearing.
qwen-code-dev-bot
left a comment
There was a problem hiding this comment.
APPROVE
核对基线:head 32805a7954bfbbabc4d986a344f6a8553e747ff5(base 9dd689f048,3 commits,4 个文件 +214/-56)。required 档全部完成且成功:Test (ubuntu-latest, Node 22.x)、Lint & Static、Integration Tests (no-AK, No Sandbox)、web-shell E2E Smoke、TUI parity snapshots、OpenTUI no-flicker gate。
上一轮那份「核心侧失效纪律不完整 + 核心侧无测试」的意见,在这两个 commit 里已经落地,我是按 head 的代码逐条复验的,不是看 thread 的 resolved 标记:
setLastPromptTokenCount(llm-chat.ts:2595-2600)与 compression 成功路径(:2886-2894)现在成对清零lastCachedContentTokenCount并同步 telemetry 镜像;hard-rescue 回滚把 cached 也纳入了快照与恢复(:3139-3145、:3243-3266)。读侧contextCommand.ts:313-314是Math.max(0, total - cached),这正是我之前担心的「压缩后 messages 塌成 0」的那条路径。seedResumeTokenCounts(:2624-2644)没清 cached 是对的:它只在startChat之后、刚建好的 chat 上调用(client.ts:595-604、:625-632),槽按构造就是 0,不存在外来值。retainCurrentTokenCounts上方那条描述已删表达式的过时注释也去掉了。- 核心侧补了
llm-chat.test.ts:通过sendMessageStream喂真实usageMetadata再断言chat.getLastCachedContentTokenCount()(存 42 → 换 route 归 0 → 换回仍 42 → 缺字段清 0),另有一条专测非 API 写入点的清零。这是真正跑到LlmChat的用例,不是 cli 侧vi.fn()桩。
我自己另核过三点,都成立:写入值恒为数字(llm-chat.ts:6007 的 coerceUsageCount),所以读侧 ?? 全局 不会被 undefined 悄悄打开;GeminiChat 就是 LlmChat 的导出别名(llm-chat.ts:6760),新 getter 不是空挂的可选调用;total 与 cached 两条读法的 route 参数一致,配对的仍是同一条 route。
记录(不阻塞):R1-5 与 R2-1 两条在 head 上仍开着——我确认 contextCommand.test.ts 里没有 ci-bot 建议的 expect(data.breakdown.messages).toBeLessThan(data.totalTokens),也没有「无 chat 且全局 cached 非 0」的用例,所以 contextCommand.ts:313 的 > 写成 >=、或回退腿被换成 ?? 0,都不会有测试变红。两处各是一行断言的事,属于变异加固而非现存缺陷(生产行为我上面已核过是对的),按本项目约定缺测试记 Suggestion。建议直接采纳 ci-bot 给的两条断言收口。
seedResumeTokenCounts always runs on a freshly started chat, so the per-chat cached slot reset added earlier is a defensive invariant with no visible effect. The value that does go stale on an in-process resume is the global telemetry mirror, which kept the previous session's cached count next to the seeded prompt count. Clear it too (optional chaining, since subagent chats have no telemetry service) and pin the clear after each seed in the resume test. Also correct the doc comment: ResumeTokenCounts carries no cached count; the transcript itself does record it. Refs QwenLM#12066


What this PR does
Makes
/context(and the sharedcollectContextDatapath) prefer a per-session cached-content token count from the active chat, instead of always subtracting the process-global cached count when deriving the messages category.On the write side,
cachedContentTokenCountis mirrored onto the chat whenever usage metadata is recorded (including zero) and kept in the per-route retain/restore path. On the read side,collectContextDatausesactiveChat.getLastCachedContentTokenCount()when a chat exists, and falls back to the global singleton only when no chat exists yet — the same shape already used for the prompt total (#5763).Why it's needed
In a multi-session
serve/ daemon process,/contextalready reads the prompt total from the active chat, but it still subtracted the process-global cached-content count. A large cache hit from another session could silently drivemessagestoward zero viaMath.max(0, total - foreignCached)(#12047). That mis-attribution also affects other multi-session surfaces that callcollectContextData(non-interactive control and ACP), not only interactive/context.Reviewer Test Plan
How to verify
vitest run packages/cli/src/ui/commands/contextCommand.test.ts— expect 21/21, including: (1) foreign global cache of 64_653 cannot collapse messages on a 65_267-token session with chat cached=1_000; (2) chat cached=0 with foreign global 64_653 usestotal − scaledOverhead(pins??vs||).qwen servesessions in one daemon; complete a turn with cache hits in A; open/context(orGET /session/:id/context-usage?detail=true) in B and confirm B's messages are not driven by A's cache. @yiliang114 verified this end-to-end on macOS with the real bundled daemon and a local fake provider: merge base showed A messages collapse to 614 after B's 64_653 cache; PR head kept A at 64_267.Evidence (Before & After)
Unit: foreign global cache no longer collapses messages when the active chat has its own cached count. Also pins chat cached=
0/ foreign global64_653somessagescomes fromtotal − scaledOverhead(??), not the foreign cache (a future||would reintroduce the leak).Manual two-session Web Shell check (A long prompt ×2 → B
/context detail): pending — will paste B's before/after/context detailhere when available. Not inventing numbers.@yiliang114's merge-reference run (exact-revision bundle, two
sessionScope: "thread"sessions):total / messages65,267 / 64,26765,267 / 64,267total / messages65,267 / 61465,267 / 64,267Before/after captures: #12066 (comment)
Tested on
Environment (optional)
Unit tests locally. Merge-reference verification used a repository-local fake OpenAI server (no external model credentials) against a real
qwen servedaemon.Risk & Scope
Linked Issues
Fixes #12047
中文说明
这个 PR 做了什么
让
/context(以及共享的collectContextData路径)优先使用活跃会话 chat 上的 per-session cached-content token 计数,而不是在推导 messages 分类时始终减去进程级全局缓存计数。写入侧:每当记录 usage metadata(含零值)时,把
cachedContentTokenCount镜像到 chat,并纳入 per-route retain/restore。读取侧:存在 chat 时用activeChat.getLastCachedContentTokenCount(),尚无 chat 时才回退全局单例——与 prompt total 已有的写法同形(#5763)。为什么需要
在多会话
serve/ 守护进程里,/context的 prompt total 已按活跃会话读取,但 cached 仍减进程级全局计数。另一会话的大缓存命中会通过Math.max(0, total - foreignCached)把 messages 静默压向 0(#12047)。该串用也会影响调用collectContextData的其他多会话表面(非交互控制通道与 ACP),不限于交互式/context。审阅者测试计划
如何验证
vitest run packages/cli/src/ui/commands/contextCommand.test.ts—— 预期 21/21,含:(1) chat cached=1_000 时外来全局 64_653 不能压垮 messages;(2) chat cached=0 / 外来全局 64_653 走total − scaledOverhead(钉住??而非||)。qwen serve会话;A 完成带缓存命中的一轮后,在 B 打开/context(或GET /session/:id/context-usage?detail=true),确认 B 的 messages 不被 A 的缓存驱动。@yiliang114 已在 macOS 用真实打包守护进程与本地 fake provider 做端到端验证:merge base 在 B 出现 64_653 缓存后把 A 的 messages 压到 614;PR head 仍保持 A 为 64_267。证据(修改前 / 修改后)
单元:外来全局缓存在活跃 chat 自有缓存计数时不再压垮 messages。另钉住 chat cached=
0/ 外来全局64_653,messages走total − scaledOverhead(??),不会吃外来缓存(若改成||会把泄漏带回来)。双会话 Web Shell 手工验证(A 长提示×2 → B
/context detail):待补——有结果后再贴 B 前后/context detail,不编造数字。@yiliang114 的 merge-reference(精确修订包,两个
sessionScope: "thread"会话):total / messages65,267 / 64,26765,267 / 64,267total / messages65,267 / 61465,267 / 64,267前后截图见:#12066 (comment)
测试平台
环境(可选)
本地单元测试。merge-reference 使用仓库内 fake OpenAI server(无外部模型凭证)对真实
qwen serve守护进程验证。风险与范围
关联 Issue
Fixes #12047