fix(core): eliminate OOM from debugResponses accumulation by zzhenyao · Pull Request #4982 · QwenLM/qwen-code

zzhenyao · 2026-06-11T03:09:09Z

What this PR does

Removes debugResponses array and extractUsageFromGeminiClient. Both are dead code.

Turn.debugResponses pushed every streaming chunk into an array. Nothing in production ever read it. extractUsageFromGeminiClient was exported but never imported by production code. Non-interactive mode uses computeUsageFromMetrics from the telemetry pipeline instead.

Closes: #4815
Follow-up to #4824

Why it is needed

Every streaming chunk was pushed into debugResponses and held in memory for the lifetime of the Turn object. In goal mode, nested turns are not cleaned up between iterations, so the array keeps growing. A single long session can accumulate gigabytes of response objects that are never read, leading to OOM.

debugResponses holds a reference to every streaming chunk for the lifetime of the Turn object. Two scenarios cause OOM:

Non-goal mode: a single turn with the model stuck in a thinking loop produces hundreds of chunks, each a full GenerateContentResponse. The array grows unbounded within one turn.
Goal mode: nested sendMessageStream calls hold inner Turn objects as references until the outer call completes. Each nested turn accumulates its own debugResponses. The chain never releases until the top-level goal finishes, so memory grows across the entire nesting depth.

Reviewer Test Plan

Before / After

Before: Turn has a debugResponses array that collects every chunk. extractUsageFromGeminiClient is exported but unused.

After: Neither exists. Zero references left.

How to verify

git grep "debugResponses"
git grep "extractUsageFromGeminiClient"

npm run build
npm run typecheck

cd packages/core && npx vitest run src/core/turn.test.ts
cd packages/cli && npx vitest run src/utils/nonInteractiveHelpers.test.ts
cd packages/cli && npx vitest run src/nonInteractiveCli.test.ts

中文

删除 debugResponses 数组和 extractUsageFromGeminiClient 函数。

每个 streaming chunk 都被 push 进 debugResponses 并在 Turn 生命周期内一直持有。goal 模式下嵌套的 turn 不会被清理，数组持续增长，单次长会话可以累积 GB 级别的无用对象，最终 OOM。

extractUsageFromGeminiClient 从初始提交起就没有被生产代码调用过，非交互模式实际走的 computeUsageFromMetrics。

…omGeminiClient

DragonnZhang

No issues found. Clean dead-code removal: debugResponses array, getDebugResponses(), and extractUsageFromGeminiClient are fully removed with zero remaining references. Test updates are consistent — the usage test in nonInteractiveCli.test.ts correctly relies on computeUsageFromMetrics via setupMetricsMock, matching the production path. CI test failures (all 3 platforms) appear pre-existing and unrelated to this change; Lint and CodeQL pass. LGTM.

zzhenyao · 2026-06-11T21:45:50Z

Thanks for the review! @DragonnZhang

Merged main into this branch

wenshao

No issues found. Clean dead-code removal: debugResponses array, getDebugResponses(), and extractUsageFromGeminiClient are fully removed with zero remaining references (git grep confirmed). Test updates are consistent — usage test in nonInteractiveCli.test.ts correctly migrated from mockGetDebugResponses to setupMetricsMock / computeUsageFromMetrics path. Build passes, tsc 0 errors, eslint 0 errors, 120 tests pass across all three affected test files. CI 13/13 green. ✅ — qwen3.7-max via Qwen Code /review

wenshao · 2026-06-12T23:17:14Z

@qwen-code /triage

wenshao · 2026-06-12T23:50:31Z

Local verification report (maintainer)

Verified this PR locally with real runs, not just the test suite. Since the PR's merge-base is ~50 commits behind main, all checks below were run on the merge result of this PR + latest origin/main (92c4a82) to rule out semantic conflicts that GitHub's MERGEABLE flag cannot catch.

1. Dead-code claim — confirmed, plus one stronger finding

On latest main, the only references to debugResponses / getDebugResponses / extractUsageFromGeminiClient are the definitions themselves and their test files. No production reader exists.
After merging this PR into latest main: whole-repo case-insensitive grep finds zero remaining references (all file types, not just .ts).
Stronger than "unused": extractUsageFromGeminiClient is not just unused — it is a no-op. It probes geminiClient.getChat().getDebugResponses(), but on current main the GeminiChat class has no getDebugResponses method at all (only Turn has one, and the helper never touches Turn). The helper can only ever return undefined, and the mock in nonInteractiveCli.test.ts was mocking a method that doesn't exist on the real class. The non-interactive usage path goes through computeUsageFromMetrics (4 call sites in nonInteractiveCli.ts), exactly as the PR description says.
One note for the record: Turn is re-exported from core's public index.ts (export * from './core/turn.js'), so removing getDebugResponses() technically shrinks the public surface of @qwen-code/qwen-code-core for external consumers. In-repo impact is zero — nothing imports it anywhere in the monorepo.

2. Build / typecheck / tests (merged tree)

Check	Result
`npm run build`	✅ exit 0
`npm run typecheck`	✅ exit 0
`core` targeted: `turn.test.ts`	✅ 23 passed
`cli` targeted: `nonInteractiveHelpers.test.ts` + `nonInteractiveCli.test.ts`	✅ 97 passed, 1 skipped (the skip is `it.skip` on `main` already, unrelated)
`core` full suite	✅ 405 files, 11466 passed, 4 skipped
`cli` full suite	⚠️ 8123 passed, 26 failed in 9 files — none attributable to this PR (see below)

On the cli full-suite failures: all 26 are pre-existing on latest main or load-induced flakes, in files that import nothing this PR touches (AuthDialog, InputPrompt, useAtCompletion, server, …).

AuthDialog.test.tsx fails identically on pure origin/main and on the merged tree (isolation run: 11 failed / 13 passed on both) — pre-existing on latest main in this environment (vi.waitFor timeouts).
useAtCompletion.test.ts fails on both trees as well.
The other 7 files only fail under CPU contention (two full suites ran concurrently) and pass when re-run on an idle machine; the failing subset also varies between back-to-back runs on the same tree, which is the signature of timing flakes, not a regression.

3. Real-run smoke tests (bundled CLI from the merged tree, driven via tmux)

Non-interactive (-p "What is 2+2? ..." --output-format json): exit 0, correct answer, and the final result event carries real usage — {"input_tokens":19365,"output_tokens":9,"total_tokens":19374} — produced by computeUsageFromMetrics. The removed helper is demonstrably not part of this path.
Interactive (tmux): multi-chunk streaming response renders fully; a tool-call turn (ReadFile) executes and returns correct content; /stats reports correct token accounting (58.3k input, 96.9% cached). Turn.run behaves identically without debugResponses.

4. Memory-retention benchmark — the OOM claim holds empirically

Drove Turn.run directly (mocked chat.sendMessageStream) with 2000 chunks carrying ~63 MB of unique text, measured with node --expose-gc on both builds:

Build	Retained while `Turn` alive	After `Turn` released
`main` (with `debugResponses`)	469 MB	0.1 MB
this PR (merged)	0 MB	0 MB

Both builds emitted an identical event stream (2001 events) — behavior parity. A WeakRef probe confirms the mechanism: on main, chunk objects are uncollectable for the entire Turn lifetime and become collectable the moment the Turn is released — exactly the retention pattern described in the PR (long thinking loops within one turn; nested goal-mode turns held until the outer call completes). (The 469 MB > 63 MB delta is benchmark string-rope overhead; production chunks are flat JSON.parse strings, so real retention ≈ payload + object overhead — still unbounded and proportional to chunk count.)

Verdict

✅ Safe to merge. Dead code with zero references after merging into latest main; build, typecheck, all targeted tests, and the full core suite are green (the cli suite's 26 failures are pre-existing on pure main or load-induced flakes — none attributable to this PR); real CLI behavior is intact in both modes; and the memory claim is reproducible, not just theoretical.

中文版(Chinese version)

本地验证报告(维护者)

本 PR 在本地做了真实运行验证,而不仅是跑测试。由于 PR 的 merge-base 落后 main 约 50 个提交,以下所有检查都在「本 PR 合入最新 origin/main(92c4a82)的合并结果」上进行,以排除 GitHub MERGEABLE 标志无法发现的语义冲突。

1. 死代码声明 — 成立,且有一个更强的发现

在最新 main 上,debugResponses / getDebugResponses / extractUsageFromGeminiClient 的引用只有定义本身和测试文件,生产代码零调用。
合入最新 main 后:全仓库大小写不敏感 grep(不限文件类型)零残留引用。
比"未使用"更强:extractUsageFromGeminiClient 不只是没人调用——它本身就是空操作。它探测 geminiClient.getChat().getDebugResponses(),但当前 main 上的 GeminiChat 类根本没有 getDebugResponses 方法(只有 Turn 有,而该 helper 从不接触 Turn)。这个 helper 永远只能返回 undefined,nonInteractiveCli.test.ts 里 mock 的是真实类上不存在的方法。非交互模式的 usage 实际走 computeUsageFromMetrics(nonInteractiveCli.ts 中 4 处调用),与 PR 描述一致。
备注:Turn 通过 core 的公开 index.ts 对外导出(export * from './core/turn.js'),删除 getDebugResponses() 严格来说缩小了 @qwen-code/qwen-code-core 对外部消费者的公开面。仓库内影响为零——monorepo 中无任何导入。

2. 构建 / 类型检查 / 测试(合并树)

检查	结果
`npm run build`	✅ exit 0
`npm run typecheck`	✅ exit 0
`core` 定向:`turn.test.ts`	✅ 23 通过
`cli` 定向:`nonInteractiveHelpers.test.ts` + `nonInteractiveCli.test.ts`	✅ 97 通过,1 跳过(该 skip 在 `main` 上即存在,与本 PR 无关)
`core` 全套	✅ 405 文件,11466 通过,4 跳过
`cli` 全套	⚠️ 8123 通过,9 个文件 26 个失败 — 均与本 PR 无关(见下)

关于 cli 全套的失败:26 个失败全部是最新 main 上预先存在的问题或负载导致的 flaky,所在文件(AuthDialog、InputPrompt、useAtCompletion、server 等)不导入本 PR 触及的任何代码。

AuthDialog.test.tsx 在纯 origin/main 和合并树上失败完全一致(隔离运行:两边均 11 失败 / 13 通过)——是最新 main 在本环境下预先存在的问题(vi.waitFor 超时)。
useAtCompletion.test.ts 同样在两棵树上都失败。
其余 7 个文件仅在 CPU 争抢下失败(当时 core/cli 两个全套并发运行),空闲机器重跑即通过;且同一棵树连续两次批量运行的失败子集都不相同——这是时序 flaky 的典型特征,不是回归。

3. 真实运行冒烟测试(合并树打包出的 CLI,tmux 驱动)

非交互模式(-p "What is 2+2? ..." --output-format json):exit 0,回答正确,最终 result 事件携带真实 usage——{"input_tokens":19365,"output_tokens":9,"total_tokens":19374},由 computeUsageFromMetrics 产出。被删除的 helper 确实不在该路径上。
交互模式(tmux):多 chunk 流式响应完整渲染;工具调用 turn(ReadFile)执行并正确返回内容;/stats 的 token 统计正确(输入 58.3k,缓存命中 96.9%)。去掉 debugResponses 后 Turn.run 行为不变。

4. 内存滞留基准 — OOM 声明有实证支撑

直接驱动 Turn.run(mock chat.sendMessageStream),流式输入 2000 个 chunk、约 63 MB 唯一文本,用 node --expose-gc 在两个构建上测量:

构建	`Turn` 存活期间滞留	`Turn` 释放后
`main`(含 `debugResponses`)	469 MB	0.1 MB
本 PR(合并后)	0 MB	0 MB

两个构建产出完全相同的事件流(2001 个事件)——行为等价。WeakRef 探针证实了机制:在 main 上,chunk 对象在 Turn 整个生命周期内不可回收,Turn 释放的瞬间即可回收——与 PR 描述的滞留模式完全吻合(单 turn 内长思考循环;goal 模式嵌套 turn 被持有直到外层调用结束)。(469 MB 大于 63 MB 的差额是基准测试字符串 rope 结构的开销;生产环境 chunk 来自 JSON.parse 的扁平字符串,真实滞留 ≈ 载荷 + 对象开销——但仍然无上界、随 chunk 数量线性增长。)

结论

✅ 可以安全合并。 死代码在合入最新 main 后零引用残留;构建、类型检查、全部定向测试及 core 全套测试全绿(cli 全套的 26 个失败要么在纯 main 上预先存在、要么是负载导致的 flaky——没有任何一个可归因于本 PR);两种模式下真实 CLI 行为完好;内存声明可复现而非纸面推断。

…omGeminiClient (#4982)

fix(core): remove unused debugResponses array and dead extractUsageFr…

f36b622

…omGeminiClient

zzhenyao marked this pull request as ready for review June 11, 2026 03:34

DragonnZhang reviewed Jun 11, 2026

View reviewed changes

github-actions Bot mentioned this pull request Jun 11, 2026

📊 AI CLI 工具社区动态日报 2026-06-11 litang9/big_model_radar#45

Open

Merge branch 'main' into fix/remove-debug-responses

0907aa4

zzhenyao changed the title ~~fix(core): remove dead debugResponses array to prevent OOM~~ fix(core): eliminate OOM from debugResponses accumulation Jun 11, 2026

wenshao approved these changes Jun 12, 2026

View reviewed changes

zzhenyao requested a review from DragonnZhang June 12, 2026 08:59

wenshao merged commit 8000342 into QwenLM:main Jun 12, 2026
28 checks passed

wenshao mentioned this pull request Jun 14, 2026

fix(cli,core): prevent memory monitor starvation during autonomous loops via heartbeat fallback #5097

Draft

zzhenyao mentioned this pull request Jun 15, 2026

Discussion: does the cli-entry.js --expose-gc wrapper earn the extra process? #5154

Closed

doudouOUC pushed a commit that referenced this pull request Jun 15, 2026

fix(core): remove unused debugResponses array and dead extractUsageFr…

0fbac3a

…omGeminiClient (#4982)

Navigation Menu

Repository navigation

fix(core): eliminate OOM from debugResponses accumulation - #4982

fix(core): eliminate OOM from debugResponses accumulation#4982
wenshao merged 2 commits into
QwenLM:mainfrom
zzhenyao:fix/remove-debug-responses

zzhenyao commented Jun 11, 2026 •

edited

Loading

Uh oh!

DragonnZhang left a comment

Uh oh!

zzhenyao commented Jun 11, 2026 •

edited

Loading

Uh oh!

wenshao left a comment

Uh oh!

wenshao commented Jun 12, 2026

Uh oh!

wenshao commented Jun 12, 2026

本地验证报告(维护者)

1. 死代码声明 — 成立,且有一个更强的发现

2. 构建 / 类型检查 / 测试(合并树)

3. 真实运行冒烟测试(合并树打包出的 CLI,tmux 驱动)

4. 内存滞留基准 — OOM 声明有实证支撑

结论

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

zzhenyao commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does

Why it is needed

Reviewer Test Plan

Before / After

How to verify

Uh oh!

DragonnZhang left a comment

Choose a reason for hiding this comment

Uh oh!

zzhenyao commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wenshao left a comment

Choose a reason for hiding this comment

Uh oh!

wenshao commented Jun 12, 2026

Uh oh!

wenshao commented Jun 12, 2026

Local verification report (maintainer)

1. Dead-code claim — confirmed, plus one stronger finding

2. Build / typecheck / tests (merged tree)

3. Real-run smoke tests (bundled CLI from the merged tree, driven via tmux)

4. Memory-retention benchmark — the OOM claim holds empirically

Verdict

本地验证报告(维护者)

1. 死代码声明 — 成立,且有一个更强的发现

2. 构建 / 类型检查 / 测试(合并树)

3. 真实运行冒烟测试(合并树打包出的 CLI,tmux 驱动)

4. 内存滞留基准 — OOM 声明有实证支撑

结论

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

zzhenyao commented Jun 11, 2026 •

edited

Loading

zzhenyao commented Jun 11, 2026 •

edited

Loading