fix(core): bound tool output size and optimize memory lifecycle in long-running agent loops - #29451

Code Review

This pull request addresses memory leak issues and unbounded memory growth by introducing safety watermarks for chat history, collapsing older function responses, truncating large tool outputs, and explicitly dereferencing unused buffers and child process streams. A comprehensive regression test suite is also added. The review feedback highlights that the truncation logic in 'tool-executor.ts' misses handling a single 'Part' object within 'PartListUnion', which could bypass truncation, and suggests using grapheme-aware truncation to prevent character splitting.

Code Review

This pull request introduces memory leak prevention and mitigation mechanisms across the core package. Key changes include truncating large tool execution outputs to a 64 KB limit, collapsing older function response payloads from previous turns, triggering automatic compression based on safety watermarks (tokens and bytes), and explicitly dereferencing old history slices and buffers to facilitate V8 garbage collection. Comprehensive regression and unit tests have been added to validate these changes. The review feedback suggests two improvements: first, using Intl.Segmenter when slicing the preview string in collapseOlderFunctionResponses to prevent splitting multi-byte Unicode characters; second, recursively traversing and truncating nested objects and arrays in truncateFunctionResponsePart to ensure nested large strings do not bypass the truncation cap.

Code Review

This pull request introduces memory leak prevention and mitigation strategies across the core agent execution and tool execution paths. It implements safety watermarks for chat history token and byte sizes to trigger automatic compression, collapses older function responses to conserve memory, and truncates large tool outputs using grapheme-cluster-aware segmentation to preserve UTF-8 integrity. It also optimizes child process cleanup in ShellExecutionService by explicitly releasing streams and dereferencing buffers. The review feedback highlights important performance and correctness issues: first, the repeated instantiation of Intl.Segmenter inside nested loops is highly inefficient and should be refactored to reuse a single instance; second, the broad object check in truncateFunctionResponsePart needs to be narrowed to prevent corrupting non-plain objects like Buffer or Uint8Array.

_{Note: Security Review did not run due to the size of the PR.}

Code Review

This pull request introduces comprehensive memory management improvements and memory leak mitigations across the core agent execution and tool execution paths. Key changes include safety watermarks for chat history token count and byte size to trigger automatic compression, grapheme-cluster-aware truncation of large tool outputs, collapsing of older function responses, and explicit dereferencing of buffers and child process streams to facilitate garbage collection. Feedback on the changes suggests casting a response object to ensure TypeScript compilation under strict mode, and removing a redundant type cast on a child process stream.

_{Note: Security Review did not run due to the size of the PR.}

Code Review

This pull request introduces memory optimization mechanisms to prevent heap and RSS explosion during long-running agent executions, including tool output truncation (capped at 64 KB), older history collapsing (capped at 2 KB), and safety watermarks to trigger automatic compression. It also ensures explicit dereferencing of old history slices and child process streams to facilitate immediate V8 garbage collection. The feedback highlights critical type inconsistency issues where truncating or collapsing raw string responses wraps them in objects, which would break downstream consumers expecting strings. Additionally, a test mock issue was identified where mockSetHistory directly assigns the turns array, causing getHistory to return Turn[] instead of Content[] and silently bypassing the collapsing logic in tests.

_{Note: Security Review did not run due to the size of the PR.}

Code Review

This pull request introduces comprehensive memory management improvements and memory leak mitigations across the core agent execution flow. Key changes include grapheme-cluster-aware tool output truncation (capped at 64 KB), recursive truncation of large function response parts, and a mechanism to collapse older function responses in the chat history. Additionally, safety watermarks (512 KB / 50,000 tokens) are introduced to trigger automatic compression, and child process streams are explicitly released in the shell execution service to prevent heap retention. The feedback highlights an opportunity to make the history collapsing mechanism more robust by recursively traversing response objects to handle custom or third-party tools that use non-standard output keys.

_{Note: Security Review did not run due to the size of the PR.}

Code Review

This pull request introduces memory management improvements to prevent unbounded memory growth and RSS explosion during long-running agent loops. It implements grapheme-cluster-aware tool output truncation, collapses older function responses in the chat history while preserving the latest turn, and adds safety watermarks to trigger automatic compression. Additionally, it ensures proper cleanup of child process streams in the shell execution service and adds comprehensive unit and regression tests. Feedback on the changes suggests enhancing collapseOlderFunctionResponses to recursively traverse and collapse nested response objects, rather than only checking specific keys like output and content, to ensure robust memory bounding across all tools.

_{Note: Security Review did not run due to the size of the PR.}

Code Review

This pull request introduces memory management optimizations and memory leak mitigations within the Gemini CLI core. Key changes include implementing grapheme-cluster-aware truncation of large tool execution outputs (capped at 64 KB) and recursive collapsing of older function responses (capped at 2 KB) to prevent heap and RSS explosion. Additionally, it introduces safety watermarks (50,000 tokens or 512 KB) to automatically trigger chat compression, explicitly dereferences old history slices and child process streams to facilitate garbage collection, and adds comprehensive regression and unit tests. No review comments were provided, so there is no feedback to evaluate.

_{Note: Security Review did not run due to the size of the PR.}

Findings

🚨 Critical Issues (Blockers)

1. Severe Context Loss in Multi-Turn Agent Loops (collapseOlderFunctionResponses)

   • In local-executor.ts (lines 347 & 918–926), collapseOlderFunctionResponses(currentHistory) runs unconditionally at the start of every single turn inside tryCompressChat.
   • The logic finds all tool responses in history and truncates every tool response prior to the immediate last one (idx < lastToolIndex) down to 512 bytes (Math.min(maxBytesPerOldResponse, 512)).
   • Impact: In standard agent workflows requiring multi-file operations (e.g. Turn 1: read_file('foo.ts'), Turn 2: read_file('bar.ts'), Turn 3: edits or writes code):
     • At Turn 3, the contents of foo.ts from Turn 1 are wiped out and replaced with a 512-byte snippet plus ... [Tool output collapsed from previous turn: X bytes omitted to conserve memory] ....
     • The agent loses the content of previously read files and test outputs, forcing repetitive re-reading loops or inducing model hallucinations.
   • Recommendation:
     • Do not run collapseOlderFunctionResponses unconditionally on every turn in LocalAgentExecutor.
     • Unlike browser snapshot replacement (snapshotSuperseder.ts where snapshots are strictly ephemeral), file reads, search results, and tool outputs in developer workflows must remain in context.
     • If tool outputs need to be collapsed under memory pressure, it should adhere to ContextCompressionService principles: protect recent turns (e.g. RECENT_TURNS_PROTECTED = 2), exempt content-reading tools like read_file / read_many_files, or only execute when memory/context thresholds are actually reached.

2. Premature Auto-Compression Trigger (COMPRESSION_SAFETY_WATERMARK_TOKENS = 50_000)

   • In chatCompressionService.ts, isOverWatermarkTokens = originalTokenCount >= COMPRESSION_SAFETY_WATERMARK_TOKENS triggers automatic compression at 50,000 tokens.
   • Impact: Gemini models (e.g., gemini-2.5-pro, gemini-2.5-flash) offer 1,000,000 to 2,000,000 token context windows. With this watermark:
     • Compression (which drops 70% of history and replaces it with an LLM summary) is forcibly triggered at 2.5% to 5% of model context capacity.
     • The user-configured compressionThreshold (default 50%, i.e., 500,000–1,000,000 tokens) is completely bypassed and rendered dead code.
     • 50,000 tokens of text is ~200 KB in memory—negligible for the multi-gigabyte Node.js V8 heap. The 10.5 GB heap leak in #28537 was driven by uncollected child process streams and unbounded tool buffers, not a 50k token conversation.
   • Recommendation: Remove COMPRESSION_SAFETY_WATERMARK_TOKENS (or make it configurable / scale it proportionally to the model's actual token limit) and rely on the model token threshold and byte boundaries.

3. Turn ID Regeneration Breaks Session Tracking (LocalAgentExecutor)

   • In local-executor.ts:

     const turns = collapsedHistory.map((c) => ({
       id: randomUUID(),
       content: c,
     }));
     chat.setHistory(turns);

   • Whenever history is modified, every existing turn receives a newly generated random UUID. This breaks stable ID tracking across the message bus, recording services, and session resumption. Existing turn IDs should be preserved.

💡 Improvements & Suggestions

1. Mismatch between Constant and Effective Preview Size

   • COLLAPSED_FUNCTION_RESPONSE_MAX_BYTES is defined as 2048 (2 KB). However, in collapseString:

     const previewBytes = Math.min(maxBytesPerOldResponse, 512);

     This hardcodes the effective preview cap to 512 bytes regardless of maxBytesPerOldResponse. If 2 KB was intended, previewBytes should use maxBytesPerOldResponse.

2. Disk Fallback for Truncated Tool Outputs

   • In ToolExecutor, when shell outputs exceed limits, saveTruncatedToolOutput persists the full output to a temp file on disk and gives the LLM the file path.
   • For generic tool truncation at MAX_STORED_TOOL_OUTPUT_BYTES = 64 * 1024, outputs exceeding 64 KB are clipped directly with an omission notice. Consider saving the original payload to disk so users and tools (like web_fetch or custom tools) can still retrieve the complete output if needed.

Code Review

This pull request introduces memory optimization and leak prevention mechanisms to address unbounded memory growth during long-running agent execution. Key changes include truncating large tool outputs exceeding 64 KB using grapheme-cluster-aware segmentation, collapsing older function responses in the chat history while preserving the most recent three turns and exempting retrieval tools, and releasing child process streams, event listeners, and native buffers in the shell execution service. Additionally, old history slices and buffers are explicitly dereferenced to facilitate immediate V8 garbage collection. Comprehensive regression and unit tests have been added to validate these optimizations. I have no further feedback to provide as no review comments were submitted.