Skip to content

fix(core): replace fuzzy requestedExplicitly logic with glob matching in read-many-files - #29457

Closed
villahernandez-coder wants to merge 14 commits into
google-gemini:mainfrom
villahernandez-coder:FixBug-cla-561554390
Closed

villahernandez-coder wants to merge 14 commits into
google-gemini:mainfrom
villahernandez-coder:FixBug-cla-561554390

Conversation

@villahernandez-coder

@villahernandez-coder villahernandez-coder commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes a critical context-bloat bug (b/561554390 / #29045) in read-many-files where binary assets (images, PDFs, audio) were incorrectly treated as "explicitly requested" due to naive String.prototype.includes() fuzzy substring matching on file stems and extensions, causing inadvertent base64 inlining.

Details

  • Glob-Based Pattern Evaluation: Replaced raw string substring matching with isAssetExplicitlyRequested() utilizing the repository's standard glob library (picomatch, zero new dependencies).
  • Strict Differentiation: Broad wildcard patterns and directory traversals (e.g., *, **/*, src/**, **/*report*/**, assets/*) no longer trigger explicit binary inlining. An asset is only treated as explicitly requested if an include pattern matches the file AND explicitly targets either:
    1. A specific extension (e.g., *.png, **/*.png, assets/*.PNG, *.{png,jpg}) matching the file's extension case-insensitively.
    2. A specific non-wildcard filename stem/path (e.g., assets/logo.png, logo.png, myExactImage.png, report-final.pdf).
  • Path Separator & Casing Consistency:
    • Normalized all path separators (\ to /) across include patterns and file paths to adhere to repository standards on Windows and POSIX.
    • Ensured consistent case-insensitive extension matching via picomatch(patternExt, { nocase: true }).
  • Test Coverage: Added comprehensive test cases in read-many-files.test.ts covering false-positive rejection (**/*report*/** vs reports/quarterlyreport.png), explicit binary requests (assets/logo.png), path separators (assets\logo.png), case-insensitivity (assets/*.PNG, assets/Logo.png), and unit tests for isAssetExplicitlyRequested.

Related Issues

Fixes #29045

How to Validate

  1. Run unit test suite for read-many-files:
    npm test -w @google/gemini-cli-core -- src/tools/read-many-files.test.ts

     *Expected outcome*: All 40 test cases pass, including Test cases A, B, and C.
    
  2. Run typecheck and linting across the workspace:
    npm run typecheck
    npm run lint

    *Expected outcome*: 0 errors, 0 warnings.
    
  3. Run core package unit tests:
    npm test -w @google/gemini-cli-core

     *Expected outcome*: All 412 test files pass (8,057+ tests).
    

Pre-Merge Checklist

  • Updated relevant documentation and README (if needed)
  • [ X] Added/updated tests (if needed)
  • Noted breaking changes (if any)
  • [ X] Validated on required platforms/methods:
    • MacOS
      • npm run
      • npx
      • Docker
      • Podman
      • Seatbelt
    • Windows
      • npm run
      • npx
      • Docker
    • [ X] Linux
      • [ X] npm run
      • npx
      • Docker

@github-actions github-actions Bot added the size/l A large sized PR label Sep 23, 2026
@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

📊 PR Size: size/L

  • Lines changed: 582
  • Additions: +565
  • Deletions: -17
  • Files changed: 2

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical bug where binary assets were incorrectly inlined due to overly permissive fuzzy matching. It replaces this logic with a strict, glob-based evaluation system. Additionally, it enhances the stability of the CLI by introducing atomic file operations, protecting against prototype pollution, and improving handling of file access and authentication across app relaunches and platform-specific environments.

Highlights

  • Binary Asset Handling: Replaced naive substring matching with robust glob-based pattern evaluation using picomatch to prevent accidental binary asset inlining.
  • Security and Reliability: Added prototype pollution protection to JSON settings updates and implemented atomic file writes to improve configuration reliability.
  • Authentication and Relaunching: Introduced GEMINI_CLI_AUTH_OVERRIDE to support auth flow transitions during app relaunch and added retry logic for OAuth credential file access.
  • Platform Compatibility: Added detection for WSL and headless Linux environments to bypass native keychain issues that cause process lockups.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces several robustness, security, and environment-handling improvements across the CLI and core packages. Key changes include: supporting a GEMINI_CLI_AUTH_OVERRIDE environment variable to propagate authentication selection across application relaunches; implementing atomic file writes and read retries in commentJson and oauth2 to prevent corruption and handle transient file locks; adding prototype pollution protection when updating JSON settings; bypassing the native keychain on WSL and headless Linux environments to avoid libsecret lockups; and refining the ReadManyFilesTool to only include binary assets (images, PDFs, audio) when they are explicitly requested by name or extension rather than implicitly matched by broad glob patterns. I have no additional feedback to provide as no review comments were submitted.

Note: Security Review did not run due to the size of the PR.

@gemini-cli gemini-cli Bot added priority/p1 Important and should be addressed in the near term. area/core Issues related to User Interface, OS Support, Core Functionality labels Sep 23, 2026
@villahernandez-coder

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces several robustness and security improvements across the codebase. Key changes include adding prototype pollution protection and atomic file writes in JSON settings utilities, implementing retry mechanisms for reading OAuth credentials, bypassing native keychains in WSL/headless Linux environments to prevent lockups, and refining the read-many-files tool to prevent binary assets (images, PDFs, audio) from being implicitly matched by broad glob patterns. Additionally, it supports propagating authentication overrides across process relaunches. The reviewer suggests extending the binary asset check in the read-many-files tool to also cover 'video' files to prevent potential context bloat and memory issues.

Comment thread packages/core/src/tools/read-many-files.ts
@villahernandez-coder

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces several robustness and feature enhancements, including authentication type overrides via environment variables during relaunch, bypassing native keychains in WSL/headless Linux to prevent lockups, adding video file support to the file-reading tool, and implementing safer file operations (atomic writes, retries, and prototype pollution prevention). Feedback on these changes highlights two key issues in the file utility implementation: first, using Atomics.wait on the main thread throws a TypeError in Node.js, resulting in an inefficient CPU-intensive busy-wait loop; second, using Date.now() for temporary file naming is prone to collisions during rapid sequential writes, which can be resolved by appending a random suffix.

Comment thread packages/cli/src/utils/commentJson.ts Outdated
Comment thread packages/cli/src/utils/commentJson.ts
@villahernandez-coder

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces several robust enhancements across the CLI and Core packages. Key updates include support for overriding the selected authentication type via the GEMINI_CLI_AUTH_OVERRIDE environment variable across process relaunches, security improvements in JSON settings writing (such as prototype pollution prevention and atomic writes), a retry mechanism for reading OAuth credentials to handle transient filesystem errors, WSL and headless Linux detection to bypass native keychains and avoid libsecret lockups, and refined asset matching in the ReadManyFilesTool using picomatch to prevent broad globs from implicitly matching binary assets (including video). I have no feedback to provide as there are no review comments.

Note: Security Review did not run due to the size of the PR.

Comment thread packages/core/src/tools/read-many-files.ts Fixed
@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

🧠 Model Steering Guidance

This PR modifies files that affect the model's behavior (prompts, tools, or instructions).

  • ⚠️ Consider adding Evals: No behavioral evaluations (evals/*.eval.ts) were added or updated in this PR. Consider adding a test case to verify the new behavior and prevent regressions.

This is an automated guidance message triggered by steering logic signatures.

@villahernandez-coder

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces several improvements and bug fixes across the codebase. It updates temporary file path generation in writeAtomicSync and cacheCredentials to append a random alphanumeric string, preventing potential collisions. It enhances readOAuthCredsWithRetry to support a callback function for maxRetries, allowing dynamic re-evaluation. Additionally, it refactors asset handling in ReadManyFilesTool by adding support for video files and introducing a robust isAssetExplicitlyRequested helper using picomatch to accurately determine if asset files (images, PDFs, audio, and video) are explicitly requested by name or extension rather than implicitly matched by broad glob patterns. Comprehensive unit tests have been added to verify these changes. There are no review comments, so I have no feedback to provide.

@DavidAPierce
DavidAPierce self-requested a review October 1, 2026 23:38
- Introduce optional AssetRequestCache parameter in isAssetExplicitlyRequested using mnemonist LRUCache
- Instantiate assetRequestCache per ReadManyFilesToolInvocation to cache compiled matchers and scan results
- Add unit test verifying cache population and reuse
@villahernandez-coder

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for video files to the ReadManyFilesTool and introduces a robust isAssetExplicitlyRequested helper function to prevent asset files from being implicitly matched by broad glob patterns. It also integrates an LRUCache to cache compiled matchers and scan results. The review feedback highlights a potential cache key collision in isAssetExplicitlyRequested where raw pattern keys could collide with prefixed keys like scan: and matcher:, suggesting prefixing pattern matcher keys with pattern: and updating the corresponding test assertions.

Comment thread packages/core/src/tools/read-many-files.ts Outdated
Comment thread packages/core/src/tools/read-many-files.test.ts Outdated
- Prefix compiled pattern matcher cache keys with 'pattern:' to avoid collisions with 'scan:' and 'matcher:' entries
- Update unit test assertions in read-many-files.test.ts to expect the prefixed cache key
@villahernandez-coder

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the ReadManyFilesTool to support video files as assets and introduces a more robust isAssetExplicitlyRequested helper to determine if asset files (images, PDFs, audio, and video) are explicitly requested by name or extension rather than implicitly matched by broad glob patterns. It also integrates an LRUCache to cache compiled glob matchers and scan results for better performance. The review feedback points out a potential cross-platform issue where platform-dependent path.extname and path.basename are used on paths before normalization, and suggests using path.posix on the normalized path instead to ensure consistent behavior across POSIX and Windows environments.

Comment thread packages/core/src/tools/read-many-files.ts Outdated
…itlyRequested

- Use path.posix.extname and path.posix.basename on normalizedFilePath instead of platform-dependent path methods on raw filePath
- Add test case verifying Windows-style backslashes in filePath and relativePathForDisplay
@villahernandez-coder

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enhances the ReadManyFilesTool by adding support for video files and introducing a more robust, cached helper function isAssetExplicitlyRequested to determine if asset files (images, PDFs, audio, and video) are explicitly requested by name or extension rather than implicitly matched by broad glob patterns. Comprehensive unit tests have been added to validate these changes. Feedback on the code suggests using the project's robust resolveToRealPath utility for workspace directory path comparisons to ensure consistent path resolution across different environments.

Comment thread packages/core/src/tools/read-many-files.ts Outdated
…son in read-many-files

- Import and use resolveToRealPath in isAssetExplicitlyRequested for canonical path comparison
- Handle cross-platform path resolution consistently across POSIX and Windows environments
@villahernandez-coder

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a robust mechanism (isAssetExplicitlyRequested) to determine if asset files (images, PDFs, audio, and now video files) are explicitly requested by name or extension rather than implicitly matched by broad glob patterns. It also integrates picomatch and LRUCache to optimize pattern matching. The review feedback highlights a critical performance optimization to avoid redundant synchronous I/O operations (resolveToRealPathSafe) inside the loop by pre-resolving workspace directories.

Comment thread packages/core/src/tools/read-many-files.ts
…icitlyRequested

- Pre-resolve workspaceDirs and the file directory once at the beginning of isAssetExplicitlyRequested
- Avoid redundant synchronous fs.realpathSync / resolveToRealPathSafe calls inside pattern matching loops
@villahernandez-coder

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the isAssetExplicitlyRequested helper function to determine if an asset file (image, PDF, audio, or video) is explicitly requested by name or extension rather than implicitly matched by a broad glob pattern. It also adds support for video files, integrates caching of compiled matchers using an LRU cache to optimize performance, and includes comprehensive unit tests validating these changes under various glob patterns, path separators, and casing scenarios. I have no feedback to provide as there are no review comments to address.

@DavidAPierce

Copy link
Copy Markdown
Contributor

Findings

🔴 Critical / High Priority

None identified. The implementation is mathematically sound, avoids ReDoS, eliminates false-positive binary inlining, and passes all edge cases.


🟡 Improvements

  1. Pre-resolve workspaceDirs at the invocation level rather than per-asset call

    • File: packages/core/src/tools/read-many-files.ts (lines 154–156)
      const resolvedWorkspaceDirs = workspaceDirs.map((dir) =>
        resolveToRealPathSafe(dir),
      );
    • Observation: In commit 6318023ed, resolving workspace directories was moved out of the inner pattern loop. However, isAssetExplicitlyRequested is still called once per asset file inside sortedFiles.map(async (filePath) => ...). When processing batches with many binary assets, resolveToRealPathSafe(dir) performs synchronous filesystem calls (fs.realpathSync) repeatedly for the same workspace directories.
    • Recommendation: Resolve workspaceDirs once in ReadManyFilesToolInvocation (or memoize the resolved array) and pass the pre-resolved directories into isAssetExplicitlyRequested.
  2. Consistent POSIX path method usage

    • File: packages/core/src/tools/read-many-files.ts (lines 157–160)
      const fileExtension = path.posix.extname(normalizedFilePath);
      const fileName = path.posix.basename(normalizedFilePath);
      ...
      const resolvedFileDir =
        workspaceDirs.length > 0
          ? resolveToRealPathSafe(path.dirname(normalizedFilePath))
          : '';
    • Observation: Lines 148–149 use path.posix.extname and path.posix.basename on normalizedFilePath (which has forward slashes). Line 159 uses platform-dependent path.dirname(normalizedFilePath) rather than path.posix.dirname(normalizedFilePath).
    • Recommendation: Use path.posix.dirname(normalizedFilePath) for consistency across Windows and POSIX.

🟢 Nitpicks

  1. JSDoc documentation for isAssetExplicitlyRequested parameters
    • File: packages/core/src/tools/read-many-files.ts (line 145)
    • Observation: Adding JSDoc @param tags documenting workspaceDirs and cache will clarify the purpose of workspace root anchoring for slash-free patterns.

@gemini-cli

gemini-cli Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

This pull request is being closed as it has been open for 14 days without a 'help wanted' designation. We encourage you to find and contribute to existing 'help wanted' issues in our backlog! Thank you for your understanding.

This branch was successfully deployed

1 active deployment
eval-gate — 6318023e Deployed Oct 2, 2026 by villahernandez-coder via Evaluate Steering & Regressions #2132
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/core Issues related to User Interface, OS Support, Core Functionality priority/p1 Important and should be addressed in the near term. size/l A large sized PR size/xl An extra large PR status/pr-nudge-sent

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: read-many-files treats substring overlap as 'explicitly requested' binary asset, inlining unrequested images

3 participants