Skip to content

Windows: 13 core tests fail on a clean checkout from unguarded environment preconditions, and the Windows CI job cannot be used to check #28830

Description

@Chirag6722

On a clean Windows checkout, npx vitest run in packages/core reports 14 files / 13 tests failed before any change is made. None of them indicate a product defect — every one traces to an environment precondition the repo knows about but does not guard.

The problem is not that they fail. It is that a Windows contributor cannot tell these apart from a regression they just caused, and has no reference run to compare against.

Test Files  14 failed | 392 passed | 2 skipped (408)
     Tests  13 failed | 7396 passed | 72 skipped (7481)

Class A — symlink creation needs a privilege Windows does not grant by default (7 tests)

Error: EPERM: operation not permitted, symlink
  'C:\...\Temp\planUtils-test-tGG8fO\outside.md' -> '...\plans\malicious.md'
  • fileUtils > getRealPath > should resolve symbolic links
  • planUtils > validatePlanPath > should detect path traversal via symbolic links
  • AllowedPathChecker > should allow access if path contains a symlink pointing INSIDE allowed directories
  • AllowedPathChecker > should deny access if path contains a symlink pointing outside allowed directories
  • ExitPlanModeTool > validateToolParams > should reject symbolic links pointing outside the plans directory
  • Consolidated At-Reference Path Resolution Tests > EditTool.getModifyContext handles symlink loops gracefully…
  • Consolidated At-Reference Path Resolution Tests > getCorrectedFileContent handles symlink loops gracefully

fs.symlinkSync on Windows requires Developer Mode or an elevated shell. None of these tests carry a platform guard.

Worth noting what is failing here: several are the tests that prove path-traversal containment. On a default Windows dev machine those assertions do not run, and the suite reports that as red rather than as unverified — which is the least useful of the three possible outcomes.

Class B — the Windows quoting suite needs PowerShell 7 (4 tests)

services/shellExecutionService.windows.integration.test.ts documents this in its own header:

These tests exercise the full pipeline end-to-end. They pass when gemini-cli selects pwsh.exe from PATH; they fail when the pipeline routes through Windows PowerShell 5.1.

  • should preserve inline double quotes through node -e
  • should preserve double quotes inside JSON output
  • should handle a mixed-quote regex literal
  • should pass a literal double-quote byte through to stdout

All four fail with expected 1 to be +0 — the real shell returned a non-zero exit. My machine has no pwsh.exe on PATH and runs Windows PowerShell 5.1.26100.8875, which is precisely the case the header names. The file guards on describe.skipIf(!isWindows) — it does not guard on the condition it says determines the outcome.

Why CI does not help a contributor here

test_windows in ci.yml is gated on:

if: "github.repository == 'google-gemini/gemini-cli' && needs.merge_queue_skipper.outputs.skip == 'false'"

so it does not run on forks. A Windows contributor cannot get a reference result on their own PR, and cannot tell whether their 13 failures are baseline. The repo also runs 67 Ubuntu jobs against 3 Windows jobs, so Windows is the least-covered supported platform and simultaneously the one with no fork-side signal.

Suggested direction

The repo already has the convention, including in the same directory as two of the failures (utils/pathReader.test.ts:524):

it.skipIf(process.platform === 'win32')(...)
describe.skipIf(os.platform() === 'win32')(...)   // sandbox/linux/bwrapArgsBuilder.test.ts
it.skipIf(Platform.isWindows)(...)                // services/sandboxManager.integration.test.ts
  1. Class A: skip on a capability probe rather than on the platform — attempt one fs.symlinkSync in a temp dir at suite setup and skip with a reason naming Developer Mode if it throws EPERM. Skipping on win32 alone would drop the coverage for Windows contributors who do have the privilege, which is the population most likely to be changing this code.
  2. Class B: skip when pwsh.exe is absent from PATH, with a reason that says so. The header already knows the predicate; it just is not expressed as a guard.

Both turn a red suite into an honest "skipped, here is why", which is the difference between a contributor trusting the suite and learning to ignore it.

I am happy to send a PR for either or both if the approach looks right. Flagging rather than assuming, since choosing a capability probe over a platform check is a judgement about how much Windows coverage you want to keep.

Environment

Windows 11, Node v24.16.0, npm ci on a clean clone of main at 2a87e7b, npx vitest run in packages/core. Windows PowerShell 5.1.26100.8875, no pwsh.exe on PATH, Developer Mode off.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/platformIssues related to Build infra, Release mgmt, Testing, Eval infra, Capacity, Quota mgmtkind/bugpriority/p2Important but can be addressed in a future release.status/bot-triaged

    Type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions