Repository navigation
test(integration): make the utf-bom defaultFileEncoding test deterministic - #13196
Conversation
…istic The 'should create new file with BOM when defaultFileEncoding is utf-8-bom' integration test was the only case in utf-bom-encoding.test.ts that drove a real model round-trip (rig.run + waitForToolCall, up to 300s per attempt). On the loaded Docker self-hosted runner that round-trip can exceed the vitest timeout, which blocked the v0.24.8-preview.0 release (QwenLM#13066). Convert it to runForcedToolCallScenario with a fake write_file call, the same deterministic pattern as the sibling tests in this file. The real write_file tool still executes, so the BOM assertions are unchanged. Also unstub the env vars the scenario stubs, matching the first describe block. Closes QwenLM#13066 Co-authored-by: doudouOUC <[email protected]>
|
Head 319c941 is fully green now — every check has completed with 0 failures, nothing running. The earlier "Downgraded from Approve to Comment: CI still running" looks like it raced a check that was still pending at that moment. Could a maintainer re-run |
|
Verdict: 中文摘要结论:merge-ready(从验证角度可以合并)。 在 macOS(darwin/arm64,Node 22.23.1)本地真实构建环境中完成验证,head/base 双 worktree 各自独立
Central claimThe A/B load-bearing proofTwo worktrees that differ only in the PR commit, each independently installed (
The base arm's red is the bug's own signature in local form: the old test cannot pass at all without a live authenticated model, and with one it costs a real round trip per attempt — which is exactly what exceeded the 300s vitest timeout three times in a row on the loaded Docker release runner (quoted from release run Control cleanliness: Full-file parity (collateral sweep)The whole file (8 tests) run on both arms under the same credential-free environment:
The red→green delta is exactly the converted test. The 2 remaining reds are the pre-existing real-model preservation cases ( Mutation matrix (vacuity check)Two mutants applied to the converted test at the head tree, one at a time; expected outcome for both: killed (red).
M1 is the load-bearing one: it proves the converted test still exercises the full chain Targeted gates (head
|
| Gate | Result |
|---|---|
Converted test, -t targeted, credential-free |
PASS ~2s (run twice: plain + capture) |
| Full file, credential-free | 6/8 pass; 2 pre-existing real-model reds (parity with base) |
tsc -p integration-tests/tsconfig.json |
exit 0 |
eslint integration-tests/cli/utf-bom-encoding.test.ts |
0 problems |
prettier --check on the file |
pass |
ESLint liveness was proven before citing it: a planted const UNUSED_PROBE = 42; was reported (@typescript-eslint/no-unused-vars), then removed and the file re-verified clean.
Findings
None. One observation, not a code issue: the PR also adds afterEach(() => vi.unstubAllEnvs()) to the second describe block, matching the first block — this matters because runForcedToolCallScenario stubs env vars, and the new test is that block's only vi.stubEnv consumer; the addition is correct and consistent.
Not covered
- Docker sandbox lane — this round ran on macOS (darwin/arm64) with
QWEN_SANDBOX=false; the release-blocking lane is Linux/Docker. The mechanism under test (model round trip vs scripted fake call) is platform-independent, and the PR's own CI (incl.Integration Tests (no-AK, No Sandbox)) is green. - The two pre-existing real-model preservation cases — untouched by this PR; verified only for cross-arm parity (identical no-auth failure on both arms).
- Windows — the whole file is Windows-skipped by pre-existing design.
Methodology
macOS (darwin/arm64), Node v22.23.1. Head tree: git worktree at 319c941f3d; base tree at 47463b79a7 (head's parent). Each tree: corepack pnpm install --frozen-lockfile → npm run build → npm run bundle. Test env: env -i HOME=<empty tmp> PATH=$PATH QWEN_SANDBOX=false CI=true (credential-free; CI=true only extends waitForToolCall polling, never reached on red arms since the CLI exits immediately). Commands: npx vitest run --root ./integration-tests cli/utf-bom-encoding.test.ts [-t ...]. Mutants: edit at head tree, rerun, git checkout -- restore, git status verify. Counts: 2 targeted head runs + 2 targeted base runs (expected-red) + 8 full-file head cells + 8 full-file base cells + 2 mutants + 4 gates (eslint probe, eslint clean, prettier, tsc) + 2 workspace-link realpath checks = 26 pass, 0 unexpected fail.
— local verification by @wenshao




What this PR does
Converts the last real-model test in the BOM integration suite (
should create new file with BOM when defaultFileEncoding is utf-8-bom) to the deterministic forced-tool-call pattern used by the seven sibling tests in the same file: the fake server now scripts thewrite_filecall, while the real tool still writes the real file, so the byte-level assertions are unchanged.Why it's needed
The
defaultFileEncodingcase was the only test inutf-bom-encoding.test.tsthat drove an actual model round-trip (rig.run+waitForToolCall, up to 300s per attempt). On the loaded Docker self-hosted runner that round-trip exceeded the vitest timeout and failed theintegration_dockerjob, blocking the v0.24.8-preview.0 release (#13066). The issue discussion diagnoses this as a test determinism problem and suggests converting this test torunForcedToolCallScenariowith a fakewrite_filecall — this PR applies exactly that.Reviewer Test Plan
How to verify
npm run build && npm run bundle, thencd integration-tests && cross-env QWEN_SANDBOX=false npx vitest run cli/utf-bom-encoding.test.ts.Expected: the
defaultFileEncodingcase now finishes in seconds instead of waiting on a model round-trip, and still asserts the EF BB BF prefix written by the realwrite_filetool under thedefaultFileEncoding: utf-8-bomsetting. No real API key is involved (fake OpenAI server).Evidence (Before & After)
N/A — no user-facing change (test-only).
Tested on
Verified locally on Windows 11 (Node v24.12.0): the converted test passes in ~6s; the other seven cases stay skipped on Windows by pre-existing design. Linux/macOS lanes are covered by CI.
Environment
Local bundle (
npm run bundle),QWEN_SANDBOX=false, fake OpenAI server — no real model credentials.Risk & Scope
write_filetool → BOM bytes) is untouched, and unit coverage of the BOM logic remains in place.Linked Issues
Closes #13066
中文说明
这个 PR 做了什么
把 BOM 集成测试套件中最后一个走真实模型的用例(
should create new file with BOM when defaultFileEncoding is utf-8-bom)改为与同文件其余七个用例一致的确定性 forced-tool-call 模式:由假服务器脚本化下发write_file调用,真实的 write_file 工具仍然写真实文件,因此字节级断言不变。为什么需要
defaultFileEncoding这个用例原本是utf-bom-encoding.test.ts中唯一走真实模型往返的测试(rig.run+waitForToolCall,每次最长等 300s)。在高负载的 Docker 自托管 runner 上,该往返超过 vitest 超时,导致integration_dockerjob 失败并阻塞 v0.24.8-preview.0 发布(#13066)。issue 讨论中已诊断这是测试确定性问题,并建议把该用例改为runForcedToolCallScenario+ 假write_file调用——本 PR 即按此方案实现。评审验证计划
如何验证
先
npm run build && npm run bundle,再cd integration-tests && cross-env QWEN_SANDBOX=false npx vitest run cli/utf-bom-encoding.test.ts。预期:转换后的用例几秒内完成,不再等待模型往返,仍断言真实 write_file 工具在defaultFileEncoding: utf-8-bom设置下写出的 EF BB BF 前缀。无需真实 API key(假 OpenAI 服务器)。前后对比证据
N/A —— 无用户可见变更(仅测试)。
测试环境
已在 Windows 11(Node v24.12.0)本地验证:转换后的用例约 6 秒通过;其余七个用例在 Windows 上按既有设计跳过;Linux/macOS 由 CI 覆盖。
风险与范围
关联 Issue
Closes #13066