Skip to content

test(integration): support deferred MCP tool calls in SDK E2E - #12365

Merged
wenshao merged 1 commit into
QwenLM:mainfrom
dvd233:codex/fix-mcp-sdk-bridge-12357
Sep 21, 2026
Merged

wenshao merged 1 commit into
QwenLM:mainfrom
dvd233:codex/fix-mcp-sdk-bridge-12357

Conversation

@dvd233

@dvd233 dvd233 commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

What this PR does

Update the MCP SDK integration assertions to recognize deferred MCP invocations represented by the stable tool_call bridge envelope, while continuing to accept direct MCP tool-use blocks.

Why it's needed

After #10410, deferred MCP tools are invoked through tool_call with the target tool name in the input payload. The existing integration assertions still looked only for the legacy top-level MCP tool name, so the main E2E job reports seven false failures in #12357 even though the bridge executes the tools successfully.

Reviewer Test Plan

How to verify

Run the MCP SDK integration E2E with a configured model. Exercise add, multiply, chained calls, repeated calls, multi-turn calls, permission callbacks, and message-flow assertions. The calls should be recognized for both the direct form and the tool_call bridge form, and all seven failures from #12357 should pass.

Evidence (Before & After)

N/A — no user-visible change; this PR updates E2E assertions only.

Tested on

OS Status
🍏 macOS N/A
🪟 Windows ⚠️ not tested (no model credentials available locally)
🐧 Linux N/A

Environment (optional)

Node 24.15.0, pnpm 11.24.0, Windows. Prettier, ESLint, and integration TypeScript checks passed. A related fake-model SDK MCP bridge run passed 3 of 4 cases; the remaining error-handling case exited with Windows status 3221226505.

Risk & Scope

  • Main risk or tradeoff: This changes only how E2E assertions identify the target of a bridged MCP call; runtime behavior is unchanged.
  • Not validated / out of scope: The target E2E suite against a real model could not be run locally because no provider credentials are available.
  • Breaking changes / migration notes: None.

Linked Issues

Fixes #12357

中文说明

本 PR 做了什么

更新 MCP SDK 集成测试断言,使其识别由稳定的 tool_call 桥接信封表示的延迟 MCP 调用,同时继续兼容直接的 MCP 工具调用块。

为什么需要它

在 #10410 之后,延迟 MCP 工具通过 tool_call 调用,目标工具名位于输入载荷中。现有集成测试断言只查找旧的顶层 MCP 工具名,因此主分支 E2E 作业在 #12357 中报告了七个误失败,尽管桥接实际能够成功执行工具。

审查者测试计划

如何验证

在已配置模型的环境中运行 MCP SDK 集成 E2E。覆盖 add、multiply、链式调用、重复调用、多轮调用、权限回调和消息流断言。对于直接形式和 tool_call 桥接形式,都应能识别调用,并且 #12357 中的七个失败应全部通过。

证据(前后对比)

不适用——没有用户可见变化;本 PR 只更新 E2E 断言。

测试平台

操作系统 状态
🍏 macOS 不适用
🪟 Windows ⚠️ 未测试(本地没有可用模型凭据)
🐧 Linux 不适用

环境(可选)

Node 24.15.0、pnpm 11.24.0、Windows。Prettier、ESLint 和集成测试 TypeScript 检查均通过。相关的 fake-model SDK MCP 桥接运行通过 4 个用例中的 3 个;剩余的错误处理用例以 Windows 状态码 3221226505 退出。

风险与范围

  • 主要风险或取舍:本改动只改变 E2E 断言识别桥接 MCP 调用目标的方式;运行时行为不变。
  • 未验证/不在范围内:由于本地没有 provider 凭据,无法在真实模型上运行目标 E2E 套件。
  • 破坏性变更/迁移说明:无。

关联 Issue

Fixes #12357

@wenshao

wenshao commented Sep 21, 2026

Copy link
Copy Markdown
Collaborator

Local verification — real CLI build + real MCP server, scripted model

I built the E2E environment locally and ran sdk-typescript/mcp-server.test.ts on both arms. The PR does what it says: the exact 7 failures from #12357 reproduce on unpatched main and all 9 tests go green with this patch, while the legacy direct-call shape keeps passing. Recommending merge.

Worth flagging up front: this PR comes from a fork, so e2e.yml skips the E2E Tests job by design (github.event.pull_request.head.repo.full_name == github.repository). The green checks on this PR therefore do not exercise the tests it changes — the run below is the substitute for that.

Rig

CLI under test real dist/cli.js, bundled by pnpm install's prepare from origin/main cc5ee0a4 — the same path globalSetup.ts hands to TEST_CLI_PATH
MCP server the suite's own stdio math server (createMCPServer), real node child process
Model a local OpenAI-compatible server reusing the repo's integration-tests/fake-openai-server.ts wire implementation, scripted per prompt
Isolation pinned QWEN_HOME, --retry=0, one fork, no proxy for 127.0.0.1
Arms one build shared by both arms; the only difference is mcp-server.test.ts (+32 / −7)

Only the model is substituted — everything the assertions read (SDK transcript, bridge resolution, permission path, MCP execution) is the real product code.

Result

A/B run

Arm A's failure text is identical to main CI run 35545401904 — same 7 tests, same messages, including the [ 'tool_search', 'tool_call', …(N) ] to include 'mcp__test-math-server__add' array shape, expected 0 to be greater than or equal to 2 and expected undefined to be defined.

The failures really were false

A probe through the same CLI and SDK, printing what the transcript carries:

transcript probe

The MCP tool ran and returned 15 with is_error=false; the only thing wrong on main was that the assertion looked at the envelope's model-facing name. That matches the bridge contract in docs/design/deferred-tool-call-bridge.md ("the function response ... retains the bridge name") and coreToolScheduler.resolveToolCallBridgeRequest, which keeps modelFacingName on the wire and resolves the target only for execution.

Three wire shapes × two arms

wire shape matrix

The top row is the positive control the PR description promises: with the pre-#10410 top-level name, both arms stay green, so the patch adds recognition of the envelope without dropping the direct form.

Arm A, one failure in detail

failure detail

Cross-checks

  • canUseTool assertion (line 463) is untouched and still correct. The permission callback receives the resolved MCP names, not tool_call, because _schedule unwraps the envelope before anything downstream sees it. Measured: arm B's should support multi-turn MCP tools with canUseTool passes through line 463, which arm A never reached.
  • Gates on the patched file: typecheck:integration ✓, eslint --max-warnings 0 ✓, prettier --check ✓.
  • Scope looks complete. In CI run 35545401904 the sdk shard is 1 failed | 12 passed — this file is the only casualty — and the cli shard stays green, including cli/simple-mcp-server.test.ts, which matches through telemetry (rig.waitForToolCall) and therefore already sees the resolved target name.
  • Merge result against today's main (17b5c940) still touches only this file; no other open PR edits it.

Non-blocking observations

  1. These assertions prove an invocation was attempted, not that it succeeded — and that is pre-existing, not something this PR introduces. Probe: have the model send tool_call with a required argument missing. The bridge refuses it (params must have required property 'b'), the MCP tool never runs, the model answers "15" anyway — and should use MCP add tool to add two numbers goes green on both arms (the same holds for the legacy direct shape on main). If the suite should guard execution, pair the tool_use id with a non-error tool_result the way sdk-mcp-server.test.ts already does.
  2. getInvokedToolName unwraps the target name but not input.arguments. Today the only input assertion is expect(addToolUseBlock.input).toBeDefined(), so nothing breaks; a future assertion on tool arguments would see the envelope { name, arguments } instead. A short comment there would save the next reader a detour.
  3. Optional: getInvokedToolName / findMcpToolUseBlocks would be reusable by other suites if they lived in sdk-typescript/test-helper.ts next to findToolUseBlocks.

Repro

git worktree add --detach /tmp/wt origin/main && cd /tmp/wt && pnpm install   # prepare builds dist/cli.js
# start a local OpenAI-compatible server that answers the suite's prompts with
#   tool_search{query:"select:mcp__test-math-server__add"} then
#   tool_call{name:"mcp__test-math-server__add",arguments:{a,b}}
QWEN_HOME=<clean home with security.auth.selectedType=openai> \
OPENAI_API_KEY=fake-key OPENAI_BASE_URL=<server>/v1 OPENAI_MODEL=fake-model QWEN_MODEL=fake-model \
NO_PROXY=127.0.0.1,localhost QWEN_SANDBOX=false \
pnpm exec vitest run --root ./integration-tests --retry=0 \
  --poolOptions.forks.minForks 1 --poolOptions.forks.maxForks 1 \
  sdk-typescript/mcp-server.test.ts
# arm A (main's mcp-server.test.ts): 7 failed | 2 passed
# arm B (this PR's file, same build):        9 passed
中文说明

本地验证 —— 真实 CLI 构建 + 真实 MCP 服务器 + 脚本化模型

我在本地搭了 E2E 环境,对 sdk-typescript/mcp-server.test.ts 跑了 A/B 两臂。结论与 PR 描述一致:未打补丁的 main 上 #12357 的那 7 个失败逐条复现,打上本 PR 后 9 个用例全绿;同时旧的「直接按 MCP 工具名调用」形态依然全绿。建议合入。

先说一件与合并决策直接相关的事:本 PR 来自 fork,e2e.yml 里的 E2E Tests 作业按设计被跳过(条件是 head.repo.full_name == github.repository)。所以这个 PR 自身的绿色检查并没有覆盖它所修改的那些用例,下面这轮本地运行就是用来补这个缺口的。

装置

被测 CLI 真实的 dist/cli.js,由 pnpm install 的 prepare 从 origin/main cc5ee0a4 构建 —— 正是 globalSetup.ts 交给 TEST_CLI_PATH 的那个路径
MCP 服务器 套件自带的 stdio 数学服务器(createMCPServer),真实 node 子进程
模型 本地 OpenAI 兼容服务,复用仓库自己的 integration-tests/fake-openai-server.ts 协议实现,按提示词脚本化应答
隔离 固定 QWEN_HOME、--retry=0、单 fork、127.0.0.1 不走代理
两臂 共用同一份构建产物,唯一差异是 mcp-server.test.ts(+32 / −7)

只有模型被替换;断言真正读到的东西(SDK transcript、桥接解析、权限链路、MCP 执行)全部是真实产品代码。

结果

上方第一张图:Arm A(main)7 failed / 2 passed,Arm B(本 PR)9 passed。Arm A 的失败文本与 main CI run 35545401904 完全一致 —— 同样的 7 个用例、同样的消息,包括 [ 'tool_search', 'tool_call', …(N) ] to include 'mcp__test-math-server__add' 这种数组形态,以及 expected 0 to be greater than or equal to 2、expected undefined to be defined。

这些失败确实是误报

第二张图是用同一套 CLI 与 SDK 做的探针输出:MCP 工具确实执行了,返回 15 且 is_error=false;main 上唯一的问题是断言只认信封的模型可见名。这与 docs/design/deferred-tool-call-bridge.md("function response ... retains the bridge name")以及 coreToolScheduler.resolveToolCallBridgeRequest 的实现一致:wire 上保留 modelFacingName,只在执行时解析出真实目标。

三种 wire 形态 × 两臂

第三张图给出完整矩阵。第一行是 PR 描述承诺的阳性对照:用 #10410 之前的顶层工具名时两臂都绿,说明补丁是在不丢掉直接形态的前提下新增了对信封形态的识别。

交叉检查

  • 未被本 PR 修改的第 463 行 canUseTool 断言依然成立。 权限回调拿到的是解析后的 MCP 名而不是 tool_call,因为 _schedule 在下游任何环节之前就把信封拆开了。实测依据:Arm B 的 should support multi-turn MCP tools with canUseTool 越过了第 463 行,而 Arm A 根本走不到那里。
  • 补丁文件的门禁: typecheck:integration ✓、eslint --max-warnings 0 ✓、prettier --check ✓。
  • 范围看起来是完整的。 CI run 35545401904 中 sdk 分片是 1 failed | 12 passed,只有这一个文件受害;cli 分片全绿,其中 cli/simple-mcp-server.test.ts 走的是 telemetry(rig.waitForToolCall),本来看到的就是解析后的目标名。
  • 与今天的 main(17b5c940)合并后依然只改这一个文件;没有其它开放 PR 在改它。

非阻塞观察

  1. 这些断言证明的是「发起过调用」,而不是「调用成功」 —— 而且这是既有属性,不是本 PR 引入的。探针:让模型发一个缺少必填参数的 tool_call,桥接会拒绝(params must have required property 'b'),MCP 工具根本没跑,模型照样回答 "15",于是 should use MCP add tool to add two numbers 在两臂都绿(main 上用旧的直接形态同样如此)。如果希望这套 E2E 守住"执行成功",可以像 sdk-mcp-server.test.ts 那样按 tool_use id 配对一条非 error 的 tool_result。
  2. getInvokedToolName 只拆了目标名字,没有拆 input.arguments。当前对 input 的唯一断言是 expect(addToolUseBlock.input).toBeDefined(),所以不受影响;但将来若有人断言工具参数,拿到的会是信封 { name, arguments }。这里加一行注释能省下后来者的一次绕路。
  3. 可选:getInvokedToolName / findMcpToolUseBlocks 若放到 sdk-typescript/test-helper.ts 里紧挨 findToolUseBlocks,其它套件也能复用。

@wenshao
wenshao added this pull request to the merge queue Sep 21, 2026
Merged via the queue into QwenLM:main with commit da300d5 Sep 21, 2026
96 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Main CI failed: E2E Tests — sdk-typescript/mcp-server.test.ts > … > should use MCP add tool to add two numbers (+6 more)

2 participants