Repository navigation
feat(chrome): add Qwen WebBridge direct browser control - #8707
yiliang114 wants to merge 44 commits into
Conversation
E2E test reportEnvironment: macOS, Node.js 22 workspace runtime, Microsoft Edge 150 (Chromium) with a disposable browser profile.
The real-browser run exercised all 17 actions through the packaged MV3 extension and |
Code Coverage Summary
CLI Package - Full Text ReportCore Package - Full Text ReportFor detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run. |
github-actions
Bot
added
the
review/self-reported
qqqys
left a comment
qqqys
left a comment
There was a problem hiding this comment.
[Critical] 本 PR 导致 Test (ubuntu-latest, Node 22.x) 红，阻塞合并。
位置
packages/chrome-extension/public/manifest.json:8-16——permissions从["tabs","storage","debugger","alarms","sidePanel"]扩为 7 项，新增tabGroups、windows。packages/chrome-extension/src/sidepanel/sidepanel-assets.test.ts:56—— 守卫用例requests only permissions used by the extension仍然全等断言旧的 5 项清单，本 PR 未同步修改该文件。
触发条件
npm run test:ci -w @qwen-code/chrome-bridge（CI 中 Test (ubuntu-latest, Node 22.x) 的 chrome-extension 分片）。head 0c80bd5 实测：
FAIL src/sidepanel/sidepanel-assets.test.ts > requests only permissions used by the extension
AssertionError: expected [ Array(7) ] to deeply equal [ 3.8., .storage., .debugger., …(2) ]
+ "tabGroups"
+ "windows"
（run 31231026337，Test Files 1 failed | 9 passed，其余 workspace 全绿——唯一失败点就是这一条。）
影响
required check 持续红，PR 无法合并；同时该用例是扩展权限白名单的评审守卫，红着的时候后续任何权限增删都失去把关。
修复方向
chrome.tabGroups.* / chrome.windows.getLastFocused 在本 PR 中确有使用（web-bridge-actions.ts 的分组标题与窗口聚焦路径），所以是断言该更新、而不是权限该回退。把 sidepanel-assets.test.ts:56 的期望数组按 manifest 的新顺序补上 tabGroups、windows 即可——建议放在同一 commit 里，让这次权限扩张在 diff 中显式留痕。
|
Addressed on the current head 中文说明当前 head |
qqqys
left a comment
qqqys
left a comment
There was a problem hiding this comment.
上一条 CHANGES_REQUESTED 的问题（manifest 新增 tabGroups 但 sidepanel-assets.test.ts 的权限白名单未同步）在当前 head 6304334 已修复：packages/chrome-extension/src/sidepanel/sidepanel-assets.test.ts:56-63 的断言现在是 6 项，与 public/manifest.json 的 permissions 完全一致。
但当前 head 上出现了另一个由本 PR 引入的 CI 失败，Test (ubuntu-latest, Node 22.x) 仍然红，仍然阻塞合并。
位置
packages/cli/src/serve/capabilities.ts:418—— 本 PR 在SERVE_CAPABILITY_REGISTRY新增webbridge: { since: 'v1' }（main上没有这一项）。integration-tests/cli/qwen-serve-routes.test.ts:301-405—— 该用例用toEqual全量断言caps.features，数组最后一项仍是第 404 行的'voice_transcribe'，没有'webbridge'。这个文件不在本 PR 的改动清单里。
触发条件
qwen serve — capabilities envelope > advertises all baseline capabilities 拉取 /capabilities 后与硬编码清单做全等比较；注册表多了一项，toEqual 即失败。
影响
集成测试红，阻塞合并：
FAIL cli/qwen-serve-routes.test.ts > qwen serve — capabilities envelope > advertises all baseline capabilities
AssertionError: expected [ 'health', 'daemon_status', …(102) ] to deeply equal [ 'health', 'daemon_status', …(101) ]
@@ -100,6 +100,7 @@
"voice_transcribe",
+ "webbridge",
Tests 1 failed | 130 passed (131)
当前 head 6304334 的 run 31238156058 里全仓库仅此一个失败用例，main 的 capabilities.ts 无 webbridge，因此该失败只来自本 PR。
修复方向
在 integration-tests/cli/qwen-serve-routes.test.ts:404 的 'voice_transcribe' 之后补一行 'webbridge',，顺序与 SERVE_CAPABILITY_REGISTRY 中的声明顺序保持一致（该用例依赖顺序）。docs/developers/daemon/11-capabilities-versioning.md 本 PR 已同步，无需再动。
|
@qwen-code /review |
| _Qwen Code review request accepted. Review is queued in [workflow run](https://github.com/QwenLM/qwen-code/actions/runs/31243241879)._ |
|
Refreshed this branch onto the latest The failing CI evidence was: received 104 baseline features versus 103 expected, with only 中文说明已在 失败证据是实际收到 104 个 baseline feature、预期 103 个，唯一新增项为 |
原问题已在 5828a0b 修复：integration-tests/cli/qwen-serve-routes.test.ts:406 已补上 'webbridge'；该 tag 未登记在 capabilities.ts 的 CONDITIONAL_SERVE_FEATURES 中，属无条件广告，且在 registry 中位于 capabilities.ts:419（realtime_voice 之后、末位），故断言顺序与 getAdvertisedServeFeatures 的输出一致；packages/cli/src/serve/server.test.ts:504 / :563 / :623 三处全量清单同样已同步。上一条 manifest 权限白名单断言的问题已在 6304334 修复。解除本条 CHANGES_REQUESTED，不代表对本 PR 的整体批准。
|
Changed: after the ordinary base refresh, exact-head CI exposed a PR-caused process-environment guard failure. The WebBridge token now reads from the daemon handle frozen environment snapshot instead of mutable global Verified: the focused process-env guard passes 3/3, plus ESLint, build followed by typecheck, and diff check. The earlier pre-build typecheck was blocked only by missing workspace dist artifacts and passed after the required build order. Pending: CI and automatic review for the new exact head are running. 中文摘要普通 base refresh 后，exact-head CI 暴露出 PR 引起的 process-env guard 失败。WebBridge token 现在从该 daemon handle 的冻结环境快照读取，不再直接读可变全局 |
|
Closeout for the current head
中文摘要
|
|
No code changed. The exact-head automatic review reached its 360-minute timeout without findings. A failed-only rerun was requested; attempt 2 is now in progress. 中文摘要未修改代码。exact-head 自动 review 在没有 finding 的情况下达到 360 分钟超时；已请求 failed-only rerun，第 2 次 attempt 正在运行。 |
|
Status update from resolve-pr-comments: Changed: fixed same-URL reload Verified: focused Chrome extension WebBridge/CDP tests, focused WebBridgeService test, ESLint, Prettier, and Intentionally not changed: network capture retention after stop/detach, WebBridge route predicate extraction, table/grid snapshot refs, and per-PR integration job wiring were resolved as outside this closeout scope. Pending: CI and automatic review on head 87335b9. |
|
Status update from resolve-pr-comments: Changed: fixed the raw CDP tab-switch detach race, element screenshot scroll offsets, Verified: Intentionally not changed: service-worker handshake harness, listener mock hardening, and download URL policy were left out to avoid expanding this PR. Pending: CI and automatic review on head 4157a19. |
- docs: move `webbridge` out of the conditional-serve-features table; it is an always-on baseline capability (fixes capabilities-docs-contract) - skill: drop bare run_shell_command/write_file allowedTools so skill activation cannot grant session-wide auto-allow rules - cdp-bridge: the 55s action timeout now detaches only the tabs the timed-out action used instead of the global teardown, preserving other sessions' attachments and network captures - actions: snapshot refs fail closed — store the top-frame loaderId and continue ref numbering across re-snapshots so stale refs error instead of resolving to shifted/new-document elements - service: enforce cross-session tab guards on the long-lived daemon (close_tab 409, close_session close-set filter) so they survive MV3 service-worker restarts - actions: key_type focus check walks iframes/shadow roots; find_tab matches about:blank and other non-http(s) scheme URLs; network list/detail fall back to the session's capture after the current tab changes; send_keys bounds input size; idle-tab release no longer replaces a successful action result - service: prune screenshot/PDF artifact tmpdirs beyond a cap - tests: cover all of the above plus direct-subscriber event fan-out
doudouOUC
left a comment
doudouOUC
left a comment
There was a problem hiding this comment.
Not explored to full depth (tool budget reached): "This PR adds a new \"Qwen WebBridge\" feature — a Chrome…": None — I read the full file, enumerated every early return, and traced every state transition across the exported API surface. The config-fields dimension was a…; chunk 6: I was able to read both the full test file and the full source file within budget. No unfinished checks.; "This PR adds a new \"Qwen WebBridge\" feature. The review…": none. All layers within the execution model were walked.; chunk 17: None. I completed my analysis within the tool budget.; chunk 15: None. I completed my full analysis within the allocation..
中文说明
未探索到全部深度（达到工具调用预算）："This PR adds a new \"Qwen WebBridge\" feature — a Chrome…"：None — I read the full file, enumerated every early return, and traced every state transition across the exported API surface. The config-fields dimension was a…；chunk 6：I was able to read both the full test file and the full source file within budget. No unfinished checks.；"This PR adds a new \"Qwen WebBridge\" feature. The review…"：none. All layers within the execution model were walked.；chunk 17：None. I completed my analysis within the tool budget.；chunk 15：None. I completed my full analysis within the allocation.。
— deepseek-v4-flash via Qwen Code /review (v0.21.11)
|
|
||
| function checkedValue(action: string, result: JsonRecord): unknown { | ||
| throwOnCdpException(action, result); | ||
| const value = record(record(result['result'])['value']); |
There was a problem hiding this comment.
[Suggestion] checkedValue silently converts non-object CDP result values to empty objects — Failure scenario: The chain record(record(result['result'])['value']) calls record() on the result value. record() returns {} for any non-object value (string, number, boolean). Currently all callers pass Runtime.evaluate results whose expressions return object literals, so this is not a runtime bug. However, a future caller that expects a primitive value would silently get {} instead, causing a hard-to-debug downstream failure.
| const value = record(record(result['result'])['value']); | |
| function checkedValue(action: string, result: JsonRecord): unknown { | |
| throwOnCdpException(action, result); | |
| const runtimeResult = result['result']; | |
| const value = isRecord(runtimeResult) ? runtimeResult['value'] : undefined; | |
| return value; | |
| } |
— deepseek-v4-flash via Qwen Code /review (v0.21.11)
There was a problem hiding this comment.
Valid observation, but no current runtime bug: every present caller passes Runtime.evaluate results whose expressions return object literals, so the {} coercion for primitives is unreachable today. Changing checkedValue's return semantics is better done together with the first caller that actually needs a primitive (which would also pin the behavior with a test) — leaving this thread open as a tracked follow-up rather than fixing speculatively.
|
Closeout from resolve-pr-comments automation: Changed: no product code. The failed Ubuntu test log ended with runner termination after normal resource samples, so I requested a failed-job rerun. Verified: rerun request was accepted by GitHub. Pending: rerun CI and automatic review on the current head. |
…igh-water - withCdpTab: attach a guard catch to the operation promise so the timeout teardown cancelling pending CDP commands cannot surface an unhandled rejection after the race already settled on the timeout - detachDirectOperationTabs: cancel pending commands on the empty-tab early return too — otherwise their promises hang forever and the pending closures leak - snapshot refs: number from the highest ref number EVER issued for the session (high-water mark), not the previous snapshot's count, which re-issued numbers from two snapshots back and let stale held refs collide with fresh ones inside the same document (loaderId guard cannot fire there); regression test covers the 5->1->3 shape
- a second `network start` for the same session+tab keeps the running capture instead of wiping already-recorded requests (a failed re-start no longer loses everything captured so far) - `network stop` tolerates a per-tab attach failure and keeps disabling the remaining captures instead of aborting the whole loop (leaked entries no longer accumulate against the capture limit) - `network list`/`detail` merge every capture owned by the session across tabs instead of reading only the first one, matching the write path (`stop` already iterated all of them) - a capture destroyed by a debugger detach is surfaced instead of being silently discarded: list/detail/stop report 'capture was invalidated by a debugger detach', and a fresh start clears the mark - regression tests cover all four paths
Same trailing-newline gotcha the daemon-token path already handles: `export QWEN_WEBBRIDGE_TOKEN=$(cat token.txt)` keeps the file's final newline in the value, and a newline-bearing token can never match an HTTP Authorization header — every WebBridge request would 401 with no diagnostic. Trim once at boot; empty-after-trim falls back to a fresh token. The deferred-runtime route test now pins the trim by exporting a token with a trailing newline and authenticating with the trimmed value.
- send_keys: shift+<digit> now dispatches the US-layout shifted character (shift+1 -> '!') instead of the bare digit with Shift held, a combination no physical keyboard produces - find_tab(active:true) no longer demotes a session-created tab to borrowed: the demotion removed the tab from the close set permanently (close_tab rejects borrowed tabs, close_session filters them out) - close_session hands ownership of a borrowed tab to the borrower before the owning session is deleted, so the tab stays closable instead of being orphaned in the user's browser - regression tests cover all three paths
…itle - mouse_click and element screenshot now scroll with behavior:'instant': the default 'auto' follows the page's scroll-behavior CSS, so on smooth-scroll pages the geometry was sampled while the scroll animation was still in flight and clicks landed at stale coordinates while still reporting success - save_as_pdf caps page-controlled pageTitle at 1 KiB: document.title is uncapped page input riding the unchunked metadata frame and could exceed the daemon's WS maxPayload, dropping the bridge for every session - regression tests pin the instant-scroll declaration and the title cap
…-tab retry - find_tab: schemeless requests are parsed as host[:port][/path] so the port and path participate in the match — 'localhost:9222' no longer matches a :3000 tab, and 'host:port/path' is no longer misparsed as an opaque-scheme URL that can never match; bare '*' matches any tab; wildcard hosts, about:/data: href matching, and full-URL exact matching are preserved - stale-tab retry: drop the stale tab only after the recovery succeeds — mutating first left an emptied session entry behind when the retry itself threw, leaking it against MAX_SESSIONS until daemon restart - regression tests pin the port/path matching and the failed-retry state
doudouOUC
left a comment
doudouOUC
left a comment
There was a problem hiding this comment.
Not explored to full depth (tool budget reached): "This PR adds Qwen WebBridge. Already confirmed: 3…": None — I examined all files in my chunk, walked the permission model (auth regex, two-token routing, /webbridge/status exemption), the CDP tunnel acceptance t…; chunk 11: None. The full territory was examined.; chunk 3: None. I completed the full trace within the budget.; chunk 8: 无——我在到达预算上限之前完成了审查。.
[Critical] R1-1: close_session with close_tabs=false releases CDP for tabs shared with other sessions — Session A and B share tab 17. Session A calls close_session({close_tabs: false}). shared=true, captured=false, releaseCdpTab(17) is called. Session B's CDP operations on tab 17 fail with detached-debugger error. Fix: gate releaseCdpTab on !shared && !captured.
[Critical] R1-2: close_session ownership transfer not rolled back on failure — Session A owns tab 1, B borrows it. close_session ownership transfer runs before registry.call. On failure, tab 1 is in both ownedTabIds. Neither session can close it. Orphaned until daemon restart. Fix: move ownership transfer after the extension call succeeds.
[Critical] R1-3: directTabIds leaked when withCdpTab's operation throws — attachTab succeeds, directTabIds.add(tabId), then operation throws. No try/finally cleans up. Raw CDP tunnel permanently disabled. Fix: wrap operation in try/finally that deletes from directTabIds.
[Critical] R1-15: releaseIdleTabs releases CDP for shared tabs, called after every action — Same shape as R1-1 but in releaseIdleTabs, called after every WebBridge action. Session B's CDP operations on shared tab fail after every Session A action. Fix: add tabUsedByAnotherSession guard in releaseIdleTabs.
[Critical] R1-16: cdpCall WebSocket in E2E test does not handle close events — cdpCall registers ws.once('error') and ws.once('open') but not a 'close' handler. If the WebSocket connects and then closes without a message, the promise never resolves or rejects. E2E test hangs indefinitely. Fix: add ws.once('close', () => rejectCall(...)).
中文说明
未探索到全部深度（达到工具调用预算）："This PR adds Qwen WebBridge. Already confirmed: 3…"：None — I examined all files in my chunk, walked the permission model (auth regex, two-token routing, /webbridge/status exemption), the CDP tunnel acceptance t…；chunk 11：None. The full territory was examined.；chunk 3：None. I completed the full trace within the budget.；chunk 8：无——我在到达预算上限之前完成了审查。。
[Critical] R1-1: close_session with close_tabs=false releases CDP for tabs shared with other sessions — Session A and B share tab 17. Session A calls close_session({close_tabs: false}). shared=true, captured=false, releaseCdpTab(17) is called. Session B's CDP operations on tab 17 fail with detached-debugger error. Fix: gate releaseCdpTab on !shared && !captured.
[Critical] R1-2: close_session ownership transfer not rolled back on failure — Session A owns tab 1, B borrows it. close_session ownership transfer runs before registry.call. On failure, tab 1 is in both ownedTabIds. Neither session can close it. Orphaned until daemon restart. Fix: move ownership transfer after the extension call succeeds.
[Critical] R1-3: directTabIds leaked when withCdpTab's operation throws — attachTab succeeds, directTabIds.add(tabId), then operation throws. No try/finally cleans up. Raw CDP tunnel permanently disabled. Fix: wrap operation in try/finally that deletes from directTabIds.
[Critical] R1-15: releaseIdleTabs releases CDP for shared tabs, called after every action — Same shape as R1-1 but in releaseIdleTabs, called after every WebBridge action. Session B's CDP operations on shared tab fail after every Session A action. Fix: add tabUsedByAnotherSession guard in releaseIdleTabs.
[Critical] R1-16: cdpCall WebSocket in E2E test does not handle close events — cdpCall registers ws.once('error') and ws.once('open') but not a 'close' handler. If the WebSocket connects and then closes without a message, the promise never resolves or rejects. E2E test hangs indefinitely. Fix: add ws.once('close', () => rejectCall(...)).
— deepseek-v4-flash via Qwen Code /review (v0.21.11)
# Conflicts: # docs/developers/qwen-serve-protocol.md # packages/cli/src/serve/server.ts
# Conflicts: # packages/cli/src/serve/run-qwen-serve.test.ts # packages/cli/src/serve/run-qwen-serve.ts
…idge # Conflicts: # docs/developers/qwen-serve-protocol.md # packages/cli/src/serve/capabilities.ts # packages/cli/src/serve/run-qwen-serve.ts # packages/cli/src/serve/server.test.ts
…idge # Conflicts: # integration-tests/cli/qwen-serve-routes.test.ts
Only conflict was the daemon index capability-count row: this branch and main each added one registered tag (`webbridge` here), and main also added one conditional tag. Counted from the merged `capabilities.ts` rather than by arithmetic, since `capabilities-docs-contract.test.ts` asserts the row against `SERVE_CAPABILITY_REGISTRY` and `CONDITIONAL_SERVE_FEATURES`: 151 registered, 44 conditional. Every conditional key in the code is still documented in `qwen-serve-protocol.md`. Claude-Session: https://claude.ai/code/session_01AWWgJEqafyAT1Mc75T8N7h
|
Closing this as parked, which is a resourcing decision, not a quality judgement. Context: the maintainer plan recorded on 2026-08-02 removes Chrome work from the active mainline ("stop Chrome / generic reliability as the near-term mainline; do not start new Chrome work"), and this PR has seen no activity since 2026-09-03 while conflicting with Nothing is thrown away: the branch |
5 participants
What this PR does
This adds Qwen WebBridge, a direct browser-control path from
qwen serveto the Qwen Chrome extension and the user's real Chromium profile. It exposes Kimi WebBridge-compatible/commandand/statusendpoints, implements the full 17-action surface, tracks task-scoped owned and borrowed tabs, persists screenshots and PDFs locally, and bundles an Agent Skill describing the workflow.The existing raw
/cdptunnel remains optional. Direct WebBridge commands and raw CDP ownership share the extension debugger safely, reject conflicting operations, preserve direct attachments when the raw tunnel disconnects, and isolate bounded network captures by session. Direct CDP-backed actions use a 55-second extension deadline: pending commands are invalidated, debugger ownership is detached, and only then is the serial lock released with timeout semantics. The side panel reads live extension readiness from a daemon-authenticated, read-only/webbridge/statuspath;/statusand/commandremain restricted to the route-scoped extension token.Why it's needed
Qwen Code could already reach Chrome through an external Chrome DevTools MCP adapter, but it had no first-party Agent-to-extension command contract. This change removes that required MCP hop for browser tasks while retaining the user's existing tabs, profile, and login state.
Reviewer Test Plan
How to verify
qwen serve --allow-origin chrome-extension://<extension-id>.GET /statusreportsextension_connected: true.navigatecommand with a new session, then usesnapshot, interaction, network, screenshot/PDF, upload, tab, and lifecycle actions; confirm the created tabs stay in one task group andclose_sessioncloses owned tabs without closing a borrowed foreground tab.QWEN-WEBBRIDGE-17-ACTIONS: PASSandQWEN-WEBBRIDGE-REAL-CHROME: PASS.Evidence (Before & After)
qwen servehad no direct/commandbrowser API; browser automation required the optional external Chrome DevTools MCP adapter.Tested on
Environment (optional)
macOS with Microsoft Edge 150 in a disposable profile; Node.js 22 workspace runtime.
Risk & Scope
/cdpand external MCP adapter path remains available. Focused deadline, late-callback, timeout-mapping, status-auth, and side-panel regressions pass on the current head, along with the full repository build and typecheck.Linked Issues
Fixes #8699
中文说明
本 PR 做了什么
本 PR 新增 Qwen WebBridge，建立从
qwen serve到 Qwen Chrome 扩展、再到用户真实 Chromium profile 的直接浏览器控制链路。它提供与 Kimi WebBridge 兼容的/command和/status接口，实现完整的 17 个动作，维护任务级自有/借用标签页状态，将截图和 PDF 落盘，并内置描述操作流程的 Agent Skill。现有 raw
/cdp隧道仍然是可选能力。直接 WebBridge 命令和 raw CDP 安全复用扩展调试器：冲突操作会被拒绝，raw 隧道断开不会误释放仍被直接链路使用的 attachment，网络抓包有上限并按 session 隔离。CDP-backed 直接动作使用 55 秒扩展侧截止：先让 pending command 失效并 detach debugger 所有权，再以 timeout 语义释放串行锁。侧边栏通过 daemon-auth 的只读/webbridge/status读取扩展连接状态；/status与/command仍只接受 route-scoped 扩展 token。为什么需要
Qwen Code 原本可以通过外部 Chrome DevTools MCP adapter 控制 Chrome，但缺少第一方的 Agent 到扩展命令协议。本改动让浏览器任务不再必须经过 MCP，同时继续复用用户已有的标签页、profile 和登录态。
Reviewer 验证计划
如何验证
qwen serve --allow-origin chrome-extension://<extension-id>。GET /status返回extension_connected: true。navigate，依次验证snapshot、交互、网络、截图/PDF、上传、标签页和生命周期动作；确认新建页面位于同一任务分组，且close_session只关闭自有标签页、不关闭借用的前台标签页。QWEN-WEBBRIDGE-17-ACTIONS: PASS和QWEN-WEBBRIDGE-REAL-CHROME: PASS。前后对比证据
qwen serve没有直接的/command浏览器 API，浏览器自动化依赖可选的外部 Chrome DevTools MCP adapter。测试平台
环境（可选）
macOS，使用 Microsoft Edge 150 和一次性 profile；Node.js 22 workspace runtime。
风险与范围
/cdp和外部 MCP adapter 链路继续可用。当前 head 上 deadline、late callback、timeout 映射、status 鉴权与侧边栏聚焦回归均通过，全仓 build 与 typecheck 也通过。关联 Issue
Fixes #8699