Skip to content

Commit 7a32435

Browse files
wenshaowenshao
committed
chore: merge main into managed MCP H1
2 parents e3ecba6 + a765229 commit 7a32435

73 files changed

Lines changed: 7765 additions & 542 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/design/2026-09-27-hosted-shell-tool-turn.md‎

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
# Hosted foreground Shell tool turns
2+
3+
[English](2026-09-27-hosted-shell-tool-turn.md) | [简体中文](2026-09-27-hosted-shell-tool-turn.zh-CN.md)
4+
5+
Status: implemented behind an explicit private profile. Builds on #12831 and
6+
merged O1c #12821.
7+
8+
## Problem and scope
9+
10+
Hosted can run Read, Write and Edit through its saved Workspace, but cannot run
11+
Shell. O1c captures foreground process pipes and admits complete output under a
12+
local Session writer. Hosted instead uses the HTTP Session Store: its ordinary
13+
resource publication is staged in memory until a journal transaction, with a
14+
64 KiB inline limit. Injecting that store into O1c would falsely acknowledge
15+
durability and lose unreferenced pages.
16+
17+
Add an explicitly selected `hosted-workspace-shell/1` profile containing the
18+
existing file tools and foreground `run_shell_command`. Default no-tool and
19+
`hosted-workspace-files/1` behavior remain unchanged. This first bridge supports
20+
the existing same-host process provisioner. It does not add public downloads,
21+
object storage, PTY/background jobs, automatic garbage collection, arbitrary
22+
remote provisioners, or restart/replay of uncertain executions.
23+
24+
## Ownership and transport
25+
26+
The Hosted Session owner opens an ephemeral loopback publisher at
27+
`http://127.0.0.1:<port>/internal/hosted-shell-publisher/v1` with a random capability.
28+
After acquisition, a new private Broker publisher registration installs that
29+
descriptor into the original Runtime Session and returns its binding generation.
30+
The worker accepts only a canonical loopback URL, rejects redirects, and stores
31+
the capability in memory; neither journal references nor model arguments contain
32+
it. No SQL writer token crosses into the worker.
33+
34+
The worker prepares each capture through the private owner listener. Its raw pipe sink forwards bounded write/finish/finalize requests to the owner. Writes and finish are serialized per stream, including pipe callbacks
35+
that overlap when Node resumes a paused stream during process exit. The owner reuses `LocalShellResultCapture`, including its 1 MiB
36+
segments, bounded pages, two-stream backpressure and final manifest. A raw write
37+
reply acknowledges bounded capture buffering; only segment publication replies
38+
acknowledge durable bytes. Lost raw-write replies are not retried. Any uncertain
39+
write or finish irreversibly fails capture, drains the physical process with
40+
bounded memory, and prohibits a complete receipt. Finalization preserves the
41+
physical exit/cancellation result. Model-facing text previews have an additional
42+
8 KiB UTF-8 budget so escaped JSON and receipt metadata fit the existing 64 KiB
43+
history resource limit. Longer previews retain up to 2 KiB from the head and
44+
use the remaining budget for a truncation marker and the tail, cutting only at
45+
UTF-8 boundaries. This preserves trailing failure summaries and exit status.
46+
The tail is from the bounded 64 KiB process buffer; retrieving the true tail of
47+
larger output from durable capture remains follow-up work.
48+
Raw capture bypasses the ordinary temporary-file output
49+
truncator: its preview is not a complete local file. Truncated model text explicitly
50+
reports the execution status and points to retained Session output, without
51+
recommending an inaccessible worker path.
52+
53+
| Route | Owner and checks |
54+
| -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
55+
| Broker publisher registration | Original selected Runtime Session, binding, lease and Workspace ownership |
56+
| Worker publisher registration | Authenticated selected runtime; immutable registration for the named Runtime Session |
57+
| Publisher prepare | Live Session owner; writer and activation rechecked after checkpoint and argument reads, original execution mapping |
58+
| Publisher write/finish/finalize/accept | Live Session owner; capability, activation, registered original execution, runtime/model call mapping and capture identity |
59+
| Durable output publication | Persisted Session; tenant, Workspace, writer token, writer generation and unexpired lease |
60+
| Broker start/status/cancel/ACK | Original execution and saved protocol selection; no v2 fallback |
61+
62+
## Durable output storage
63+
64+
Add a narrow writer-fenced Session Store resource publication operation, using
65+
the existing SQL resource table. It admits only tool-result content (at most
66+
1 MiB), pages (256 KiB), and manifests (64 KiB). Ordinary inline publication and
67+
journal transaction bounds are not increased. A successful response follows
68+
the SQL transaction commit. Stable resource IDs are immutable and idempotent;
69+
reusing an ID with different bytes or metadata conflicts.
70+
71+
A Session-owned sequential segment adapter stores each capture/stream/ordinal
72+
under a deterministic resource ID, computes lengths and SHA-256 itself, and
73+
persists an immutable seal after verifying the exact accepted prefix. It is a
74+
foreground producer adapter, not an implementation of arbitrary out-of-order
75+
O1a uploads. Only stdout/stderr and contiguous ordinals are admitted. Active
76+
upload cursors are bounded in memory and are not resumed after owner loss.
77+
Pages and manifests are immediately durable too, avoiding the staged-resource
78+
closure problem. A new reader can verify a retained manifest, pages, seals and
79+
segment bytes after all producer processes have stopped. Resources are retained
80+
with the Session; ACK does not delete them. Publication without admission never
81+
permits model continuation, and abandoned bytes await future Session cleanup.
82+
83+
## Identity, admission and model history
84+
85+
Keep two digests: the existing exact `payloadJson` byte digest protects deferred
86+
Broker start, while `managedToolDigest(input)` identifies Tool v3 arguments.
87+
Persist the model call ID, unique worker call ID, execution ID, input digest and
88+
the explicit v3 selection. Shell `tool.intent.argsRef` holds the actual input;
89+
its route resource retains the Broker payload. The owner verifies this mapping
90+
against the covered `await_runtime` checkpoint before the first side effect.
91+
Hosted explicitly binds that checkpoint to the current turn, prompt and
92+
committing activation, including after detach/load or a Harness restart. An
93+
unfinished turn cannot be relabeled. Each runtime binding stores the worker
94+
call ID in `invocationBindingId`, while its tool item keeps the model call ID. Preparation
95+
rechecks the writer and activation after asynchronous checkpoint/argument reads;
96+
writer checks also revalidate activation after their await.
97+
98+
Extract O1c admission into a shared Session implementation. The local wrapper
99+
retains its current lease/root guards and automatic checkpoint advancement.
100+
Hosted uses its existing HTTP writer and activation, and advances only after
101+
committing the model-facing result. Full output is re-read in bounded ranges
102+
and checked against the original identity and stream digests before committing
103+
`tool.receipt`. The receipt event factory checks the original activation inside
104+
the authority commit queue, so replacement during output verification cannot
105+
admit an old owner’s result. Complete output receives `committed`; partial/unavailable output
106+
receives `blocked`, preserves its physical result and stops the turn.
107+
108+
The order is: acquire/register, persist assistant and intent/checkpoint, execute,
109+
durable capture, durable receipt, model result, checkpoint resolution, exact ACK,
110+
next inference, result consumption, release. The worker's remote accept returns
111+
the same durable Session receipt; later ACK only replays it. Lost execute replies
112+
use bounded status reconciliation on the original v3 reference. Never reissue a
113+
Shell side effect after unknown status or missing admission. Lost receipt replies
114+
may replay the identical candidate against the recorded receipt. New activations
115+
continue to refuse unresolved old tool turns.
116+
117+
An admitted Shell result must fit the complete serialized history record. If
118+
this invariant fails, retain recovery-blocked ownership without committing an
119+
omission response or acknowledging the result: changing history while reusing
120+
the admitted outcome reference would make them disagree. File-tool results
121+
continue to use their bounded omission response when oversized.
122+
123+
## Lifecycle and failure handling
124+
125+
Runtime warming still overlaps inference. File-only or text-only turns do not
126+
need Shell publication. Shell parameters reject background execution and use the
127+
saved Workspace directory. Invalid Shell arguments return durable function
128+
errors before acquisition or dispatch. If any Shell call is invalid, the entire
129+
batch is refused and each other call explicitly reports that it did not run;
130+
the model can correct the batch within the same turn. Failure to persist that
131+
refusal blocks recovery. An unavailable publisher is rejected before spawn.
132+
Publisher failure, writer loss, unknown execution, receipt/history/ACK failure,
133+
or unconfirmed cancellation retains recovery-blocked ownership. Cancellation
134+
before start remains `not_started` with null capture; after start it requires the
135+
physical process outcome and capture admission. Stop publisher ingress and drain
136+
pending operations before closing the Session writer. A completed turn closes
137+
its private listener without deleting retained output.
138+
139+
## Implementation and validation
140+
141+
Affected layers: core HTTP resource client and shared Shell admission/segment
142+
adapter; CLI Hosted profile, publisher and worker proxy; Java Session Store,
143+
Broker original-execution routing and Workspace transport. No ordinary daemon
144+
route is added or rerouted.
145+
146+
Focused tests cover immutable publication and stale writers, segment prefix and
147+
seal integrity, cross-Session/activation identity, lost raw replies, admission
148+
before history, exact ACK, cancellation and unchanged file-only/no-tool behavior.
149+
They also pin head/tail UTF-8 previews, refused-batch recovery, publisher bearer
150+
authentication, listener closure after completed and failed turns, and the
151+
admitted-result history bound with worst-case JSON escaping.
152+
A real-process test runs the packaged Harness and worker through production Java
153+
Broker and SQL Store, checks the selected Workspace with a Harness decoy, writes
154+
100 MiB with independent stdout/stderr digests and tails, then re-reads retained
155+
output after producer shutdown. Faults must leave side effects at most once and
156+
must not trigger the next model request. Build, typecheck, bundle and two clean
157+
full-diff audits precede review. The global CLI baseline is recorded separately;
158+
absence of its private Hosted routes is not reported as a passing feature test.
159+
160+
Acceptance requires actual cross-process complete capture and durable admission,
161+
not merely a v3 HTTP response. There are no unresolved product choices; public
162+
artifact access, distributed storage and recovery remain follow-up work.
163+
Per-call/Session storage quotas and reducing writer-lease renewal frequency also
164+
remain follow-ups; this change does not alter durable publication or fencing.
165+
166+
Local validation uses macOS, Node.js 22 and Java 21 with real processes and H2 in
167+
MySQL mode. The six-Workspace fixture verifies complete 100 MiB output, SQL
168+
publication failure, lost raw/start replies, cancellation, and a fresh reader
169+
after producer shutdown. It does not validate a real MySQL deployment, Windows,
170+
Linux, or a real model provider.

0 commit comments

Comments
 (0)