Summary
The "Deferred Tools" listing (MCP tools — always shouldDefer=true, reachable only via ToolSearch) is baked into the cached system prompt. Whenever the deferred set changes — MCP progressive discovery completing after startChat(), or the model revealing a tool via ToolSearch — setTools() / startChat() call setSystemInstruction() to rewrite the whole system instruction. That mutates the cached prefix, invalidating the prompt cache for the rest of the conversation.
Impact
- Every late MCP discovery and every
ToolSearch reveal drops the prompt-cache hit → higher latency + cost (cache_read → full input) for the remainder of the session.
- Affects both Anthropic (explicit
cache_control on the system block) and Gemini (implicit prefix caching).
- Non-interactive
--prompt runs are hit hardest (that's where progressive MCP discovery fires).
Root cause
getMainSessionSystemInstruction() embeds the deferred-tools section (getCoreSystemPrompt(..., deferredTools)), and setTools() rewrites the system instruction to refresh it on every registry change.
Proposed fix
Move the deferred-tools listing out of the cached prefix into a per-turn <system-reminder> in the message tail, rebuilt from live registry state each UserQuery/Cron turn. The system prefix then stays byte-stable across MCP discovery / tool reveals; deferred tools still surface, and remain visible across turns because the reminder persists in chat history.
Files involved:
packages/core/src/core/client.ts
packages/core/src/core/prompts.ts
A fix with tests is ready; happy to open a PR.
Summary
The "Deferred Tools" listing (MCP tools — always
shouldDefer=true, reachable only viaToolSearch) is baked into the cached system prompt. Whenever the deferred set changes — MCP progressive discovery completing afterstartChat(), or the model revealing a tool viaToolSearch—setTools()/startChat()callsetSystemInstruction()to rewrite the whole system instruction. That mutates the cached prefix, invalidating the prompt cache for the rest of the conversation.Impact
ToolSearchreveal drops the prompt-cache hit → higher latency + cost (cache_read → full input) for the remainder of the session.cache_controlon the system block) and Gemini (implicit prefix caching).--promptruns are hit hardest (that's where progressive MCP discovery fires).Root cause
getMainSessionSystemInstruction()embeds the deferred-tools section (getCoreSystemPrompt(..., deferredTools)), andsetTools()rewrites the system instruction to refresh it on every registry change.Proposed fix
Move the deferred-tools listing out of the cached prefix into a per-turn
<system-reminder>in the message tail, rebuilt from live registry state eachUserQuery/Cronturn. The system prefix then stays byte-stable across MCP discovery / tool reveals; deferred tools still surface, and remain visible across turns because the reminder persists in chat history.Files involved:
packages/core/src/core/client.tspackages/core/src/core/prompts.tsA fix with tests is ready; happy to open a PR.