tracking(core): non-conversation context token governance #12028

Non-conversation context — the system prompt, built-in tool schemas, context (QWEN.md) files and the skill listing — is sent and paid for on every request. On a large-context model this block easily dwarfs the conversation itself without anyone noticing, because it surfaces as a small percentage of the window.

This issue tracks the individual findings; each gets its own issue below.

Evidence

A /context detail sample from an interactive session on a 1M-context model, taken one turn in:

98.7% of that request's input was prefix, while the window read as only 6.5% used. That gap is the theme of this issue: the health metric everyone looks at is a ratio against the window, and the costs are absolute.

The built-in tool block, itemised:

plus create_sub_session 409, tool_search 375, get_goal 365, enter_plan_mode 343, grep_search 301, list_agents 292, notebook_edit 291, zoom_image 279, glob 266, record_source 220, read_mcp_resource 191, cron_delete 116, task_stop 102, cron_list 80.

Two things stand out. The largest single entry, workflow at 3,829 tokens, is a feature that is off by default upstream (isWorkflowsEnabled, packages/core/src/config/config.ts:8637) — enabling one experimental feature costs more than the six file tools combined. And the seven tools this session actually needed to do file work — run_shell_command, edit, read_file, write_file, grep_search, glob, tool_search — total 4,080 tokens, 19% of the block.

The same mis-parameterization shows up twice in the code, and both instances get less protective as windows grow:

• tools.toolSearch.threshold — 10% of the window. At 1M that is a 100,000-token preload budget against a ~4.2k deferred pool, so nothing ever stays deferred.
• MEMORY_CONTEXT_WARNING_RATIO — 15% of the window. At 1M the always-on context warning fires at 150,000 tokens, so 15,400 tokens of context files draws no warning at all.

Findings

1. Percentage-of-window budgets scale the wrong way. Both knobs above protect against costs that scale with the deferred set or the context files, not with the window.
2. Every active extension's context file is unconditionally resident. Nine extension QWEN.md files totalled 9,989 tokens — 21% of all non-conversation context — concatenated into every request regardless of relevance. There is no path-gating, no per-extension budget, no truncation. The same extensions already ship skills, which cost 100–200 tokens each in the listing and load their bodies on demand. (Correction: the same sample measured 84 skills at 4,620 tokens, about 55 each.)
3. The system prompt is ~46% tool-specific but is assembled independently of which tools are resident. ## Using Your Tools (4,031 chars) is ~66% tool policy; # Examples (3,283 chars) is entirely [tool_call: …] transcripts. Deferring or denying a tool leaves its policy and examples in the prompt, which both wastes tokens and points the model at tools it was not given.
4. /context accounting does not close. Categories sum to 47,348 against a reported 65,267. Token governance cannot be driven from numbers that do not add up. (Correction 2026-09-17: the 1.378 ratio is not CJK undercounting — the display path uses the CJK-aware estimator for every category. The gap is structural: messages is derived as total − apiCachedTokens, and the startup prelude belongs to no category. See fix(cli): /context category breakdown does not close (skills listing unattributed, messages derived from a cache subtraction, startup prelude uncounted) #12033.)
5. tools.disabled does not always stop the schema being sent (tools.disabled removes zoom_image from the registry but its schema is still sent to the model #11814).

Status (2026-10-07)

The 46,734-token sample above and the 2026-09-29 first-main comparison at fa4a4c92ce (17,794 → 13,956) are historical measurements with different configurations, not refreshed baselines. The direct list below has 19 issues: 12 completed, 1 closed as not planned, and 6 open. Five open items are on the main path; #12235 is a P3 follow-up on hold.

No implementation PR under this tracker is open any more. Everything that was open on 2026-10-04 has merged:

v0.25.0 does not contain #13033 or #12901; no stable release does yet.

Open main-path items and what each one is waiting on:

• feat(ci): the token work has no recall or task-success gate — teach the existing benchmark to compare two configurations #12333(b): the only gate that can close the main path. The external benchmark runner has to consume a settings overlay so that one commit can run paired configurations. The internal runner repository was located on 2026-10-04, but the current account is denied source access. Blocked on access, not on code in this repository.
• feat(core): the eager tool surface is a hand-maintained static list — let something choose it, without invalidating the prompt prefix #12326: the eager-set decision. feat(core): defer agent and goal declarations by default #13033 already narrowed the default, so this is now a check of a shipped default rather than a pre-merge gate. Waits on feat(ci): the token work has no recall or task-success gate — teach the existing benchmark to compare two configurations #12333(b).
• fix(core): deferred tools lose their "use me instead of X" rules — guidance line gated away, reminder keeps only description line 1 #12702: the rules are restored (fix(core): deferred-tool selection rules in the reminder line and a reviewed-schema gate for tool_call #13020); three bounded real-model tasks on current main chose correctly. Representative model-choice acceptance remains.
• Track structured Auto Memory rollout readiness on main #12947: foreground main-model input −11.65% measured earlier; selector ablation (perf(memory): skip the selector after a delivered unique strong recall hit #13003, now merged default-off) and P95 tail validation remain.
• qwen code causes prompt reprocessing on memory write #11550: a client-side fix is up as fix(memory): preserve the prompt prefix when memory indexes change #13521 (changes requested). Acceptance needs warm-KV/prefill behavior on the original llama.cpp backend.

Related, tracked separately: #12889 (provider acceptance, above), #13003 (opt-in experiment, above), #13004 (window-bounded phase 1 up as #13571, default-off; process exit and replay deferred).

Acceptance order is unchanged: #12333(b) runner consumes a settings overlay → matched recall/task-success and full-cost runs → #12326 decision (now including whether #13033's default holds) → refreshed baseline → default-enable decisions. Memory experiments remain off by default. Deployment-side savings additionally require upgrading to a release that contains these PRs and configuring tools.eager; neither is a code change in this repository.

Sub-issues

• tools.disabled removes zoom_image from the registry but its schema is still sent to the model #11814 — tools.disabled removes zoom_image from the registry but its schema is still sent — closed; fix(core): hide unavailable image zoom guidance #12271 covers the only evidenced path, and no raw schema leak was reproduced
• feat(extensions): let an extension ship saved workflows, so its context file can name one instead of spelling out the procedure #11631 — let an extension ship saved workflows so its context file can name one — available in v0.24.0 via feat(extensions): ship dynamic workflows from extensions #11805, feat(core): run saved workflows by name and pin workflow grants to script content #11943, and feat(workflows): let the model invoke extension workflows that declare whenToUse #11957
• qwen code causes prompt reprocessing on memory write #11550 — memory write causes prompt reprocessing — still reproducible on prefix-matching backends; client-side fix proposed in fix(memory): preserve the prompt prefix when memory indexes change #13521 (changes requested) under Split System Prompt & System Reminder to improve context management and avoid hallucination #2653/Better Prompt Caching #8277; original llama.cpp backend acceptance pending
• Deferred review findings from PR #10410: feat(core): preserve prompt cache for deferred tools #11321 — preserve prompt cache for deferred tools — closed 2026-10-07: both residual items resolved on main (fix(core): keep the deferred-tool bridge halves on the same tool #12539 shared name resolution and fingerprint check; Deferred-tool bridge: a hidden tool whose schema left context via /compress is still invocable by name (remaining half of #11321) #12569 review gate via fix(core): deferred-tool selection rules in the reminder line and a reviewed-schema gate for tool_call #13020)
• Deferred-tool bridge: a hidden tool whose schema left context via /compress is still invocable by name (remaining half of #11321) #12569 — a hidden tool whose schema left context via /compress is still invocable by name — closed by merged fix(core): deferred-tool selection rules in the reminder line and a reviewed-schema gate for tool_call #13020; the broader residual items in Deferred review findings from PR #10410: feat(core): preserve prompt cache for deferred tools #11321 remain separate
• Percentage-of-context-window budgets scale the wrong way: ToolSearch preload never engages, and the always-on context warning never fires, on large windows #12029 — percentage-of-context-window budgets scale the wrong way (ToolSearch preload; always-on context warning) — warning ceiling in feat(core,docs): add conditional extension rules and cap context warnings #12034; preload half made moot by feat(core): preserve prompt cache for deferred tools #10410
• feat(extensions): an extension's context file is unconditionally resident, with no path gating, budget, or attribution #12030 — an extension's context file is unconditionally resident, with no path gating or budget — implemented by feat(core,docs): add conditional extension rules and cap context warnings #12034
• feat(core): assemble the tool-policy and example sections of the system prompt from the resident tool set #12032 — assemble the tool-policy and example sections of the system prompt from the resident tool set — implemented by feat(core): assemble the system prompt from the resident tool set #12145
• fix(core): deferred tools lose their "use me instead of X" rules — guidance line gated away, reminder keeps only description line 1 #12702 — deferred tools lose their "use me instead of X" rules before reveal (a side effect of feat(core): assemble the tool-policy and example sections of the system prompt from the resident tool set #12032's gating) — both compact comparison rules landed in fix(core): deferred-tool selection rules in the reminder line and a reviewed-schema gate for tool_call #13020; the report on main at 728c13de did not reproduce the original information loss. The issue is still open; model-choice acceptance is unverified. Current-main reproduction report
• perf(core): the built-in tool descriptions and schemas are the largest block of non-conversation context, and have no size tracking #12054 — the built-in tool descriptions and schemas are the largest block of non-conversation context, and have no size tracking — completed by fix(core): gate teammate guidance and budget large tool surfaces #12142, perf(core): move the Agent tool's prompt-writing guidance into a bundled skill #12323 and perf(core): trim restated tool guidance and budget the resident tool surface #12532
• test(core): nothing bounds a bundled skill's <available_skills> entry size, so the 8,000-char listing trim recovers budget from the user's own skills instead #12472 — nothing bounds a bundled skill's <available_skills> entry, so the 8,000-char listing trim takes its budget from the user's own skills — closed by the entry bounds in merged fix(core,cli): budget the bundled skill listing; drop OpenTUI's category heading without a provider total #13159; broader listing compression remains gated on feat(ci): the token work has no recall or task-success gate — teach the existing benchmark to compare two configurations #12333(b)
• fix(cli): /context category breakdown does not close (skills listing unattributed, messages derived from a cache subtraction, startup prelude uncounted) #12033 — /context category breakdown does not close (skills listing unattributed, messages derived from a cache subtraction, startup prelude uncounted) — implemented by fix(cli): make /context categories add up to the provider total #12119
• fix(cli): /context subtracts a process-global cached-token count, so a daemon session can be charged another session's cache #12047 — /context subtracts a process-global cached-token count, so a daemon session can be charged another session's cache — implemented by fix(cli): prefer per-session cached token count in /context #12066
• fix(core): context-usage telemetry is dropped entirely when a non-function tool is present, and mixes two token estimators #12048 — context-usage telemetry is dropped entirely when a non-function tool is present, and mixes two token estimators — closed as not planned after fix(core): account for CJK in context message estimates #12273 did not meet the Bailian-alignment requirement
• Follow-up: deferred Suggestions from #12119 (/context category accounting) #12235 — follow-ups deferred from fix(cli): make /context categories add up to the provider total #12119 (/context accounting) — fix(cli): close the deferred /context accounting follow-ups #12540 landed the actionable fixes; residual Suggestions on hold (P3)
• /context shows a conversation-sized "Messages" row under "Estimated pre-conversation overhead" when usage is estimated #12606 — /context shows a conversation-sized Messages row under pre-conversation overhead when usage is estimated — closed by the OpenTUI heading fix in merged fix(core,cli): budget the bundled skill listing; drop OpenTUI's category heading without a provider total #13159
• Track structured Auto Memory rollout readiness on main #12947 — structured Auto Memory rollout gate — prior matched runs at 36533afb82 reported foreground main-model input −11.65%; selector ablation and P95 tail validation remain open. The default-off perf(memory): skip the selector after a delivered unique strong recall hit #13003 selector experiment merged in feat(memory): Opt-in selector skip for a unique new recall hit #13158; perf(memory): add a bounded cooldown after no-op extraction #13004 was withdrawn and stays open for preservation/replay design. Rollout is not complete
• feat(core): the eager tool surface is a hand-maintained static list — let something choose it, without invalidating the prompt prefix #12326 — who chooses the eager tool set — candidate set measured on 2026-09-29; feat(core): defer agent and goal declarations by default #13033 merged a narrower default (five Agent/Goal declarations deferred) ahead of the gate; confirming or reverting it is blocked on feat(ci): the token work has no recall or task-success gate — teach the existing benchmark to compare two configurations #12333(b)
• feat(ci): the token work has no recall or task-success gate — teach the existing benchmark to compare two configurations #12333 — recall / task-success gate for the token work — (a) landed in test(core): report what a tool-surface configuration actually withholds #12346; (b) needs the external benchmark pool, currently blocked on runner source access