Skip to content

fix(#722,#724,#726): escape placeholder tokens, sandbox the nested summarizer, stop trusting a repo-local haiku config - #732

Merged
fdaviddpt merged 6 commits into
mainfrom
fix/722
Sep 19, 2026
Merged

fdaviddpt merged 6 commits into
mainfrom
fix/722

Conversation

@fdaviddpt

Copy link
Copy Markdown
Contributor

Three security-scan findings (2026-09-18 panel-verified scan), all read-only reasoning turned into landed fixes plus tests.

#722 -- literal {{...}} in a transcript permanently stalls memory saves

Untrusted transcript content substituted into the save/ndc/consolidation prompts is now escaped (zero-width-space inserted between any {{/}} pair) before interpolation, so a literal placeholder token surviving in attacker content cannot fool save-session.sh:700's placeholder guard into aborting a save with no cursor advance. Added a docs/diagnostics.md recovery section for anyone already stuck (clearing last-save.json).

Closes #722

#724 -- nested summarizer sandbox too wide

Both the Claude-CLI and Codex summarizer routes now spawn in _isolated_summarizer_cwd() (a fresh mkdtemp per call, torn down after) instead of the shared system tempdir. The Claude-CLI route now uses --tools "" -- the CLI's own disable-all-tools primitive -- in place of a hand-maintained _ALL_BUILTIN_TOOLS deny-list that missed PowerShell and REPL. The Codex route's child env is now an allow-list (_codex_child_env(): PATH/HOME/LANG/LC_ALL/CODEX_HOME/TMPDIR/TEMP/TMP) instead of a deny-list, and its own -o output file is now created inside the isolated cwd rather than the shared tempdir.

Closes #724

Below-bar, not fixed by this PR: cwd isolation does not stop a command running under Codex's --sandbox read-only (which still permits command execution) from reading OTHER files by absolute path -- e.g. the shared tempdir's own merged-config file (which can carry a live oauth token) or another concurrent save's own remember-prompt-* file. Closing that needs either disabling command execution in Codex's sandbox entirely (unclear if the CLI supports this) or a different integration shape than subprocess-based Codex invocation -- a design decision out of scope for this targeted fix. The same gap pre-existed this diff; this diff narrows rather than widens it.

#726 -- repo-local .remember/config.json selects summarizer credential and key-stripping policy

The per-project .remember/config.json's haiku block is no longer merged into REMEMBER_CONFIG when REMEMBER_DIR resolves inside the project checkout (both the jq merge path and the no-jq python fallback in scripts/lib-memory-dir.sh). pipeline/haiku.py's _config_candidates() also skips the raw ${REMEMBER_DIR}/config.json fallback when it can positively determine, via a new _remember_dir_is_project_local(), that it sits inside the project. Updated docs/configuration.md and docs/git-backup-security.md.

Closes #726

Below-bar items from self-review (not fixed, recorded here per the filing bar)

  • Prompt-template adjacency (Class C boundary check, security(scan-20260918): literal {{...}} in transcript stalls memory saves permanently #722's escaping fix). None of the three prompt templates currently place two placeholders back-to-back or a literal brace beside one, so the boundary case (a substituted value ending/starting in a literal brace recombining with adjacent template text into a fresh unescaped token) is not live today -- but would be if a future template edit changed that adjacency. Confirmed by reading all three .prompt.txt templates directly.
  • Windows coverage gap in the new security(scan-20260918): repo-local .remember/config.json selects summarizer credential and key-stripping policy #726 test module. Five of the new Python-level tests in tests/test_untrusted_project_haiku_config_726.py (testing _remember_dir_is_project_local/_config_candidates, pure Python, no bash subprocess) inherit that module's blanket win32 skip even though they need no POSIX-only primitive and could run on Windows. Separately, _remember_dir_is_project_local()'s string comparison (remember_abs == project_abs) uses no os.path.normcase(), so an NTFS case-insensitive path mismatch could theoretically misclassify on Windows -- this is untested on any platform since the module-level skip blocks it there too. Reasoned, not observed; no Windows host available in this run.

Tests

tests/test_prompts.py (updated + 2 new), tests/test_summarizer_sandbox_724.py (new), tests/test_untrusted_project_haiku_config_726.py (new). All red before their respective fixes, green after; combined targeted run across the affected modules: 338 passed in 21.07s. Full repo pytest not run locally, per policy -- CI is the matrix authority.

[AI-generated]

fdaviddpt and others added 3 commits September 19, 2026 14:14
…he nested summarizer, and stop trusting a cloned repo's own haiku config

- #722: substituted values in the save/NDC/consolidation prompts are now
  escaped so a literal {{TIME}}/{{BRANCH}}/{{LAST_ENTRY}}/{{EXTRACT}} token
  reaching the pipeline via transcript content cannot be mistaken by
  save-session.sh's placeholder guard for a genuinely unsubstituted
  placeholder and permanently stall memory capture for that session.

- #724: every built-in Claude Code tool is now explicitly disallowed
  unless requested (--allowedTools "" alone left approval-free tools like
  Read/Glob/Grep/Task still callable), both summarizer routes spawn in a
  fresh, empty directory instead of the shared system tempdir, and the
  Codex route's child environment is now an allow-list of only what the
  CLI needs rather than the full parent environment minus a deny-list.

- #726: a cloned repository's own .remember/config.json can no longer
  choose the nested summarizer's credential or force ANTHROPIC_API_KEY to
  be stripped -- the per-project haiku config block is no longer merged in
  at all when REMEMBER_DIR sits inside the project checkout, at both the
  shell merge layer and the Python fallback-candidate layer.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
…ase, safe cwd for codex output, fail-safe path check

Follow-up to the previous commit, addressing findings from self-review
(Explore + oss:auditor against the committed diff):

- #724: replace the hand-maintained _ALL_BUILTIN_TOOLS deny-list with
  `--tools ""`, the Claude CLI's own disable-all-tools primitive (verified
  on 2.1.261) -- the deny-list was already missing PowerShell/REPL and can
  only ever be as complete as its last manual update against the CLI's
  actual tool inventory.
- #724: the Codex route's own `-o` output file is now created inside the
  isolated per-call cwd rather than the shared system tempdir, closing the
  one artefact from that route _isolated_summarizer_cwd's introduction
  still left in the shared location.
- #726: the new bracket-range case switch in lib-memory-dir.sh moved into
  its own function carrying `local LC_ALL=C`, matching the file's existing
  convention for _resolve_remember_dir/_set_store_root, fixing a real
  tests/test_locale_ranges_695.py failure the previous commit introduced.
- #726: _remember_dir_is_project_local() now fails safe (treats an
  unresolvable path as project-local) rather than fails open, when
  MEMORY_PROJECT_DIR is set but os.path.realpath() raises.
- Fixed a vacuous `assert x or True` in tests/test_summarizer_sandbox_724.py
  that could never fail regardless of the isolation it claimed to check;
  restructured to capture the isolated cwd's existence from inside a
  subprocess.run side_effect, while the directory is still open.
- Corrected stale `cwd=gettempdir()` prose in pipeline/haiku.py's module and
  _child_env docstrings, scripts/resolve-paths.sh, and
  tests/test_nested_summarizer.py's docstring, all left describing the
  pre-#724 shared-tempdir behavior.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
@fdaviddpt

Copy link
Copy Markdown
Contributor Author

Maintainer review (tick review pass): CI is RED on this branch (sha e2f3394), consistently across all 4 completed legs so far (pytest ubuntu-latest 3.9/3.10/3.11/3.12) — 5 main legs still pending at last check. Not a flake: same two failures, same lines, on every leg.

Both trace to the scripts/lib-memory-dir.sh change for #726 (the --argjson strip_last_haiku wrapper around the layered-merge jq -s call):

  1. tests/test_case_divergence_298.py::test_the_per_tool_call_path_is_not_touched — this is the byte-pin guard that requires any executable-code change to scripts/lib-memory-dir.sh (relative to origin/main) to be explicitly registered in that test's own _SANCTIONED_DIVERGENCE dict. The new _classify_project_cfg_haiku_trust function and the --argjson strip_last_haiku jq rewrite are real, deliberate additions to that file's hot-path-adjacent config-merge chain, but nothing in this diff adds a corresponding entry there — so the guard fires exactly as designed.

  2. tests/test_git_restore_no_resource_when_disabled_663.py::test_enabled_still_runs_the_restore — asserts the literal substring "jq -s reduce" appears in the observed spawn trace for the layered merge. The jq program's own text now starts (if $strip_last_haiku then ... else . end) | reduce ... rather than reduce ..., so that substring no longer appears and the assertion fails, even though the merge itself still runs correctly.

Read as review.md's blast-radius question: this diff changes scripts/lib-memory-dir.sh's file convention (its jq program shape) and doesn't touch the one diagnostic (_SANCTIONED_DIVERGENCE) that exists specifically to require sign-off on changes to this file, nor the one other test hard-coded against the old program text. Whichever of those two needs updating is a question for you, not a finding against the PR — but it does need one before this can go green.

Decision: needs-fix. Not merging on this red. Will re-check once pushed.

(4 below-bar self-review items already recorded in the PR body — read, no disagreement with either routing.)

[AI-generated]

…gnostics

PR #732's CI was red on 4 legs (pytest ubuntu-latest 3.9-3.12), both
failures caused by #726's fix to scripts/lib-memory-dir.sh changing the
file's code shape without updating the two tests that pin it exactly.

- tests/test_case_divergence_298.py: `_SANCTIONED_DIVERGENCE` for
  scripts/lib-memory-dir.sh did not register #726's new
  `_classify_project_cfg_haiku_trust` function or the changed jq/Python
  merge programs -- the byte-pin guard compared this branch against
  origin/main and failed on every diverging line. Four new sanctioned
  substitution pairs cover the insertion and both merge-path changes,
  each verified (in isolation, before this commit) to reproduce this
  branch's file exactly when applied to origin/main's.
- tests/test_git_restore_no_resource_when_disabled_663.py: the positive
  control asserted the literal substring "jq -s reduce", which #726
  split by inserting `--argjson strip_last_haiku ...` between `-s` and
  the program text. Replaced with two substrings either side of the new
  flag, so the assertion still proves the full merge chain ran rather
  than being skipped.
- tests/test_config_flatten_cache_668.py:149 carried the same stale
  "jq -s reduce ..." string in a comment only (never asserted) --
  updated for accuracy while in the file.

Reproduced both failures locally first (red), then green after each
fix; ran all four touched test files together afterward (47 passed).
Narrowed to these files per this repo's own guard-lookup -- no
cross-cutting guard test matches (edited test files, not the guarded
source). No changelog fragment needed: #722/#724/#726 already have one
describing the shipped behavior; this is a CI-only test-pin fix with
no behavior change. `.oss/assemble_changelog.py --check` still passes.

Co-Authored-By: Max <noreply>
…ergence docstring

Self-review (Explore reviewer) found that the guard's own docstring still
described "One EXECUTABLE line ... allowed to differ" naming only #429,
while `_SANCTIONED_DIVERGENCE["scripts/lib-memory-dir.sh"]` already
carried entries for #695 (x2), #662 and #679 before this branch, and the
previous commit on this branch added four more for #726 -- nine entries
total, none but #429 documented in the prose above the dict. Reworded to
describe the mechanism generically (one entry per deliberate change,
each judged on its own, see the dict's own per-entry comments) with #429
and #726 as examples, rather than re-enumerating every entry by hand
where it will drift again the next time this list grows.

Ran the four touched test files together afterward: 47 passed.

Co-Authored-By: Max <noreply>
@fdaviddpt

Copy link
Copy Markdown
Contributor Author

Maintainer review (tick review pass, sha 68eabd3): the two stale-diagnostic failures are fixed and gone. CI is now RED on a different, deterministic failure -- all three macOS legs (pytest macos-latest 3.9, 3.10, 3.11) fail identically:

E       AssertionError: 20 external spawns per tool call (cold run: 24). Spawned:
E           git -C <tmp>/project rev-parse --path-format=absolute --git-common-dir --git-dir
E           jq -s --argjson strip_last_haiku true
E                   (if $strip_last_haiku then (.[-1] |= del(.haiku)) else . end)
E                   | reduce .[] as $x ({}; . * $x)
E                   | with_entries(select(.key | startswith("_") | not))
...
tests/test_post_tool_hook_spawns.py:247: AssertionError

tests/test_post_tool_hook_spawns.py::test_the_post_tool_hook_stays_inside_its_spawn_budget -- POST_TOOL_SPAWN_BUDGET = 19 (measured 17 + 2 slack), warm run measured at 20 on all three legs, byte-identical numbers (20/24) on 3.9, 3.10 and 3.11, and reproduced again after a manual re-run of the 3.10 job (gh run rerun 35444925715 --failed) -- so this is not a flake, it is deterministic given this branch's code on this platform.

Confirmed not a stale-diagnostic repeat: origin/main is currently green (14/14 legs, including this same macOS/pytest matrix, 2h old at dde843f), and this branch's only functional diff since main is #726's change to scripts/lib-memory-dir.sh (the last two commits on this branch touch only test files). So the extra spawns trace to that diff, even though the jq merge is still a single spawn on its face.

Locally (macOS, same branch/sha, system /bin/bash 3.2.57) I could not reproduce the failure directly -- a manual run of the same test gave cold=20, warm=16 (under budget), a full 4 fewer than CI's cold=24, warm=20. In both my local run and CI's, cold and warm differ by exactly the same offset, and CI's extra items include three date calls (date +%Y-%m-%d, date +%H:%M:%S x2, date +%s) that are entirely absent from my local run's spawn list. scripts/lib-clock.sh's own docstring says exactly this shape of divergence is possible: _remember_date is free on bash >= 4.2 (printf '%(FMT)T', no fork) and pays one process per call on stock bash 3.2 -- but neither this diff nor any commit on this branch touches lib-clock.sh, save-session.sh, or the spawn-budget test itself, so I can't yet say why the save/header path that spends those four calls fires deterministically on CI's macOS runner (image macos-26-arm64) and not in my own shell. Reasoned, not observed, past that point -- I did not chase it further, since isolating why one code path fires is a developer diagnosis, not a maintainer-review one.

Decision: needs-fix. Not merging on this red. Re-run confirmed it isn't transient; please chase whether #726's _classify_project_cfg_haiku_trust / the new jq merge shape changed something upstream of the save/entry-header path (config content, a computed threshold, cooldown timing) rather than the spawn count itself, on a real macOS box if you have one.

(Below-bar self-review items already recorded in the PR body from the previous pass -- no new disagreement.)

… a macOS-only spawn-budget CI failure

PR #732's own follow-up self-review (a separate maintainer session) found
CI red on all 3 macos-latest legs (py 3.9/3.10/3.11), all 12 other legs
green, on tests/test_post_tool_hook_spawns.py::
test_the_post_tool_hook_stays_inside_its_spawn_budget (observed 20 spawns
against POST_TOOL_SPAWN_BUDGET=19). The maintainer's own local macOS run
could not reproduce it (cold=20, warm=16, under budget) and traced the gap
to `date` calls without being able to isolate it further, since their
local `bash` resolves to a homebrew/GNU build rather than the CI runner's
stock /bin/bash 3.2.

Reproduced locally by prefixing PATH with `/bin` so the test's `bash`
resolves to this machine's own stock 3.2.57 (same binary family as the
macOS CI runner), confirming the exact failure (20 <= 19 assertion, same
20-line spawn dump CI reported). A/B-swapped scripts/lib-memory-dir.sh
in place between this branch's content and `git show origin/main:...`
under that same PATH (restoring and verifying `git status` clean
before/after each swap) to isolate the cause: origin/main's version
measures 16 (comfortably under budget); this branch's #726 change alone
pushes it to 20, a delta of exactly +4.

Root cause: not an extra process. #726 wrote the jq merge filter as a
multi-line quoted string. The spawn-counting shim
(tests/spawn_counting.py) logs each process as `printf "%s %s\n" "$name"
"$*"`, and the filter's own embedded newlines reach that log line
verbatim -- one jq process now writes 5 physical log lines instead of 1
(the miscount tests/spawn_counting.py's `spawns()` makes by splitting on
newlines), a phantom +4 with no real spawn behind it. jq is
whitespace-insensitive, so collapsing the filter to a single line is a
pure counting fix: verified the collapsed filter produces byte-identical
output on a hand-built fixture (haiku key still stripped from the last
source, other keys still merged).

Updated the byte-pin guard's sanctioned-divergence tuple for this hunk
(tests/test_case_divergence_298.py) to match the new one-line jq text,
since the guard pins scripts/lib-memory-dir.sh byte-for-byte against
origin/main.

Verified green under stock bash 3.2 (PATH-prefixed /bin): the full
spawn-budget test class this repo carries (tests/test_post_tool_hook_spawns.py,
test_post_tool_fast_path_350.py, and 14 other files matching
`grep -l spawn_counting`) -- 147 passed, 11 pre-existing loud skips
(bash-version-floor guards unrelated to this change), 0 failures. Also
re-ran tests/test_untrusted_project_haiku_config_726.py (13 passed) and
the four files from the previous commit on this branch (60 passed) to
confirm #726's actual behavior is unchanged.

Logged the investigation as trap.d/732.jq-multiline-filter-inflates-spawn-count.md
-- a reusable lesson (a multi-line quoted filter/heredoc argument silently
inflates every spawn-count test that logs via newline-split "$*", with no
functional change and no signal at write time) for `/oss:curate` to weigh
against a jit-context rule later.

Co-Authored-By: Max <noreply>
@fdaviddpt
fdaviddpt merged commit 7ba7609 into main Sep 19, 2026
16 checks passed
@fdaviddpt
fdaviddpt deleted the fix/722 branch September 19, 2026 14:17
fdaviddpt added a commit that referenced this pull request Sep 19, 2026
…s own composition (#735)

* fix(#734): drop the #662 sanctioned-divergence tuple stranded by #726's own composition

`tests/test_case_divergence_298.py::test_the_per_tool_call_path_is_not_touched`
was red on ubuntu-latest and macos-latest after #732 merged. The
`_SANCTIONED_DIVERGENCE["scripts/lib-memory-dir.sh"]` entry for #662's
`_LAZY_PYTHON_GUARD` insertion still pinned the pre-#726 two-argument
python-fallback invocation as both its old_code and new_code, while #726
(shipped in #732) composed the guard directly into a three-argument
invocation and never updated this older entry to match. Since origin/main
only moves forward, neither shape is a substring of the real file, so the
guard's own "neither old nor new" assertion fired.

Remove the stale tuple: #726's own tuple already re-asserts the guard line
as the first line of its new_code, so removing the redundant #662 entry
loses no coverage of the guard-precedes-invocation invariant. Verified every
remaining `_SANCTIONED_DIVERGENCE` entry (both files) resolves to
new-present against current origin/main -- no other entry carries the same
staleness risk right now.

Closes #734

Co-Authored-By: Max <noreply>

* fix(#734): update the stale #662-example cross-reference after the tuple's removal

Self-review (Explore reviewer) flagged that removing #662's own
_SANCTIONED_DIVERGENCE tuple in 1cd7679 left two docstrings still citing
"#429 and #662 both touch lib-memory-dir.sh" as the illustrative example of
a file carrying more than one allowance. #662's guard insertion no longer
has a standalone tuple -- it now only survives embedded inside #726's -- so
#662 is no longer an accurate second example; #726 is. Updates both
occurrences (tests/test_case_divergence_298.py and
tests/test_sanctioned_divergence_state_440.py). Cosmetic/documentation only,
no behavior change; targeted suite re-run green (26 passed).

Co-Authored-By: Max <noreply>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment