feat: opt-in structural repo map provider by SK-DEV-AI · Pull Request #1320 · 1jehuang/jcode

SK-DEV-AI · 2026-09-19T11:11:37Z

Closes #1230 as built, exactly the agreed boundary: read-only provider, never core, never default-on.

Regex grammar sets (Rust, TS/JS, Python; unlisted extensions yield nothing, never fail), file-reference graph (unique-owner edges only, ambiguous names carry none), hand-rolled PageRank with seed personalization, token-budgeted stubs without bodies. No tree-sitter, no petgraph, no new dependencies.

Cache under .jcode/cache/repomap.json keyed per file by mtime-nanos+size (millis truncation fixed); stale files rebuild alone. Tool registers only when repomap_token_budget > 0 (default 0, 0 disables and the tool is unregistered so models never see a dead tool); env JCODE_REPOMAP_TOKEN_BUDGET wins over file, registered in the CONFIG_ENV_KEYS fingerprint, documented in the default file. Measured prototype numbers from the thread (1350 files / 4.6s cold / 0.066s warm at 61x) carry over: same graph shape, per-file invalidation preserved.

Since filed: symbol-name seeds (multi-anchor personalization toward defining files, not just paths); git co-change edges (files committed together 2+ times in the last 200 commits, single shared commits ignored) fixing the base-module blind spot — fail-soft outside git, nested roots rebased via show-toplevel; symlink confinement both directions plus a 2MB per-file read bound; first-block budget respect; export-pattern dedup; commit markers require full structure (40 or 64 hex — SHA-256 repos pair normally) so COMMIT:-prefixed filenames parse as paths.

Tests: 27 engine + 3 tool, all green, including real-repo e2e commits (proven non-vacuous) and old-code-fails controls. Clippy and fmt clean.

greptile-apps · 2026-09-19T11:32:10Z

Confidence Score: 3/5

Not safe to merge until repository-map cache loading rejects redirected paths and bounds the data it reads.

Findings

Bound cache reads ▶

Fix with agent prompt

### Issue 1
crates/jcode-base/src/repomap.rs:272
When repository maps are enabled, `load_cache` follows the repository-controlled `.jcode/cache/repomap.json` path and reads it with no size limit before any symlink validation. A repository can redirect that path, or one of its parents, to a very large external file or an unbounded device such as `/dev/zero`; invoking the map can then hang the process or exhaust its memory. Reject symlinked cache components and enforce a bounded cache read before deserializing it.

> **How this was verified:** A repository cache symlink to an external 16 MiB file was accepted and read by the public map-building flow.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds an opt-in repository-map provider with bounded source parsing, cache support, reference and co-change ranking, and configuration-driven registration. A repository-controlled cache path can still redirect cache loading to an external file that is read without a size limit, so this must be fixed before merging.

_{Reviews (8) · Last reviewed commit: "fix(rebase): keep upstream replace/sdk m..."}

greptile-apps · 2026-09-19T11:32:23Z

Comments Outside Diff

These findings sit on lines the diff does not cover, so they could not be posted inline. Each one leaves this list once its file changes.

Repo-map cache loader follows and reads unbounded cache symlinks ▶
- Bug
  - load_cache directly calls std::fs::read on .jcode/cache/repomap.json. A runtime test created that repository path as a symlink to an external 16 MiB valid JSON file, then invoked public build_map; it completed successfully, proving the path is reached and followed. A /dev/zero target would therefore remain unbounded rather than being rejected or capped.
- Cause
  - At crates/jcode-base/src/repomap.rs:272, load_cache performs an unbounded follow-by-default std::fs::read(cache_path(root)). The symlink checks in save_cache protect only the later write path, not cache loading.
- Fix
  - Before opening the cache, reject symlinked .jcode, cache, and repomap.json components (including a no-follow final-file check), and enforce a small maximum cache size before reading. Prefer no-follow OS open semantics where available to avoid check/open races.

Closes 1jehuang#1230 as built, exactly the agreed boundary: read-only provider, never core, never default-on. Regex grammar sets (Rust, TS/JS, Python; unlisted extensions yield nothing, never fail), file-reference graph (unique-owner edges only, ambiguous names carry none), hand-rolled PageRank with seed personalization, token-budgeted stubs without bodies. No tree-sitter, no petgraph, no new dependencies. Cache under .jcode/cache/repomap.json keyed per file by mtime+size; stale files rebuild alone. Tool registers only when repomap_token_budget > 0 (default 2000, 0 disables); env JCODE_REPOMAP_TOKEN_BUDGET wins over file, registered in the CONFIG_ENV_KEYS fingerprint, documented in the default file. Tests: 13 engine (extraction rs/ts/py, skip dirs, hub rank, seed personalization, ambiguous/unique edges, budget truncation, budget-0 disable, cache rebuild + byte-identical warm hit, seeds end to end, empty tree) + 3 tool (stubs, budget-0 notice, missing dir errors). Clippy clean, fmt clean.

…ardening - Default budget 2000 -> 0: the provider is opt-in, so the default must not register the tool. Opt in via file or JCODE_REPOMAP_TOKEN_BUDGET. - Symlink confinement (both security findings): the walker never follows file/dir symlinks and every candidate is canonicalized under the root; the cache writer refuses symlinked .jcode/cache components and destination. Reads bounded at 2MB per file (no-symbol skip above). - First block over budget yields no map (was: always emitted). - Fingerprint mtime millis -> nanos plus size (same-ms equal-length edits no longer reuse stale symbols). - Dropped the export-specific fn patterns overlapping the general optional-export ones (ts/js emitted every exported fn twice). - Renamed DEFAULT_TOKEN_BUDGET to DEFAULT_REPOMAP_TOKEN_BUDGET (a different 200_000 const of the same name already exists).

Dedicated research pass (Aider RepoMap pipeline, 2025-26 refinements: multi-anchor personalization, co-change signals for the base-module blind spot): - Seeds accept symbol names, not just path prefixes: naming an identifier personalizes the teleport toward the files defining it. Case-insensitive substring; documented on the tool schema. - Recent-history co-change pairs (>=2 shared commits in the last 200) become bidirectional reference edges, so dependents boost their base even though no reference edge points back. Single shared commits ignored (bulk adds/renames are noise). Fail-soft outside git repos; repo-nested roots rebased via show-toplevel. - Synthetic-graph tests point at isolated roots so they never inherit real co-change edges from the checkout the suite runs in.

greptile-apps · 2026-09-23T17:02:31Z

+}
+
+fn load_cache(root: &Path) -> HashMap<String, CachedFile> {
+    let Ok(bytes) = std::fs::read(cache_path(root)) else {


Bound cache reads

When repository maps are enabled, load_cache follows the repository-controlled .jcode/cache/repomap.json path and reads it with no size limit before any symlink validation. A repository can redirect that path, or one of its parents, to a very large external file or an unbounded device such as /dev/zero; invoking the map can then hang the process or exhaust its memory. Reject symlinked cache components and enforce a bounded cache read before deserializing it.

How this was verified: A repository cache symlink to an external 16 MiB file was accepted and read by the public map-building flow.

Artifacts

Executable finite cache-symlink validation script

Creates and runs the temporary public-entry-point integration test using an external finite 16 MiB cache target, then removes the temporary test source; it is the executed source.

Symlinked 16 MiB external cache accepted by build_map

Captured `cargo test` output for the symlink-controlled cache path; it shows the path is a symlink, the target is 16 MiB, and `build_map` completes, proving the unsafe read path is reachable.

Regular small cache control reaches build_map

Captured control `cargo test` output for a 2-byte regular cache; it shows the same public map-building flow completes normally.

_{Ran code and verified through T-Rex}

Prompt To Fix With AI

This is a comment left during a code review. Path: crates/jcode-base/src/repomap.rs Line: 272 Comment: **Bound cache reads** When repository maps are enabled, `load_cache` follows the repository-controlled `.jcode/cache/repomap.json` path and reads it with no size limit before any symlink validation. A repository can redirect that path, or one of its parents, to a very large external file or an unbounded device such as `/dev/zero`; invoking the map can then hang the process or exhaust its memory. Reject symlinked cache components and enforce a bounded cache read before deserializing it. > **How this was verified:** A repository cache symlink to an external 16 MiB file was accepted and read by the public map-building flow. --- For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

greptile-apps Bot reviewed Sep 19, 2026

View reviewed changes

github-actions Bot added area: config Configuration, defaults, and environment overrides. area: tools Agent tools, integrations, and tool execution. type: feature Adds a new user-facing capability. labels Sep 19, 2026

greptile-apps Bot reviewed Sep 19, 2026

View reviewed changes

Comment thread crates/jcode-base/src/repomap.rs

Comment thread crates/jcode-base/src/repomap.rs

SK-DEV-AI force-pushed the feat/repomap-provider branch from 8053699 to 93d8747 Compare September 20, 2026 11:39

greptile-apps Bot reviewed Sep 20, 2026

View reviewed changes

Comment thread crates/jcode-base/src/repomap.rs Outdated

greptile-apps Bot reviewed Sep 20, 2026

View reviewed changes

Comment thread crates/jcode-base/src/repomap.rs Outdated

SK-DEV-AI added 8 commits September 23, 2026 22:10

fix(review): bound co-change, NUL git paths, real e2e commits

2c9ea44

fix(rebase): register upstream Jev provider env var in fingerprint

c56b4a5

fix(review): strict commit-marker structure, COMMIT:-prefixed files pair

9b5a1fc

fix(review): accept SHA-256 commit markers in co-change parse

b29ba02

fix(rebase): keep upstream replace/sdk mods alongside repomap

754a986

SK-DEV-AI force-pushed the feat/repomap-provider branch from c260066 to 754a986 Compare September 23, 2026 16:49

greptile-apps Bot reviewed Sep 23, 2026

View reviewed changes

Navigation Menu

Repository navigation

feat: opt-in structural repo map provider - #1320

feat: opt-in structural repo map provider#1320
SK-DEV-AI wants to merge 8 commits into
1jehuang:masterfrom
SK-DEV-AI:feat/repomap-provider

SK-DEV-AI commented Sep 19, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Sep 19, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot commented Sep 19, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot Sep 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

SK-DEV-AI commented Sep 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps Bot commented Sep 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 3/5

Findings

Summary

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot commented Sep 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comments Outside Diff

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot Sep 23, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

SK-DEV-AI commented Sep 19, 2026 •

edited

Loading

greptile-apps Bot commented Sep 19, 2026 •

edited

Loading

greptile-apps Bot commented Sep 19, 2026 •

edited

Loading