Skip to content

docs: update outdated stats across all documentation - #774

Closed
telagod wants to merge 1 commit into
nullclaw:mainfrom
telagod:docs/fix-outdated-stats
Closed

telagod wants to merge 1 commit into
nullclaw:mainfrom
telagod:docs/fix-outdated-stats

Conversation

@telagod

@telagod telagod commented Apr 5, 2026

Copy link
Copy Markdown
Contributor

Problem

Documentation across README.md, CLAUDE.md, AGENTS.md, and architecture docs contains outdated numbers that no longer match the actual codebase.

Changes

All numbers verified against source code:

Metric Old New Verified by
Binary size 678 KB ~2.7 MB (minimal) / ~4.1 MB (full) zig build -Doptimize=ReleaseSmall && stat zig-out/bin/nullclaw
Tests 5,300+ / 5,640+ 6,300+ zig build test --summary all → 6,337 passed
Channels 19 / 17 24 ChannelId enum in src/channel_catalog.zig
Providers 50+ (9+41) 100+ (12+92) compat_providers table + core factory
Tools 35+ / 30+ 38+ src/tools/ file count
Source files ~230 / 245 ~260 find src/ -name '*.zig' | wc -l → 259
Lines of code ~249K / ~204K ~237K wc -l across all .zig files

Files Modified

  • README.md — 7+ stat locations updated
  • CLAUDE.md — binary size, test count, provider count
  • AGENTS.md — source files, LOC, test count, provider count, channel count
  • docs/en/architecture.md — provider/channel counts and lists
  • docs/zh/architecture.md — provider/channel counts and lists (Chinese)

Update binary size, test count, channel count, provider count, tool count,
source file count, and lines of code to match current codebase reality.

Changes across 5 files:
- Binary size: 678 KB → ~2.7 MB (minimal) / ~4.1 MB (full ReleaseSmall)
- Tests: 5,300+/5,640+ → 6,300+
- Channels: 19/17 → 24 (added WeChat, WeCom, Teams, Max, External)
- Providers: 50+ (9+41) → 100+ (12 core + 92+ compatible)
- Tools: 35+/30+ → 38+
- Source files: ~230/245 → ~260
- Lines of code: ~249K/~204K → ~237K
@eabase

eabase commented Apr 15, 2026

Copy link
Copy Markdown

Not only numbers, also missing clear instructions on how to setup gateway and web UI when using Talescale and how to setup tunnels!

vernonstinebaker pushed a commit to vernonstinebaker/nullclaw that referenced this pull request Sep 24, 2026
Maintainer edits available on all branches; CI green where run; nullclaw#777/nullclaw#775/nullclaw#774
conflict (branch updates required); all PRs BLOCKED by review-required
protection — first merge will test whether maintain role clears it. donprus
courtesy done via Discord by user.
@vernonstinebaker

Copy link
Copy Markdown
Contributor

This needs an update before it can merge. The branch conflicts with current main, and the figures were counted on 5 April.

Two checks from this week, on a ReleaseSmall macOS arm64 build: the default binary is about 4.9 MB, and zig build test --summary all passes about 7,380 tests. Please re-count every figure in this pull request against current main rather than carrying the April table forward.

The toolchain pin is Zig 0.16.0. Please do not restore 0.15.2 while rebasing. #775 also edits CLAUDE.md, so these two need to be updated against the same tree.

vernonstinebaker added a commit to vernonstinebaker/nullclaw that referenced this pull request Oct 5, 2026
Supersedes nullclaw#774. Every figure below was recomputed from source at
main (5f1cade) rather than carried forward, because nullclaw#774's own numbers
had themselves gone stale: it proposed 6,300+ tests against an actual
7,499, and ~2.7 MB for a binary that measures 4.66 MB.

## Scale figures

- Source files 245 -> 293; src LOC ~204K -> ~290K; tests 5,640+ -> 7,499.
- Providers: "50+ implementations (9 core + 41 compatible services)" ->
  10 core implementations plus 110 OpenAI-compatible registry entries,
  counted from the `.{ .name = ... }` table in `src/providers/factory.zig`,
  which is the actual single source of truth for compatible providers.
- Channels 17 -> 24; the architecture tables listed a partial set that
  omitted Teams, Max, WeChat/WeCom, and Weixin. Now labelled with the
  count and the omissions filled in. English and Chinese kept in sync.
- Tools "30+" -> 40 registered implementations, counted from
  `tool_name` constants in `src/tools/`.

## Binary size — a real discrepancy, not just a stale number

`README.md` and `CLAUDE.md` claimed 678 KB, and AGENTS.md set a sub-1 MB
ReleaseSmall target. A host build measures 4,889,528 bytes (~4.66 MB,
aarch64-macOS):

    zig build -Doptimize=ReleaseSmall && ls -l zig-out/bin/nullclaw

That is roughly 7x the long-quoted figure and about 4.7x the stated
target. All three now state the measured value, AGENTS.md marks the
sub-1 MB goal as an open gap rather than a met constraint, and a
verification command is included so the figure is re-checkable instead
of asserted.

This is a product-accuracy finding, not only a docs fix: the headline
size claim has been wrong for a long time. Worth deciding separately
whether to shrink the binary or restate the target.

## Keeping the numbers current

The recurring failure is that these figures are hand-maintained and
silently rot — nothing in CI checks them, which is also why nullclaw#774 went
stale in the first place. AGENTS.md §1 now carries the exact commands
used to derive each number, and all of them were confirmed to run as
written. A repo-wide sweep confirms no remaining 5,640 / 5,300 / 678 KB
/ 0.15.x references in the touched files.

Note for reviewers: the "last verified at 5f1cade" marker is the point
at which these numbers stop being true. Please re-derive rather than
increment them.

Docs only, no code touched. `zig build test --summary all` 13/13 steps,
7490/7499 passed, 9 skipped, 0 failures, 0 leaks. `zig build
-Doptimize=ReleaseSmall` and `zig fmt --check src/` both clean.
@vernonstinebaker

Copy link
Copy Markdown
Contributor

Closing as superseded. Full credit to @telagod for this one — auditing the documented scale figures was the right call, and the fact that it was needed at all is a criticism of the repo, not of the PR.

The idea is sound and is being carried forward in #1039. The numbers, however, had to be re-derived from scratch rather than adjusted:

#774 proposed Actual at main (5f1cade0)
Source files 259 293
src LOC ~237K ~290K
Tests 6,300+ 7,499
Binary (ReleaseSmall) ~2.7 MB ~4.66 MB (4,889,528 bytes)

Six months of drift on a repo that was mostly inactive, so the delta is larger than the correction.

Two further reasons it could not have landed as written:

  • The Zig pin. The patch leaves "Zig 0.15.2" in the README.md body while the build has been on 0.16.0 — it only corrects the binary-size figure. AGENTS.md §7.6 treats a version-pin mismatch as a hard fail.
  • Unverifiable counts. "12 core + 92+ compatible services" could not be reproduced. The compatible-provider count is actually derivable from src/providers/factory.zig (110 registry entries), which docs: refresh stale scale figures across the documentation #1039 now uses and documents.

#1039 also adds the derivation commands to AGENTS.md §1, so these figures can be recomputed instead of hand-edited — which is the part #774 was really pointing at, even if the numbers had gone stale in the meantime. Nothing checks them in CI, which is how they rotted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants