Skip to content

fix(mcp-builder): write evaluation report as UTF-8 - #1720

Open
CJstate wants to merge 1 commit into
anthropics:mainfrom
CJstate:fix/mcp-builder-evaluation-utf8
Open

fix(mcp-builder): write evaluation report as UTF-8#1720
CJstate wants to merge 1 commit into
anthropics:mainfrom
CJstate:fix/mcp-builder-evaluation-utf8

Conversation

@CJstate

@CJstate CJstate commented Sep 3, 2026

Copy link
Copy Markdown

Fixes #1669.

Path.write_text() defaults to the locale encoding. On Windows that is cp1252, and the report contains the ✅/❌ correctness markers from TASK_TEMPLATE. As soon as at least one task is marked incorrect, the run raises UnicodeEncodeError on the final write_text() call, and the entire paid-for evaluation result is discarded.

PYTHONIOENCODING does not help (it governs stdio, not pathlib).

Pass an explicit encoding="utf-8" so the report survives on any locale.

@CJstate
fix(mcp-builder): write evaluation report as UTF-8
d486607 
Path.write_text() defaults to the locale encoding. On Windows that is
cp1252, and the report contains the ✅/❌ correctness markers from
TASK_TEMPLATE. As soon as at least one task is marked incorrect, the
run raises UnicodeEncodeError on the final write_text() call, and the
entire paid-for evaluation result is discarded.

PYTHONIOENCODING does not help (it governs stdio, not pathlib).

Write the report with an explicit UTF-8 encoding so the result survives
on any locale.

Fixes anthropics#1669
@CJstate

CJstate commented Sep 22, 2026

Copy link
Copy Markdown
Author

Added independent verification of the failure mode, plus a sweep for sibling exposure.

Reproduced on a Windows machine with a non-UTF-8 locale (cp936):

locale preferred encoding: cp936
write_text(report)              -> UnicodeEncodeError: 'gbk' codec can't encode character '✅'
write_text(report, encoding=..) -> ok, round-trips byte-for-byte

The report content is TASK_TEMPLATE populated at line 263 with correct_indicator="✅" if result["score"] else "❌", so the crash needs at least one incorrect task — a fully-passing run writes fine, which is what makes it look intermittent (as @Hahaknight noted on the issue, it is in fact deterministic once any task fails).

Sibling sweep in skills/mcp-builder/scripts/: evaluation.py:366 is the only locale-encoding exposure. parse_evaluation_file uses ET.parse, which honours the XML declaration's own encoding; the remaining json.dumps calls only produce strings and never touch the filesystem. So the one-line fix is complete for this skill — no need to widen it.

This was referenced Sep 27, 2026
Open
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

mcp-builder: evaluation.py discards a completed run on Windows — write_text() without encoding hits cp1252 on the report's emoji

1 participant

@CJstate