Repository navigation
Conversation
![devin-ai-integration[bot]](/web-api/iframe-proxy?url=https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| await self._broadcast(_decode_worker_output(line).rstrip("\n")) | ||
|
|
||
| # Handle out message | ||
| try: |
There was a problem hiding this comment.
🟡 json.loads(line) on raw bytes raises uncaught UnicodeDecodeError for non-UTF-8 stdout lines
On line 339, _decode_worker_output(line) safely handles non-UTF-8 bytes for the broadcast. However, on line 343, json.loads(line) still receives the original raw bytes. When those bytes contain invalid UTF-8 (e.g., \x97), json.loads raises UnicodeDecodeError (confirmed: it is NOT a subclass of json.JSONDecodeError). This exception escapes the inner except json.JSONDecodeError: handler at line 362 and propagates to the outer except Exception as e: at line 367, terminating the entire read loop and putting the session into error state. The intent of the inner handler is to gracefully skip unparseable lines, but this case slips through. The fix is to pass the already-decoded string to json.loads so that invalid-UTF-8 lines produce a json.JSONDecodeError (due to the replacement character making invalid JSON), which IS caught by the inner handler, allowing the loop to continue.
(Refers to lines 339-343)
Was this helpful? React with 👍 or 👎 to provide feedback.
|
Follow-up pushed in 48d2b29. The read loop now parses the decoded worker line instead of the original raw bytes, so invalid UTF-8 stdout is broadcast with replacement characters and then skipped as invalid JSON without escaping as UnicodeDecodeError. Validation on Windows: uv run pytest tests/web/test_session_error_recovery.py -q -p no:cacheprovider: 6 passed; uv run ruff check src/kimi_cli/web/runner/process.py tests/web/test_session_error_recovery.py; uv run ruff format --check src/kimi_cli/web/runner/process.py tests/web/test_session_error_recovery.py; python -m py_compile src/kimi_cli/web/runner/process.py tests/web/test_session_error_recovery.py; git diff --check. |
|
Still relevant — a worker emitting non-UTF8 bytes currently throws instead of being decoded leniently, killing the run. Rebased and green if you get a chance to look. |
|
Still the case on main: process.py:318/324/329/335 decode subprocess output with bare .decode(utf-8), so a single non-UTF-8 byte from the process still raises here. Happy to adjust the approach if maintainers prefer something different. |
This branch has not been deployed
1 participant
Summary
Fixes #2313.
The web session runner decoded worker stdout and crash stderr with strict UTF-8. On Windows, child processes can still emit locale-encoded bytes such as cp1252 smart punctuation. A single invalid byte then hides the real worker failure behind a UnicodeDecodeError.
This change decodes worker output with replacement characters at the process boundary, matching the existing uploaded text-file behavior. Wire JSON is still parsed after decoding, but the runner no longer crashes while trying to report a worker error.
To verify
uv run pytest tests\web\test_session_error_recovery.py -quv run ruff check src\kimi_cli\web\runner\process.py tests\web\test_session_error_recovery.pypython -m py_compile src\kimi_cli\web\runner\process.py tests\web\test_session_error_recovery.pygit diff --check