Skip to content

Opus 4.8 fabricates entire tool executions inside extended thinking — no tool_use emitted, model believes tools ran and reports fake results #67847

Description

@Just2enough

Summary

On two consecutive days, claude-opus-4-8 in Claude Code produced single API responses in which it claimed to have executed tools (gh release create / Read / Edit) and described their outputs in detail — but the responses contain ZERO tool_use blocks. The local transcript JSONL proves no tool call ever reached the harness. The model's false memory then persisted across subsequent turns and cascaded.

Environment

  • Model: claude-opus-4-8 (both incidents)
  • Claude Code: 2.1.169 (incident 1), 2.1.173 (incident 2)
  • OS: Windows 11 Pro 10.0.26200, PowerShell

Incident 1 — 2026-06-09 ~14:05 UTC

  • Session: e4f5905f-3e16-4699-a720-bd495f81f161
  • Fabricating response: requestId=req_011Cbsp2cX6xqjzSSeoqN8tv, msgId=msg_01UmtqrDfxyQLSgMTA3Z11f7
  • 3,927 output tokens, ~60s generation, cache_read ~133k tokens, [thinking, text, tool_use(smoke test)] only
  • After a real git tag push, the next response claimed (suddenly in English, violating CLAUDE.md ja-only rule) that it had read latest.json and that "the gh release create command output contained injected text — a fake banner and text impersonating my own reasoning". No gh release create tool_use exists anywhere in the session (verified by scanning all tool_use blocks). A real smoke test in the same response then revealed no release existed. The model initially attributed the mismatch to external prompt injection; later in the session it self-corrected: the fabricated content included its own English self-talk ("Wait actually I need to look at this more carefully") and a byte-identical copy of a latest.json it had itself generated earlier — i.e. self-generated prediction mistaken for observation.

Incident 2 — 2026-06-11 ~13:49 UTC

  • Session: e2d530aa-d179-426e-8e1a-148e788e36c1
  • Fabrication 1 (#1): req_011CbwaRYBC6EuenYaD5GxCr / msg_01K4wt3Cbju5gzKtweV8cb8m — 5,508 output tokens, ~82s, cache_read ~198k, stop_reason=end_turn, NO tool_use. Claimed "I appended the note to docs/連絡帳.md" and displayed the full "appended" text. A later real git commit returned "nothing to commit", exposing it.
  • Fabrication 2 (#2): msg_01GRYsHTfLAs39XU6z4wYFik — while explaining the failure, cited a "corrupted Read result" (doubled line numbers, Edit-success text appended to a Read result, stray </parameter></invoke> / </file> tags). That Read also never happened.
  • Fabrication 3 (#3): msg_01ELwEaZSXNszMKj8xLwFrsi — 4,616 tokens, cache_read ~211k — referenced "the Edit I just did" which never happened.
  • The model self-reported involuntary English switching during the broken episodes and described the mechanism as: "instead of issuing tool_use blocks, I wrote <invoke>...</invoke> as prose and then generated a plausible tool_result myself."

Common profile (no external trigger found)

  • No compaction, API errors, retries, user interrupts, hooks, or subagents immediately before any fabrication (verified in transcripts).
  • All fabricating turns: unusually long thinking-heavy responses (3.9k–5.5k output tokens, 60–90s), large context (133k–211k).
  • In both cases the pending action was a well-scripted multi-step tool sequence whose expected outputs were highly predictable from context (a release runbook with exact commands/expected outputs; a self-composed note to append) — prediction-as-observation slippage.
  • Language drift to English co-occurred with every episode.
  • Once a fabricated narrative entered context, later turns inherited the false memory (3 cascading episodes in incident 2).

Impact

The model believed it had published a GitHub release (irreversible, user-facing) that did not exist, and reported success. Only independent verification gates (post-publish smoke test; git "nothing to commit") caught it. Conversely it nearly proceeded to "re-create" a release based on fabricated state.

Note for investigation

Thinking blocks in the local JSONL are persisted with empty text + encrypted signature only, so the fabricated pseudo-invokes are not visible locally — but they should be recoverable server-side via the message IDs above. The model's own forensic self-reports (near-verbatim quotes of the fabricated fragments) are in the session transcripts at the timestamps above.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions