Backtrack rewind sets context remaining to 0% and triggers auto-compaction #9601

Summary

After using the TUI backtrack/rewind flow (Esc to open transcript overlay, select a previous user message, Enter to rollback), the context indicator jumps to 0% left and the next turn is auto-compacted, even when the post-rollback thread should be far below the model context window.

Version

• Repo: openai/codex
• Commit: 16b9380e99474c87502c22ed99bac497d116e724

Repro

1. Start a Codex TUI session and exchange a few turns (enough to have non-empty history).
2. Confirm the bottom-bar context indicator is not near 0% left.
3. Press Esc twice to open the transcript overlay (backtrack preview).
4. Use Left to select an earlier user message.
5. Press Enter to rollback.
6. Observe the context indicator becomes 0% left and the next user turn triggers an automatic compact.

Expected

• After rollback, token/context usage reflects the smaller history and the context indicator shows substantial remaining context.
• Auto-compact should not run unless the post-rollback prompt actually exceeds the auto-compact threshold.

Actual

• Immediately after rollback, remaining context shows 0% left.
• A compact runs even when the post-rollback thread should be well below the context window.

Suspected cause

The rollback handler recomputes token usage from a heuristic estimate rather than server-provided token usage:

• codex-rs/core/src/codex.rs:2522 handlers::thread_rollback(...) calls sess.recompute_token_usage(...).
• codex-rs/core/src/codex.rs:1626 Session::recompute_token_usage(...) uses ContextManager::estimate_token_count(...).
• codex-rs/core/src/context_manager/history.rs:87 estimate_token_count(...) estimates by JSON-serializing history items (serde_json::to_string(item)) and then calling byte/token heuristics.

This can drastically overestimate the prompt size (JSON framing overhead, large structured fields, image/base64 fields, etc.), which then drives:

• The TUI context indicator (codex-rs/tui/src/chatwidget.rs:906 uses info.last_token_usage.percent_of_context_window_remaining(window))
• The auto-compact guard (token usage estimate feeds get_total_token_usage / auto_compact_token_limit).

Possible fixes

• After rollback, clear token usage info (emit TokenCount { info: None }) until the next real model response provides accurate counts.
• Or, compute the estimate from the normalized prompt representation (same normalization used for for_prompt()), and avoid counting JSON framing. If images are present, estimate image token cost instead of base64 length.