Skip to content

Include tools changes in existing turn analytics - #50943

Closed
aibrahim-oai wants to merge 6 commits into
openai:mainfrom
aibrahim-oai:codex/tools-cache-change-metric
Closed

aibrahim-oai wants to merge 6 commits into
openai:mainfrom
aibrahim-oai:codex/tools-cache-change-metric

Conversation

@aibrahim-oai

@aibrahim-oai aibrahim-oai commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Turn analytics currently cannot show how often the available tools change during a session. Include tools_change_count on codex_turn_event, next to the existing sampling and retry counts, so the backend can break it down using the event's usual originator/client tags.

Compare the existing shared Prompt.tools list at the common inference entry point. Increment once whenever it differs from the previous inference call, including across turns, and report the count on the current turn. The first call, identical retries and compaction do not increment it. This counts all tool-definition differences, including fields that are not serialized for some models; it measures changes to tools and does not establish an actual cache miss.

Validation: just fmt and git diff --check passed. The existing core turn-profile test covers baseline, unchanged retries, tool fields, additions, ordering, removal and continuity after a transport fallback. Analytics serialization was tested in the preceding PR revision.

just test -p codex-core: 3,694 passed, 22 failed, 9 skipped. The tools-change regression passed; the 22 failures are exactly the same tests as the saved run before this revision (no new failures).

@aibrahim-oai aibrahim-oai changed the title Count tools changes between inference requests by originator Include tools changes in existing turn analytics Oct 4, 2026
@jgershen-oai jgershen-oai self-assigned this Oct 4, 2026
@jgershen-oai
jgershen-oai self-requested a review October 4, 2026 20:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants