Skip to content

ACP: PromptResponse usage omits cached/thought tokens, inflating client cost estimates #27985

Description

@VascoSch92

What

When running as an ACP server (gemini --acp), the token usage reported for a prompt turn only includes input and output tokens. The cached and thought/reasoning token counts are dropped — even though Gemini's usageMetadata provides them and the CLI already consumes them elsewhere (telemetry, UI, chat recording).

In packages/cli/src/acp/acpSession.ts, the GeminiEventType.Finished handler reads only promptTokenCount and candidatesTokenCount:

const usage = event.value.usageMetadata;
if (usage) {
  turnInputTokens = usage.promptTokenCount ?? turnInputTokens;
  turnOutputTokens = usage.candidatesTokenCount ?? turnOutputTokens;
}

cachedContentTokenCount and thoughtsTokenCount are never read, and the emitted _meta.quota.token_count contains only input_tokens / output_tokens. By contrast, these fields are read in packages/core/src/agent/event-translator.ts, packages/core/src/telemetry/types.ts, and packages/core/src/services/chatRecordingService.ts — so the data is available; only the ACP path discards it.

Why it matters

ACP clients that estimate cost from token counts treat all input tokens as uncached, because the cached count is absent/0. For long agentic sessions where most of the prompt is cached turn-to-turn, this overstates the estimated cost by ~3× relative to the real (cached) spend.

We hit this analyzing OpenHands eval runs through the Gemini CLI ACP harness: every instance reports cache_read_tokens = 0, so the token-based cost estimate is ~3× the actual proxy bill (the underlying spend already benefits from caching — only the reporting is wrong).

Relationship to #24280

This is related to but distinct from #24280 (“populate the standard PromptResponse.usage field”). That issue's proposed payload lists only input_tokens / output_tokens / total_tokens, so implementing it as written would not close the cache-token gap. The standard ACP Usage type already has cachedReadTokens and thoughtTokens fields — they just need to be populated.

Proposed fix

In acpSession.ts, also capture usageMetadata.cachedContentTokenCount and usageMetadata.thoughtsTokenCount, and populate the standard ACP PromptResponse.usage field (inputTokens, outputTokens, cachedReadTokens, thoughtTokens, totalTokens). This aligns Gemini CLI with Claude Agent ACP / Codex ACP and lets any ACP client read accurate usage without Gemini-specific _meta.quota parsing. The existing _meta.quota payload can be kept for backward compatibility.

I have a small PR ready for this.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/non-interactiveIssues related to GitHub Actions, SDK, 3P Integrations, Shell Scripting, Command line automationeffort/small1 day or less: trivial logic, UI adjustments, docsstatus/bot-triaged

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions