Skip to content

[Bug] OTel metrics: skills from non-official plugin marketplaces report skill.name="third-party" with no opt-out — teams lose analytics for their own skills #77541

Description

@matthewalton

Environment

  • Claude Code 2.1.209, macOS (behaviour observed across 2.1.206–2.1.209 in live telemetry)
  • OTEL metrics enabled via managed settings, exported OTLP → Grafana Cloud

What happens

On the claude_code.token.usage and claude_code.cost.usage metrics, the skill.name attribute is reported inconsistently depending on where the skill is installed from:

Skill source skill.name on metrics
Built-in / official marketplace (anthropics/claude-plugins-official) verbatim
User skills (~/.claude/skills) and project skills (repo .claude/skills) verbatim
Any other plugin marketplace — including one the team publishes itself the literal string "third-party"

The related attributes are masked on the same series, so the bucket cannot be split downstream: plugin_name is also "third-party", and MCP attribution is masked to "custom".

There is no control that changes this. OTEL_LOG_TOOL_DETAILS=1 unmasks skill names on skill_activated events only; no environment variable or managed setting affects the metrics-side mask.

Why this matters

Plugin marketplaces are the recommended way to distribute a team's shared skills. A team that follows that recommendation loses exactly the telemetry that skill.name exists to provide: every one of their skills collapses into a single "third-party" series. In our live capture, a 30-day window shows ~97% of all skill-attributed tokens (~147M of ~152M) in the one "third-party" bucket, because our own plugins (published from our own GitHub repos, installed via our own marketplace) are treated identically to genuinely third-party code.

The same skill copied into ~/.claude/skills reports verbatim — so the masking rule keys on the distribution channel, not on anything about the skill itself.

The mask is not protective

As documented in #76957 (event-side masking regression, filed with OTLP captures), api_request events in the same default export already carry skill.name verbatim. Masking the metrics therefore breaks analytics without hiding the names.

Steps to reproduce

  1. Publish a minimal plugin marketplace repo containing one skill; install and enable it.
  2. Enable OTEL metrics export (any backend; a local OTLP dump listener suffices).
  3. Invoke the skill; observe claude_code.token.usage reports skill.name="third-party", plugin_name="third-party".
  4. Copy the same skill folder into ~/.claude/skills; invoke it; observe skill.name verbatim.

Ask

Either:

  1. Report plugin skill names verbatim on cost/token metrics (matching user/project/official skills), or
  2. Provide a managed-settings control — e.g. an allowlist of trusted marketplaces or plugins whose skills report verbatim — so organisations can opt their own plugins into the telemetry they already operate.

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:pluginsarea:skillsbugSomething isn't workinghas reproHas detailed reproduction stepsreproducedBug reproduced by maintainers on a released build

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions