Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion mods/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ source, published as it is built into the binary.
| --- | --- | --- |
| [`sec-default`](sec-default) | Keeps an organization's classic hooks, prompt content, managed settings and tool policy out of reach of the plugins a person installs; adds no policy of its own. | Outermost, on a machine with managed settings or for a Team or Enterprise organization, unless managed `prependPlugins` says otherwise |
| [`diff`](diff) | `/diff`: the session's uncommitted changes in a pane beside the transcript, file by file with their hunks, refreshed as Claude edits files and runs commands. | Built in |
| [`telemetry`](telemetry) | Adds `$.telemetry` (`log`, `mark`) in the `engine.create` fold so a built-in plugin can record an event as a first-party analytics row, sent in batches; refuses installed plugins; sends nothing wherever Claude Code's analytics are off. | Built in |
| [`telemetry`](telemetry) | Hooks `$.telemetry`'s two events (`log`, `mark`), adding the noun in the `engine.create` fold where the engine has none, so a built-in plugin can record an event as a first-party analytics row, sent in batches; refuses installed plugins; sends nothing wherever Claude Code's analytics are off. | Built in |
| [`agents-md`](agents-md) | `AGENTS.md` as project instructions, by one option: loaded where the project has no `CLAUDE.md` of its own (`claude-md-or-agents-md`, the default) or beside it (`claude-md-and-agents-md`), placed and framed exactly as the engine places `CLAUDE.md`, nested ones on a `Read`; or the project's and the person's instruction files dropped and the organization's kept (`managed-only`); or `CLAUDE.md` alone, as the engine reads it (`claude-md`). | Built in |

Each folder is a complete plugin: `.claude-plugin/plugin.json`, a
Expand Down
2 changes: 1 addition & 1 deletion mods/telemetry/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "telemetry",
"version": "0.1.0",
"description": "Plugin analytics: adds $.telemetry in the engine.create fold, so a plugin logs an event or marks a feature's use as a first-party row, sent in batches with the session's own credential.",
"description": "Plugin analytics: hooks $.telemetry's two events, so a plugin logs an event or marks a feature's use as a first-party row, sent in batches with the session's own credential.",
"author": {
"name": "Anthropic"
},
Expand Down
34 changes: 23 additions & 11 deletions mods/telemetry/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,16 @@
# telemetry

Plugin analytics as a plugin: one `engine.create` step adds `$.telemetry` to
the engine interface every plugin above it is handed, built over the nouns
beneath, and a hook on its own two events serves the plugins built into
Claude Code alone: a call from a plugin a person installed or an
Plugin analytics as a plugin: its hooks on `telemetry.log` and
`telemetry.mark` are what those two events do, built over the nouns its
`engine.create` step is handed, and a gate above them serves the plugins
built into Claude Code alone: a call from a plugin a person installed or an
administrator listed is refused with a reason (the host stamps every call
with the plugin that raised it, `next.origin`, and the gate reads its tier).
On an engine that has no `$.telemetry` of its own the same step adds the
noun, so the calls exist there too. An entry names where it goes with `to`:
`anthropic`, the default, is this mod's; one for `collector`, the telemetry
collector an operator configured, is passed on beneath untouched, and `to`
is never part of a row.
`$.telemetry.log({ event, props })` queues one event as one first-party
row, `tengu_plugin_<event>`; `$.telemetry.mark({ feature, kind, reason?,
props? })` marks one use of a feature as the CLI's own feature events do,
Expand Down Expand Up @@ -59,11 +64,17 @@ noun and a test answering it all read.

## What it hooks

`engine.create`: `{ ...await next(e), telemetry }`, so the noun is added and
nothing beneath is replaced. `telemetry.*`, the gate: a caller in the
built-in tier (or the engine) goes on, any other is refused, and a gate
that throws refuses too. `session.start`, to learn whether a person is at
the prompt; `session.end`, to send what still waits.
`telemetry.*`, the gate: a caller in the built-in tier (or the engine) goes
on, any other is refused, and a gate that throws refuses too.
`telemetry.log` and `telemetry.mark`, beneath the gate: the entry is checked
and its row queued, the hook answering `{ value }`, or `{ deny }` with the
reason for an entry that breaks a rule, so the caller's promise rejects
naming it. `engine.create`: the sender is built over `await next(e)`, and
the step hands up `{ ...{ telemetry }, ...beneath }`: what is beneath is
spread last, so its own `telemetry` stands where it has one and this mod's
is added where it has none; nothing beneath is replaced.
`session.start`, to learn whether a person is at the prompt; `session.end`,
to send what still waits.

## What it calls on `$`

Expand All @@ -81,5 +92,6 @@ are on, and nowhere else; it serves the plugins bundled with the CLI and
refuses every other caller. It is not meant to be installed or loaded with
`--plugin-dir`; the folder has a manifest so it reads like every other
plugin, not so it can stand alone. A built-in that calls `$.telemetry`
where this one is absent finds no such noun and should treat that as "no
analytics here".
where this one is absent finds no such noun, or, on an engine with the noun
of its own, one whose calls queue nothing; either way that is "no analytics
here".
24 changes: 24 additions & 0 deletions mods/telemetry/hooks/answer-of/answer-of.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import Entries from '../entries'

/**
* A row's queueing as the hook on its event answers it: `{ value }` once
* the row is queued, and a refused entry as `{ deny }` with its reason, so
* the caller's promise rejects naming what was wrong.
*
* Anything else that went wrong is thrown on: the hook failed, and the
* engine goes on beneath it as it does for any failed hook.
*
* @param queued the queueing, settled once the entry is checked and queued
* @returns the hook's answer
*/
export const answerOf = (queued: Promise<void>) =>
queued.then(
() => ({ value: undefined }),
(error: unknown) => {
if (Entries.isRefusedEntry(error)) {
return { deny: error.what }
}

throw error
},
)
3 changes: 3 additions & 0 deletions mods/telemetry/hooks/answer-of/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
export * from './answer-of.js'

export * as default from '.'
2 changes: 2 additions & 0 deletions mods/telemetry/hooks/entries/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,14 @@ export * from './fields-of'
export * from './ingest-url.js'
export * from './is-mark-kind'
export * from './is-record'
export * from './is-refused-entry'
export * from './mark'
export * from './mark-fields-of'
export * from './mark-kinds'
export * from './method'
export * from './prop-limit'
export * from './refusal'
export * from './refused-entry'
export * from './token'
export * from './wire-of'

Expand Down
3 changes: 3 additions & 0 deletions mods/telemetry/hooks/entries/is-refused-entry/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
export * from './is-refused-entry.js'

export * as default from '.'
11 changes: 11 additions & 0 deletions mods/telemetry/hooks/entries/is-refused-entry/is-refused-entry.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
import type { RefusedEntry } from '../refused-entry'

/**
* Whether an error is an entry's refusal (`refusal`), and so carries the
* reason to deny with, rather than something that went wrong on the way.
*
* @param error what a check threw
* @returns whether it is a refused entry's error
*/
export const isRefusedEntry = (error: unknown): error is RefusedEntry =>
error instanceof Error && 'what' in error && typeof error.what === 'string'
8 changes: 5 additions & 3 deletions mods/telemetry/hooks/entries/refusal/refusal.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
import type { Method } from '../method'
import type { RefusedEntry } from '../refused-entry'

/**
* The error a refused entry rejects with, naming the method and what was wrong.
* The error a refused entry rejects with, naming the method and what was
* wrong, the reason kept beside the message as `what`.
*
* The text carries a key that passed TOKEN or a status code, nothing the caller
* wrote as free text.
Expand All @@ -11,5 +13,5 @@ import type { Method } from '../method'
* @returns the error to reject the call with, naming the method and what was
* wrong
*/
export const refusal = (what: string, method: Method = 'log'): Error =>
new Error(`$.telemetry.${method}: ${what}`)
export const refusal = (what: string, method: Method = 'log'): RefusedEntry =>
Object.assign(new Error(`$.telemetry.${method}: ${what}`), { what })
3 changes: 3 additions & 0 deletions mods/telemetry/hooks/entries/refused-entry/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
export type * from './refused-entry.js'

export * as default from '.'
6 changes: 6 additions & 0 deletions mods/telemetry/hooks/entries/refused-entry/refused-entry.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
/**
* The error a refused entry rejects with: its message names the method and
* what was wrong, and `what` is that reason alone, for a hook to answer as
* its `{ deny }`.
*/
export type RefusedEntry = Error & { readonly what: string }
2 changes: 1 addition & 1 deletion mods/telemetry/hooks/hooks.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"description": "Plugin analytics: an engine.create step adds $.telemetry (log, mark) over $.session, $.env, $.settings, $.fs, $.process, $.clock and $.http beneath; rows go out in batches; sends nothing wherever the CLI's analytics are off",
"description": "Plugin analytics: hooks on telemetry.log and telemetry.mark queue first-party rows over $.session, $.env, $.settings, $.fs, $.process, $.clock and $.http beneath, the noun added in engine.create where the engine has none; rows go out in batches; sends nothing wherever the CLI's analytics are off",
"modules": [
"./register.ts"
]
Expand Down
1 change: 1 addition & 0 deletions mods/telemetry/hooks/index.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
export * from './answer-of'
export * from './batching'
export * from './config-location'
export * from './context'
Expand Down
27 changes: 21 additions & 6 deletions mods/telemetry/hooks/register.ts
Original file line number Diff line number Diff line change
@@ -1,18 +1,22 @@
import type { EngineInterface, On } from 'claude-code'

import { answerOf } from './answer-of'
import Context from './context'
import Gate from './gate'
import IsAnalyticsOff from './is-analytics-off'
import type { Sender } from './sender'
import { telemetryOf } from './telemetry-of'

/**
* Registers the plugin's hooks: its engine.create step adds `$.telemetry`
* over the nouns beneath; session.start and session.end feed and flush it.
* Registers the plugin's hooks: the gate on `telemetry.*`, the two hooks
* that are what `telemetry.log` and `telemetry.mark` do, and its
* engine.create step, which builds the sender over the nouns beneath.
*
* `log` and `mark` queue a row for a built-in caller and refuse any other;
* a gate that throws refuses too, a refusal from beneath stays as it is. A
* batch goes out on a timer, when full, and when the session ends.
* A row is queued for a built-in caller and refused to any other; a
* malformed entry is denied with its reason; an entry for `collector` is
* not this plugin's and goes on beneath. On an engine with no `telemetry`
* of its own the step adds the noun, its methods the same queueing. A batch
* goes out on a timer, when full, and when the session ends.
*
* @param on the engine's registrar
*/
Expand All @@ -24,6 +28,16 @@ export function register(on: On) {
(_$, e, next) => Gate.caught(e, next),
)

on('telemetry.log', (_$, e, next) =>
sender === undefined || e.to === 'collector'
? next(e)
: answerOf(sender.telemetry.log(e)),
)

on('telemetry.mark', (_$, e, next) =>
sender === undefined ? next(e) : answerOf(sender.telemetry.mark(e)),
)

on('session.start', (_$, e, next) => {
isInteractive = e.isInteractive

Expand Down Expand Up @@ -348,7 +362,8 @@ export function register(on: On) {
})

const telemetry: EngineInterface['telemetry'] = sender.telemetry
const added = { telemetry }

return { ...beneath, telemetry }
return { ...added, ...beneath }
})
}
22 changes: 22 additions & 0 deletions mods/telemetry/tests/fixtures/plugins/annotating.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
import type { Plugin } from 'claude-code/testing'

/**
* A plugin a person installed whose one hook is on every event: it sends
* each `telemetry.log` entry on beneath with a note of its own beside the
* entry's fields, and the rest untouched.
*/
export const annotating: Plugin = {
name: 'annotating',
tier: 'user',
register(on) {
on('*', (_$, e, next) => {
if (!next.is('telemetry.log', e)) {
return next(e)
}

const noted = { ...e, note: 'what the person typed at the prompt' }

return next(noted)
})
},
}
2 changes: 2 additions & 0 deletions mods/telemetry/tests/fixtures/plugins/index.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
export * from './annotating.js'
export * from './holding.js'
export * from './managing.js'
export * from './marking.js'
Expand All @@ -6,6 +7,7 @@ export * from './reaching.js'
export * from './recording.js'
export * from './replacing.js'
export * from './swallowing.js'
export * from './sweeping.js'
export * from './visiting.js'

export * as default from '.'
22 changes: 22 additions & 0 deletions mods/telemetry/tests/fixtures/plugins/sweeping.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
import type { Plugin } from 'claude-code/testing'

/**
* A plugin a person installed whose one hook is on every event: it sends
* each `telemetry.log` entry on beneath turned to the other destination,
* and the rest untouched.
*/
export const sweeping: Plugin = {
name: 'sweeping',
tier: 'user',
register(on) {
on('*', (_$, e, next) => {
if (!next.is('telemetry.log', e)) {
return next(e)
}

const turned = { ...e, to: 'collector' }

return next(turned)
})
},
}
56 changes: 56 additions & 0 deletions mods/telemetry/tests/gate.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -124,4 +124,60 @@ describe('gate', () => {
})
},
)

test(
'a hook on every event cannot turn a row to the other destination',
{ plugins: [Fixtures.recording, Fixtures.sweeping] },
async ($, on) => {
mock.env(on, Fixtures.SENDING_ENV)

const session = Fixtures.firstPartySession(on)

await $.session.start(Fixtures.STARTED)

const answer = (
await $.command.run(Fixtures.record(Fixtures.surveyAnswer()))
).text

await session.clock.advance(Hooks.BATCH_WINDOW_MS)

expect({
answer,
posts: session.posts.length,
rows: Fixtures.rowsOf(session),
}).toEqual({
answer: 'queued',
posts: 1,
rows: [Fixtures.EXPECTED_ROW],
})
},
)

test(
'what a hook on every event adds beside a row never leaves in it',
{ plugins: [Fixtures.recording, Fixtures.annotating] },
async ($, on) => {
mock.env(on, Fixtures.SENDING_ENV)

const session = Fixtures.firstPartySession(on)

await $.session.start(Fixtures.STARTED)

const answer = (
await $.command.run(Fixtures.record(Fixtures.surveyAnswer()))
).text

await session.clock.advance(Hooks.BATCH_WINDOW_MS)

expect({
answer,
posts: session.posts.length,
rows: Fixtures.rowsOf(session),
}).toEqual({
answer: 'queued',
posts: 1,
rows: [Fixtures.EXPECTED_ROW],
})
},
)
})
31 changes: 31 additions & 0 deletions mods/telemetry/tests/register.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -496,6 +496,37 @@ describe('register', () => {
},
)

test(
'a refused entry is denied by the hook, naming the caller and the reason',
{ plugins: [Fixtures.recording, Fixtures.marking] },
async ($, on) => {
mock.env(on, Fixtures.SENDING_ENV)

const session = Fixtures.firstPartySession(on)

const logged = (
await $.command.run(Fixtures.typed('record', { event: 'Survey' }))
).text

const marked = (
await $.command.run(
Fixtures.typed('mark', { feature: 'learn_page', kind: 'meh' }),
)
).text

await session.clock.advance(Hooks.BATCH_WINDOW_MS)

expect({ logged, marked, posts: session.posts }).toEqual({
logged:
'HooksError: recording: $.telemetry.log: takes an event name, a ' +
'snake_case token',
marked:
"HooksError: marking: $.telemetry.mark: kind: 'ok', 'sad' or 'bad'",
posts: [],
})
},
)

test(
'a number that is not finite is refused, nothing queued',
{
Expand Down
Loading
Loading