skill-creator: don't early-return run_eval detection on non-Skill/Read tools - #1208

Summary

run_single_query() in skill-creator/scripts/run_eval.py early-returns False the moment Claude's first tool_use event is anything other than Skill or Read. When the query needs any exploration (Bash/LS/Glob/TodoWrite to figure out which file/skill the user means), the detector misses the eventual Skill invocation that follows — silent false-negative.

This is the third independently-traced bug in #556's code path:

1. Worker-UUID mismatch — fixed by skill-creator: fix parallel worker false negatives in run_eval.py #794 (rhys-childs)
2. Architectural: .claude/commands/ not surfaced in claude -p mode — fixed by fix(skill-creator): swap real SKILL.md instead of temp .claude/commands #1027 (john-savepoint)
3. This PR: early-return on non-Skill/Read tools — surfaces once fix(skill-creator): swap real SKILL.md instead of temp .claude/commands #1027 makes triggering actually possible

Under the current main code path the early-return is moot (the skill is never surfaced anyway, per #1027's analysis). Under #1027 the skill IS surfaced, Claude can invoke it, but only queries whose first tool call happens to be Skill or Read are detected. Queries that need any exploration first remain silently false-negative.

Change

Replace return False with state-clear, let the loop's natural terminator (message_stop event or timeout) decide:

```diff

• if tool_name in ("Skill", "Read"):

• pending_tool_name = tool_name
  

• accumulated_json = ""
  

• else:

• return False
  

• if tool_name in ("Skill", "Read"):

• pending_tool_name = tool_name
  

• accumulated_json = ""
  

• else:

• # Don't early-return — Claude may use exploration tools
  

• # (Bash, LS, Glob, TodoWrite, etc.) before invoking the
  

• # skill. Clear any in-progress Skill/Read state so this
  

• # non-skill tool's input deltas don't get confused with
  

• # a prior Skill invocation.
  

• pending_tool_name = None
  

• accumulated_json = ""
  

```

Why state-clear (not just remove)

The downstream content_block_delta handler at lines 143–148 keys off pending_tool_name. If a Skill tool_use started, accumulated some input deltas, then a non-Skill tool_use started, we'd want the delta processor to stop appending to accumulated_json (it now belongs to a different tool). Setting pending_tool_name = None and accumulated_json = "" cleanly separates the state per tool_use boundary.

Test plan

• Run python -m scripts.run_eval --skill-path <skill> --eval-set <eval.json> against a query Claude would naturally explore for first (e.g. "audit my latest skill") — should now correctly observe the subsequent Skill invocation rather than returning False on the exploration tool.
• Verify a should-not-trigger query that uses many exploration tools without invoking the skill still returns False (via timeout or message_stop, not via early-return).
• Both should work cleanly when merged on top of fix(skill-creator): swap real SKILL.md instead of temp .claude/commands #1027 (the architectural fix that makes triggering observable at all).

Refs

• run_eval.py: claude -p never triggers skills/commands (0% trigger rate across all queries) #556 — bug report (added a comment with reproduction data)
• fix(skill-creator): swap real SKILL.md instead of temp .claude/commands #1027 — architectural fix this PR sits on top of

🤖 Generated with Claude Code