feat: add AWT (AI Watch Tester) — AI-powered E2E testing skill - #822

Demo

Here's a 30-second demo showing the full workflow — install, scan, run, and self-heal:

Scene 1 (0–10s): pip install aat-devqa → aat init → aat run login.yaml — 3 steps pass

Scene 2 (10–20s): AWT scans the login page, highlights the button with a red border, clicks it, and shows ✅ Login successful

Scene 3 (20–30s): Test fails → 🤖 AI analyzes → 🔧 AI patches selector → ✅ Retry passes

The skill follows the exact 4-step pattern (Scan → Generate → Run → Report) with user approval checkpoints at each stage — no autonomous actions without permission.

Updated to match AWT v1.7.0 (published to PyPI as aat-devqa 1.7.0).

What changed since the last sync

• PDF reports. aat run --report pdf|markdown writes one report per scenario under reports/<scenario id>/, and aat loop --report-format pdf does the same for a healing loop. The report is rendered as HTML and printed with the Chromium that Playwright already installs, so it adds no dependency. Screenshots of failed and warned steps are embedded as data URIs, so the file explains itself to someone who was not at the terminal.
• MCP tools. aat_run and aat_run_skill_mode take a new report argument ("" by default, or "pdf" / "markdown"). The path of each written report is printed on stdout so the calling assistant can tell the user where it landed.
• Warnings stay warnings. A run where every step ran but a click moved nothing on screen is titled PASS WITH WARNINGS on the report cover, never PASS, matching the exit code 3 the CLI already returned for that case.

The approval gate is unchanged: there is still no flag that answers the prompt on the user's behalf, and the skill instructions still require showing the scenario and getting a real "yes" before any run.

Reviewed at head 5c6af2a5 (single-file add: skills/awt/SKILL.md, +81/−0; mergeable ✓; byte-identical via API and git fetch origin pull/822/head).

• ✅ Frontmatter passes the repo's own validator (skills/skill-creator/scripts/quick_validate.py, run against these exact bytes). Control run with a mutated name is correctly rejected, so the check is real.
• ✅ Links resolve: ksgisang/AI-Watch-Tester → 200; pypi.org/project/aat-devqa → 200, latest 1.7.0 (matches your note above).
• ✅ The commands referenced here are still current: aat scan --url and aat run --skill-mode --fast both exist in src/aat/cli/commands/ (run_cmd.py still declares --skill-mode and --fast).
• ✅ No awt entry exists on main — no slug collision.

One substantive note, plus two repo-convention tips:

1. This revision looks like a stale copy of the skill. Your current public source (ksgisang/awt-skill → awt/SKILL.md) has grown to ~420 lines with references/ + templates/, adds a "When NOT to Use" section and six CRITICAL RULES, and a fuller 4-step workflow (incl. failure diagnostics like ACTUAL_CAUSE / STATUS: WARNINGS, exit code 3). Notably, current guidance says "NEVER use aat devqa — it runs the entire pipeline without user checkpoints", whereas this revision still suggests aat devqa --max-attempts 3 in its "Self-healing DevQA Loop" section, and predates the --report pdf step. If this PR is going to be merged, it should probably be re-synced from your current source.

2. Repo conventions for added skills: register it in .claude-plugin/marketplace.json — all skills merged into ./skills have an entry there, and recent add-skill PRs (Add git-commit-guidelines and pr-guidelines skills #1782, Add changelog-entry-writer skill #1783) include theirs in the same PR, e.g.:

{
  "name": "awt",
  "description": "AI-powered E2E web app testing with a self-healing DevQA loop",
  "source": "./",
  "strict": false,
  "skills": ["./skills/awt"]
}

Also, most of the other skills ship a LICENSE.txt next to SKILL.md (18 of 19 — Apache-2.0, matching this frontmatter's license: field).

No change requests beyond the above — the approval-gate posture is right for a browser-driving skill.

Synced with AWT v1.7.1.

The aat_run and aat_run_skill_mode tools gain a report_screenshots argument (failures | all | none), alongside the existing report. It controls which steps the PDF report illustrates — all is what you want when the report is meant to show that something works, since the default only embeds the screenshots of failed and warned steps.

Also fixes a defect in v1.7.0 where a PDF report was silently not printed when the reports directory was a relative path, which was the shipped default.

Thanks for the notes. I checked whether the re-sync is reflected in this PR and it doesn't appear to have landed: the branch head is still 5c6af2a5 (unchanged since my review; both commits on the branch date from March), and skills/awt/SKILL.md here is still the 81-line / 2,192-byte revision — while ksgisang/awt-skill has moved on (15,954 bytes, with commits today including the 1.7.1 registry/docs updates). If the intent was to refresh this PR from the current source, the commit doesn't seem to have been pushed to add-awt-skill; happy to re-check once it's there.