Flyway DDL migration failures classify as unknown — the class advertises Flyway but only matches checksum errors #265

A failing Flyway DDL migration — the everyday case, where a migration runs and the SQL blows up — currently classifies as unknown, even though database_migration_failure exists and its own subsystem string advertises Flyway.

This issue was rewritten after the original was found to be wrong: it asked for a fixture and promised "no classifier changes are needed". That promise is false, and following it produced a PR that could never go green. Sorry to anyone who started on the old version — the corrected scope is below, and it's a better first issue than what it replaced.

Reproduce (measured on main, patchrail 0.6.1+)

$ cat > flyway.log <<'EOF'
2026-07-08T15:54:37.0000000Z ##[group]Run flyway migrate
2026-07-08T15:54:39.0000000Z Flyway Community Edition 10.4.1 by Redgate
2026-07-08T15:54:41.0000000Z Migrating schema "public" to version "2 - add users"
2026-07-08T15:54:41.0000000Z ERROR: Migration V2__add_users.sql failed
2026-07-08T15:54:41.0000000Z SQL State  : 42S01
2026-07-08T15:54:41.0000000Z Message    : ERROR: Table 'users' already exists
2026-07-08T15:54:41.0000000Z ##[error]Process completed with exit code 1.
EOF

$ patchrail ci explain --log flyway.log --format json | jq '.failure_class, .confidence'
"unknown"
0.15

Expected: database_migration_failure. Actual: unknown — PatchRail declines to explain a migration failure it claims to cover.

Root cause

src/patchrail/ci/classify.py:1017 advertises the class as "Database schema migration (Alembic, Django, Rails, Flyway, Prisma)", but the only Flyway pattern in the rule is src/patchrail/ci/classify.py:1026:

r"FlywayException|Migration checksum mismatch|Detected failed migration",

Those three signals fire on a checksum/validation failure. A migration whose SQL fails at runtime — the far more common case — emits none of them. Flyway prints Migration V<n>__<name>.sql failed, a SQL State : <code> block, and the driver's message. No pattern matches, so the log falls through to unknown.

The three existing fixtures for this class (db-migration-alembic-revision, db-migration-prisma-drift, gitlab-ci-rails-pending-migration) are all non-Flyway, which is why nobody caught this.

What to change

No new failure class — extend the existing rule at src/patchrail/ci/classify.py:1018-1029.

1. Add a narrow pattern (or two) to the database_migration_failure rule that matches Flyway's runtime-failure output — the Migration V…__….sql failed line is the distinctive one.
2. Add the fixture pair under examples/ci-triage/:
   • github-actions-flyway-migration-failure.log (the log above, or a real sanitized one — patchrail redact failed-ci.log > …)
   • github-actions-flyway-migration-failure.expected.json:

     { "failure_class": "database_migration_failure", "minimum_confidence": 0.7 }

     If it lands below 0.7, put the observed value in (rounded down to one decimal) and say so in the PR.

Keep the pattern narrow. A bare SQL State or a lone ERROR: would match half the logs in the corpus and start stealing failures from other classes — that is the exact bug this rule already has, in reverse. Pin the guard with a negative test, in the style of tests/test_ci_classify_expansion.py:240:

def test_sql_mentions_alone_are_not_database_migration_failure(self) -> None:
    log = "psql: connected. SQL State: 00000. All migrations already applied.\n"
    self.assertNotEqual(classify_ci_log(log)["failure_class"], "database_migration_failure")

And prove the fix actually does the work — revert your pattern, confirm the new test goes red, restore it.

Acceptance criteria

• patchrail ci explain --log examples/ci-triage/github-actions-flyway-migration-failure.log reports database_migration_failure at or above the minimum_confidence in the .expected.json
• A positive test (Flyway DDL failure classifies correctly) and a negative test (incidental SQL chatter does not) in tests/test_ci_classify_expansion.py
• Mutation check, stated in the PR: with your pattern removed, the new positive test fails
• patchrail ci fixture-check examples/ci-triage --format json passes
• patchrail ci benchmark examples/ci-triage --format json passes — top-1 accuracy stays 1.0 (221 fixtures today; you make it 222)
• uv run --extra dev pytest -q green on Python 3.12 (that's what CI runs)
• No new failure class introduced

How to submit

1. Fork, branch, make the change.
2. Run every command above and paste the ci explain + fixture-check + benchmark output into the PR as evidence.
3. Open a PR using the pull request template, with Closes #265 in the description.

New here? CONTRIBUTING.md has the setup. Comment on this issue if you get stuck or if the pattern design gives you trouble — happy to help, and reviewing this one is on me.