Skip to content

Stop a cache-warning handoff from reading dune's make test as C/C++ - #385

Merged
PabloCodes7 merged 1 commit into
mainfrom
fix/dune-make-test-not-cpp
Jul 17, 2026
Merged

PabloCodes7 merged 1 commit into
mainfrom
fix/dune-make-test-not-cpp

Conversation

@PabloCodes7

Copy link
Copy Markdown
Contributor

What

A real failed CI run of ocaml/dune (run 29585452292) was classified cpp_build_failure 0.53 — but nothing in the log is a C/C++ compile. dune's own cram/blackbox tests fail through make test (a test-case diff, make: *** [test] Error 1).

Why the Crystal fix missed it

The Crystal fix demotes the bare GNU make recipe line while cpp_build_failure is the best match — but that deferral never runs if a benign warning first outscores it. The same dune job twice logs the benign cache save-warning Failed to save: Unable to reserve cache …, giving artifact_or_cache_failure two matched signals to cpp's one. So the make-recipe deferral was skipped, and the downstream benign-warning handoff — which yields to "the best rule that saw a real error" — then resurrected that same make-only cpp_build_failure as the verdict.

The fix

The benign-warning handoff now refuses to hand a verdict to a rule that is itself ambiguous-only (the bare make recipe or the RSpec-style summary line — the lines that name no ecosystem). The save-warning stays best and the run settles on unknown, the honest ceiling. Resource/network symptoms name their own category and stay eligible, so a buildkite job OOM-killed at exit code 137 still lands runner_resource_exhaustion; a genuine C/C++ build trips a real toolchain signal and is untouched.

Verification

  • examples/real-world/dune-29585452292-excerpt.log: cpp_build_failure 0.53 → unknown 0.15
  • Regression test tests/test_dune_make_not_cpp.py (incl. a guard that a real gcc error beside the same cache warning still classifies cpp_build_failure)
  • Zoo benchmark unchanged: 223/223 top-1, and all 21 prior real-world logs unchanged (buildkite OOM fixture still runner_resource_exhaustion 0.71)
  • Benchmark doc + CHANGELOG updated

The Crystal fix demotes the bare GNU make recipe line while cpp_build_failure
is the best match, but that deferral never runs if a benign warning first
outscores it. ocaml/dune's CI run 29585452292 fails its own cram/blackbox
tests through make test (a test-case diff, no C/C++ error), while the same job
twice logs the benign cache save-warning 'Failed to save: Unable to reserve
cache', giving artifact_or_cache_failure two matched signals to cpp's one. The
make-recipe deferral was skipped, and the downstream benign-warning handoff
then resurrected that same make-only cpp_build_failure as the verdict.

The handoff now refuses to hand a verdict to a rule that is itself
ambiguous-only (the bare make recipe or the RSpec-style summary line), so the
save-warning stays best and the run settles on unknown. Resource/network
symptoms name their own category and stay eligible (a buildkite OOM at exit
code 137 still lands runner_resource_exhaustion); a genuine C/C++ build trips a
real toolchain signal and is untouched.

Adds examples/real-world/dune-29585452292-excerpt.log, a regression test, a
benchmark row, and a CHANGELOG entry.
@PabloCodes7
PabloCodes7 merged commit 52c56a2 into main Jul 17, 2026
5 checks passed
@PabloCodes7
PabloCodes7 deleted the fix/dune-make-test-not-cpp branch July 17, 2026 16:11
@PabloCodes7 PabloCodes7 mentioned this pull request Jul 21, 2026
PabloCodes7 added a commit that referenced this pull request Jul 21, 2026
Cut 0.7.4 from main. The last release, 0.7.3 (2026-07-16), predates ten
false-positive fixes that are sitting on main and not on PyPI: an installed
pytest read as a failing test run (#376), a passing test's title read as a
secrets failure (#380), a PHPUnit assertion read as a Composer failure (#378),
Flutter's cached Gradle Wrapper (#382), Cabal's dependency resolution read as
Maven (#383), Crystal and dune's make targets read as C/C++ (#384, #385), a
parenthesized 504 (#391), mypy in a pixi manifest (#390), an unset TERM read as
a missing secret (#392), a warning-only yarn install and a recovered checkout
(#393), and a verdict held up by invocations alone reporting diagnosis-level
confidence (#395). Every one of those is a wrong answer a maintainer gets today
from pip install patchrail.

Version bumped in the four places that spell it out (pyproject, __init__,
README quickstart, uv.lock), CHANGELOG's Unreleased section dated, and the
real-world benchmark's release-status paragraph corrected: it no longer claims
six fixes are unreleased.

Co-authored-by: PabloCodes7 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant