feature/float_ms_ssim: reject frames below 176 in either dimension at init - #1637

float_ms_ssim cannot score a frame below 176 pixels in either dimension. It fails in every extract() call with error: scale below 1x1! and a generic warning, instead of saying why at init. This moves the check to init() and names the limit. Addresses #1414.

Cause

float_ms_ssim calls compute_ms_ssim() with default arguments: SCALES = 5 (iqa/ssim_tools.h) and the 11-tap Gaussian window (GAUSSIAN_LEN). compute_ms_ssim() (ms_ssim.c, "make sure we won't scale below 1x1") halves width and height with integer division and needs at least GAUSSIAN_LEN pixels in both dimensions at each of the 5 scales. The smallest frame is therefore GAUSSIAN_LEN << (SCALES - 1) = 176 in both dimensions. init() did not check this, so the failure came at the first frame.

Reproducer

Master 6ec23e8f2 (reproduced there; the code is unchanged in 9e48141), release build, 176x144 (QCIF-sized) 4:2:0 input:

meson setup build libvmaf --buildtype release
ninja -C build
build/tools/vmaf -r q176x144.yuv -d q176x144.yuv -w 176 -h 144 -p 420 -b 8 \
    --feature float_ms_ssim -q --json -o out.json

Fix

init() returns -EINVAL and logs the frame size and the minimum when w or h is below GAUSSIAN_LEN << (SCALES - 1). The constant comes from the two existing macros (float_ms_ssim.c now includes iqa/ssim_tools.h, as ssim.c and ms_ssim.c do), so it follows if either changes.

This does not fall back to fewer scales for small frames: that would change what the number means. It also does not make small frames work; the report asks whether to reduce the number of levels or to reject, and this takes the second option. If a reduced-scale mode is preferred, that is a separate change.

The process exit status for the failing run changes from 1 to 234 (-EINVAL as an unsigned byte), because the error from init() is now what vmaf returns.

Tests

New test_float_ms_ssim_minimum_dimension in libvmaf/test/test_feature_extractor.c: init fails with -EINVAL for 176x144, 175x176 and 176x175, and succeeds for 176x176. Against master's float_ms_ssim.c with the new test, the case fails ("float_ms_ssim init should reject a frame below 176 in either dimension"); with this change test_feature_extractor passes 5/5.

Validation

x86-64 Linux, GCC 16.2.1, on master 9e48141b. Rebased on master 9e48141 (2026-10-02).

• Release build meson test (-Denable_float=true -Denable_checkasm=true): 25/25 on master, 25/25 here. -Db_sanitize=address,undefined -Db_lto=false build: 22 pass and 3 fail on master, 22 pass and 3 fail here; test_predict and test_pic_preallocation fail on LeakSanitizer reports and checkasm aborts on a heap-buffer-overflow in adm_dwt2_16 (integer_adm.c:2603 on master), on both, and this change does not touch them.
• Scores unchanged: the three Netflix reference pairs, run with --feature float_ms_ssim added (1920x1080 and 576x324, all above the minimum), give identical per-frame and pooled output on master and on this branch, with the default CPU dispatch and with SIMD masked off. No golden assertion changes.

The workflow run on this PR needs a maintainer's approval.