ComfyUI Higgsfield Opensource Challenge Verification ← Build log

Verification

Boundary evidence — closing the "12 unconstrained constants" finding

Status: CLOSED. The adversarial review's MAJOR/systemic finding was that 12 engine constants were constrained by no check, because the 9 framing fixtures gave in_frame and level exactly one distinct value each — nothing sat near a boundary.

The fix is app/tests/test_boundaries.py: one fixture per threshold, each straddling the constant so a move in either direction is caught. The proof that the fixtures actually constrain the constants is a mutation harness, not the tests passing.

Mutation harness

verification/mutate_boundaries.py copies app/ to a temp dir, moves one constant, and runs tests/test_boundaries.py against the mutant. A mutation that the suite still passes is a constant the suite does not constrain.

$ /tmp/cine-venv/bin/python verification/mutate_boundaries.py

mutation                           verdict  detail
------------------------------------------------------------------------------
thirds tol 0.12 -> 0.13            CAUGHT   1 failed in 0.08s
centred tol 0.12 -> 0.5            CAUGHT   1 failed, 2 passed in 0.15s
in_frame |ex|<=1 -> <=2            CAUGHT   1 failed, 3 passed in 0.04s
level <1 -> <2                     CAUGHT   1 failed, 5 passed in 0.07s
lead_room -0.05 -> -0.04           CAUGHT   1 failed, 21 passed in 0.19s
thirds line 1/3 -> 1/4             CAUGHT   1 failed in 0.12s
MIN_DISTANCE 0.3 -> 0.31           CAUGHT   1 failed, 25 passed in 0.14s
MIN_ANGLE_CHANGE 30 -> 29          CAUGHT   1 failed, 27 passed in 0.13s
OVERALL_PASS_WEIGHT 0.8 -> 0.5     CAUGHT   1 failed, 23 passed in 0.11s
------------------------------------------------------------------------------
9/9 mutations caught by tests/test_boundaries.py

Exit code 0 when every mutation is caught, 1 otherwise — so this is a gate, not a report.

Two fixtures that had to be re-derived

Both were wrong on the first attempt, and the mutation harness is what caught them.

  1. camera_at_ndc_x had the wrong scale and sign. The projector computes ndcX = (cx / cz) * (f / aspect), so the horizontal scale is f / aspect, not f. And for a level camera looking down -Z, right = cross(fwd, up) = (-1, 0, 0), so a world +X offset projects to negative ndcX. The helper now moves the eye to -ndc_x * dist / (F / aspect).
  1. The lead_room fixture sat at exactly the mutated threshold. It used ndcX 0.04 (lead -0.04) and 0.06 (lead -0.06). Moving the constant to -0.04 left -0.04 >= -0.04 true, so the mutation was MISSED. The fixture now straddles at -0.045 / -0.055, which catches a move in either direction.

This is the finding in miniature: a fixture that looks like it tests a boundary can still be blind to the mutation it was written for. Only running the mutation shows it.

What this does and does not establish

Does: the 9 constants listed above cannot move without a test failing. The suite is a real constraint on them, demonstrated by mutation rather than asserted.

Does not: the remaining constants named in the adversarial review are not all covered by this harness. shot_size band edges and headroom band edges are pinned by test_boundaries.py assertions but were not mutation-tested here; the continuity axis sign flip and ang_delta > 180 -> > 179 are pinned by assertions only. Extending the harness to those is the honest next step, and until it is done they should be described as assertion-pinned, not mutation-proven.

Reproduce

cd app && /tmp/cine-venv/bin/python -m pytest tests/ -q          # 100 passed
/tmp/cine-venv/bin/python verification/mutate_boundaries.py      # 9/9 caught, exit 0