verification/README.md · ComfyUI Higgsfield Opensource Challenge

Verification

What in this entry has been mechanically checked, and what has not.

The rule for this directory: a claim lives here only if a command in this file reproduces it. Anything else is marked UNVERIFIED.

1. The port is faithful to the original Go implementation

Claim: the Python engine in app/cinematographer/ reproduces the original Go cinematography implementation value-for-value.

Status: VERIFIED — 289 differential checks, zero disagreements.

How it was checked

The original implementation lives in the Magieva monorepo at internal/contentstudio/cinematography*.go. A temporary Go harness (go-harness/zz-cine-dump.go) was compiled inside that module, ran the original Go functions over a fixed fixture set, and dumped the results as JSON. The Python port was then run over the same fixtures and compared field-by-field.

# 1. produce the Go side (from the magieva-core checkout)
cp verification/go-harness/zz-cine-dump.go cmd/zz-cine-dump/main.go
GOWORK=off go run ./cmd/zz-cine-dump > /tmp/go-cine-dump.json
rm -rf cmd/zz-cine-dump

# 2. compare against the Python port
python verification/differential_check.py /tmp/go-cine-dump.json

The captured Go output is preserved at go-reference-dump.json, so the comparison can be re-run without the Magieva checkout:

python verification/differential_check.py verification/go-reference-dump.json
# -> differential checks run: 289
# -> ALL FIXTURES AGREE with the original Go implementation.

What is compared

Surface	Compared
`EvaluateFraming` — 9 fixtures	rule list, order, pass, value, weight, target, correction, overall_pass
`AutoFrameCamera` — 18 fixtures (3 shot types × 6 intents)	camera eye, target, fov, roll, and the full report
`EvaluateContinuity` — 2 sequences	rule, shot pair, pass
`SceneFramingExtras` — 2 scenes	rule, pass, value

Tolerance is 1e-6 on every numeric field.

What this does not cover

size_importance's absolute value. The Go original derives entity half-extents from Magieva's full dimension resolver (ResolveEntityDimensions); this port uses documented role-derived defaults (app/cinematographer/scene.py). The comparison the rule makes is the same; the absolute ratio is not. The differential check compares this rule's pass/fail but deliberately skips its value, and the code says so.
The early-exit fix. That is a deliberate behavioural divergence, not a fidelity claim — see §2. The differential check runs the port with early_exit=False to prove fidelity, then separately asserts the fix engages.
AutoFrameGroup (the two-subject group framer) is not ported.

2. The convergence fix does what it claims

Claim: the port exits early on success, reports converged honestly, and never claims a pass the report does not support.

Status: VERIFIED — unit tests.

cd app && python -m pytest tests/test_autoframe.py -q

Test	What it pins
`test_solver_exits_early_rather_than_always_running_24_iterations`	`iterations < 24` on a solvable frame — the headline divergence from Go
`test_converged_flag_agrees_with_the_report`	`converged == report.overall_pass` across 9 combinations
`test_unsolvable_intent_is_reported_honestly_not_hidden`	a degenerate subject returns `converged: False` with a named residual
`test_solved_pose_reproduces_a_passing_report_when_re_evaluated`	the returned pose actually passes when fed back through the evaluator

On the reference fixture the solver converges in 3 iterations; the Go original would run all 24 and return the 24th unconditionally.

3. The rule engine is self-consistent

Claim: all 14 rules produce a measured value and a correction lever, and the overall verdict follows the documented weight rule.

Status: VERIFIED — 69 unit tests, every expected value hand-computed.

cd app && python -m pytest tests/ -q     # 69 passed

The tests derive their expectations from first principles rather than from a recorded run. For the reference camera (35 mm focal, 36 mm sensor, 16:9):

vfov = 2*atan((36/(16/9)) / (2*35))
f    = (2*focal)/(sensor/aspect) = 70/20.25 = 3.45679012345679

so ndcY(top) = 0.048 * f = 0.1659259 and headroom = 0.8340750. The suite asserts those numbers, not a snapshot.

4. UNVERIFIED — read before trusting

Claim	Status
The workflow graphs run on a live ComfyUI	UNVERIFIED — no ComfyUI instance was available. Each graph's `_README` node states its own status.
The panel's numbers match the engine	VERIFIED — `python app/web/parity_check.py` → `RESULT: PASS — 28 fixtures, 0 mismatches`. See `app/web/README.md`.
The reliability envelope (±45° azimuth)	Documented model property, taken from CrossView-Warp's published trained range. Not independently measured here.
LTX 2.3 + CrossView-Warp quality	UNVERIFIED — no render was produced.
The competition's judging criteria	Not published. No rubric exists to verify against.

5. Reproducing everything

cd app && python -m pytest tests/ -q                                   # 69 tests
cd .. && python verification/differential_check.py \
            verification/go-reference-dump.json                        # 289 checks

Both must exit 0. If either fails, the corresponding claim above is void.