The plan and the build record
Higgsfield × ComfyUI — what's being built
Prepared: 2026-10-29 · v2 (corrected after the competition-brief lane landed) Event: Comfy Org — Open Source Higgsfield · LA Build Night, in partnership with AI On The Lot Submission deadline: 2026-10-30, 09:00 PT — ≈15.5 hours from this document Status: the breakdown the operator asked for, before building starts.
v2 changes the picture in four places. The event's own premise is false; the "four inspiration prompts" are three cards; the licence landscape contains a trap that disqualifies the obvious build; and the obvious rebuilds are already taken. §1 and §5 are rewritten. The entry thesis survives all four — and is sharper for it.
0. The 60-second version
Comfy Org is running a competition inviting the community to rebuild Higgsfield's most popular apps with ComfyUI. Three app cards are named: Cinematic Studio, Re-shoot a video, Openjutsu.
We are entering with the one thing nobody else has: a cinematographer that measures.
Magieva has already built, in production Go, a 38-rule machine-applicable cinematography rulebook where each rule returns a measured value, a pass/fail, and the camera lever that fixes it — plus a 24-iteration geometric camera solver, a sequence-level continuity engine (the 180° line, the 30° rule), and a metric 3D scene layer with real optics.
Ported into ComfyUI as a custom node pack, that is a genuinely novel contribution to the ecosystem — which is precisely how the Most Innovative category is defined ("novel custom node packs or uses of ComfyUI").
One-line thesis: Every other re-shoot app asks you to aim a camera and hope. Ours tells you the frame is wrong, by how much, and which way to move — before you spend the render.
1. What Comfy Org is actually asking for — corrected
1.1 The premise is false
The event page says "Higgsfield said they'd open source their platform. Let's help." Higgsfield never open-sourced its platform, and no source shows it ever promised to. The September 2026 viral claim conflated higgsfield-ai/higgsfield — an Apache-2.0 LLM-training/GPU-orchestration framework, now archived — with the commercial studio. Higgsfield's eleven public repos are thin API clients or unrelated. The "open source Higgsfield studio" repos dominating search are third-party.
This matters for positioning, not for legality: there is nothing to fork. Cinema Studio 4.0 runs on Seedance 2.5 (ByteDance) per Higgsfield's own help centre; every other Higgsfield-branded model is closed with unpublished architecture. A rebuild is a from-scratch reimplementation of the product surface on open models.
1.2 The "four inspiration prompts" are three cards — and they are Comfy's own apps
The event page presents three app cards, each a Comfy-Org MIT reference app, not a Higgsfield product:
| Card | Tagline | Reference repo | How it actually works |
|---|---|---|---|
| Cinematic Studio | "Direct your shot like a film set" | Comfy-Org/comfy-cinematic-studio (MIT, ~10★) | Next.js over bfl/flux-2-pro via Comfy Router. It does not submit a ComfyUI workflow at all — the "director's panel" is prompt composition, not conditioning |
| Re-shoot a video | "Re-shoot from any angle" | Comfy-Org/comfy-reshoot (MIT, ~6★) | Next.js over a Comfy deployment. MoGe + CrossView-Warp + MiniMax H3, 96 GB GPU. Orbit/dolly UI with a green/yellow/red reliability zone |
| Openjutsu | "Swap a character in a video" | Comfy-Org/comfy-openjutsu (MIT, ~53★) | Next.js over a Comfy deployment. Akatz H3 character-swap workflow. Optional anchor calls GPT Image 2.5 through Router — a closed API in the critical path |
There is no fourth app. There is no Higgsfield app called "Re-shoot" (that is an unrelated iOS app). The event page's own copy calls these "inspiration prompts for your entry" — build something like this, not rebuild this.
The single most useful fact in this table: Comfy's own Cinematic Studio reference app is prompt composition over a hosted model. It does not condition on camera, lens or light. The prior internal report found Higgsfield's own claim to condition on those panels is contested by a control experiment. Nobody in this competition has actually built the conditioning. That is the gap.
1.3 Rules, prizes, mechanics
| Deadline | Oct 30, 09:00 PT. Form open. Winners Nov 6 |
| Submission form requires | GitHub repo URL · demo video <2 min · a social post tagging #OpenSourceHiggsfield · category multi-select · optional live link ("5–10 runs for judging") |
| Repo requirement | Public, OSI licence, workflow files, setup instructions that actually work |
| Judges | None published. No rubric, no panel. "One submission per person" is the only entry rule |
| Category | Prize | Criterion |
|---|---|---|
| Best Overall | $10,000 | Auto-considered |
| Most Innovative | RTX 5090 | "novel custom node packs or uses of ComfyUI. Demo video must explain why it is innovative" |
| Most Viral | RTX 5090 | Delightful, shareable |
| Open Weights Only | RTX 5090 | No closed API model in the critical path. Rented GPUs fine |
| 10 Runner Ups | $250 credits | — |
| VFX Workflow built with LTX | $2,000 cash | LTX must be the core video model |
| Built with FLUX | $1,000/$500/$250 credits | FLUX 3 core |
| Built with MiniMax H3 | $1,000 credits | H3 core |
The "$25,000" headline is a valuation; cash + credits total $16,500 plus three GPUs.
1.4 The licence trap — the thing almost nobody will check
"Open weights" ≠ "open licence".
| Model | Licence reality |
|---|---|
| Wan 2.1 / 2.2 | Apache-2.0 — the only fully unencumbered open video family. No revenue cap, no territory exclusion |
| LTX 2.3 / 2.5 | Free for commercial use under $10M revenue; 2.5 weights gated |
| MiniMax H3 | Open weights, but the United States is an excluded territory; local commercial use needs a $5,000/month licence (resold only by Comfy) |
| FLUX 3 Image Klein | Image-only, revenue-capped |
| InfiniteYou | CC-BY-NC despite Apache-2.0 code |
Only the LTX cash bonus stacks cleanly with Open Weights Only. MiniMax H3 — which both comfy-reshoot and comfy-openjutsu depend on — is the trap: for a US entrant, the H3 bonus realistically means the hosted API, which forfeits Open Weights Only.
2. What we are building
2.1 The entry
An open-source ComfyUI app for re-shooting a clip from any angle, whose differentiator is a novel custom node pack that makes the camera move verifiable.
Working name: ComfyUI-Cinematographer — the missing DP layer for Comfy's own re-shoot app.
2.2 The node pack — the novel artefact
Nothing in the ComfyUI ecosystem evaluates a frame geometrically and tells you how many metres to move the camera. These nodes do:
| Node | What it does | Ported from |
|---|---|---|
CinematographerEvaluate | Projects the subject through the camera pose; returns per rule: measured value, pass/fail, and the camera lever that fixes it — "failed rule_of_thirds by 0.18 → +0.3 m camera-x" | internal/contentstudio/cinematography.go (EvaluateFraming) |
CinematographerAutoFrame | Solves a compliant camera pose — 24 geometric iterations, no AI cost | cinematography_autoframe.go:94 |
ChorosContinuity | Sequence-level: 180° axis rule, 30° rule (flags a jump cut when consecutive shots change angle by <30°), across cuts | cinematography_continuity.go:59-83 |
SceneSizeImportance | Hitchcock size = story importance; dirty-frame / foreground-occlusion detection | cinematography_scene.go:101-142 |
CameraReliabilityZone | Grades a requested camera pose against the range the re-shoot LoRA was trained on — green / yellow / red | new; mirrors comfy-reshoot's indicator, but computed rather than hard-coded |
The last one is the bridge. comfy-reshoot already shows a green/yellow/red reliability zone. Our node pack computes the framing quality on top of it — so the user sees both "is this inside the model's training range?" and "is this a good frame?", with a number for each.
2.3 The app
A director's panel where the user:
- Loads a clip → depth is analysed once (MoGe-2)
- Orbits / dollies / keyframes a camera move — the existing re-shoot UX
- Presses Evaluate → the panel prints rule-by-rule arithmetic and correction levers
- Presses Auto-Frame → the solver moves the camera to a compliant pose
- Generates → LTX + CrossView-Warp renders the new angle
- Checks continuity → Choros validates the 180° line and the 30° rule across the shot list
Step 3 is the thing no competitor has, and it is the thing that makes the demo watchable: numbers change on screen when the camera moves.
2.4 The model stack — licence-clean
| Role | Model | Licence | Why |
|---|---|---|---|
| Core video | LTX 2.3 / 2.5 | LTX-2.x Community (<$10M rev) | Required for the $2,000 LTX bonus; CrossView-Warp is an LTX 2.3 IC-LoRA |
| View steering | CrossView-Warp (Cseti) | Apache-2.0 | The re-shoot engine. Green ±45° azimuth; steers the viewpoint, does not reproject it |
| Depth | MoGe-2 | open | Depth analysis for the warp guide |
| Licence-clean fallback | Wan 2.2 (VACE / Fun-Control / Fun-Camera) | Apache-2.0 | The defensible path if we want zero licence caveats |
Deliberately excluded: MiniMax H3 — US territory exclusion. This is a real differentiator against both reference apps.
3. What we already have — the "TON of ComfyUI functionality"
Full evidence in analysis/10-comfyui-substrate-inventory.md and analysis/11-contentstudio-inventory.md.
3.1 The ComfyUI integration — shipped
| Capability | Where |
|---|---|
Full ComfyUI connector: /system_stats, /object_info, /upload/image, POST /prompt, GET /history/{id}, /view — HTTP polling, no websocket | services/local-resource-manager/connector_comfyui.go:25 |
| Embedded (capability × model) workflow store | comfyui_workflows.go:51-93 |
Graph-agnostic param injector — injects into any API-format graph by class_type + links | comfyui_workflow_params.go |
| Platform-GPU ComfyUI bootstrap on GCE | services/gpu-runner-service/startup_script.sh |
Universal local generation path (Capability is the workflow selector) | internal/aimrs/local_exec.go:248 |
ComfyUI video provider (comfyui/wan2.2-ti2v-5b, scail-2, ltx-video, …) | internal/task-executor/providers/video/comfyui.go:41 |
| Weight-licence hosting gate, enforced on both hosting paths | internal/modellicense/validate.go:31 |
| Desktop ComfyUI install (venv + ComfyUI-LTXVideo + ComfyUI-RMBG) | web/scripts/setup-comfyui.sh |
3.2 Verified workflow graphs — shipped and executed on real GPUs
services/local-resource-manager/comfyui_workflows/ — real ComfyUI API-format graphs, converted from ComfyUI's own templates against a live /object_info:
| Graph | Nodes | Provenance |
|---|---|---|
text_to_video_wan22.json | 11 | Wan 2.2 TI2V-5B · verified 2026-08-26 · 121 frames @ 832×480 |
image_to_video_wan22_ti2v.json | 12 | verified 2026-08-28 · NVIDIA L4 · 121 frames, 255 s |
image_to_video_wan22_a14b.json | 15 | Wan 2.2 I2V-A14B |
first_last_frame_to_video_wan22_a14b.json | 16 | verified 2026-08-28 · 640×640, 81 frames, 20 m 58 s, peak 21,151 MiB |
background_removal.json | 3 | LoadImage → RMBG → SaveImage |
The _README blocks record provenance, exact template version, deliberate deltas from upstream, and — for the FLF graph — an explicit "deliberately NOT registered: the graph is verified, the request contract is not." That is the quality bar for the competition repo.
3.3 The creative engine — ContentStudio
internal/contentstudio/ — 440 Go files, ~131,000 lines, 47 http_*.go route files.
- The DLO (~3,680 LOC): metric scene graph in metres, real optics (50 mm lens, 36 mm sensor, 1.6 m eye height), USD layer stack, depth + object-ID passes, conditioned render
- The Cinematographer (~883 LOC):
EvaluateFraming,AutoFrameCamera, Choros continuity - A pure-Go software rasteriser with z-buffer and perspective-correct interpolation (
aov_renderer.go:417-461) emitting metric depth, camera-space normals, instance IDs, mattes - A 16-bit metric depth + object-ID pass with JSON sidecar (
dlo_depth_pass.go) - A node-graph engine (
graph_run_*.go,http_graph_*.go) — flag-gated OFF by default (NODE39_GRAPH_RUNS, onlyws-25), so the Run API 404s in production - Poiesis (
internal/poiesis/, ~12.6k LOC) — a separate workflow-graph engine with compile / compatibility / requirements / policy stages
3.4 The rulebook — the actual asset, verified
docs/features/cinematographer/cinematography-bible.md v1.3, 38 rules, each tagged DET (deterministic — computable from 3D coords + camera projection) or PER (perceptual — vision-model scored), each with a measurement/heuristic and a correction lever.
The prior report's four load-bearing claims, verified against the code:
| Claim | Verdict | Evidence |
|---|---|---|
| 38 cinematography rules | TRUE, with a caveat | 38 rows in the bible; but only 14 rule identifiers are machine-evaluated in Go (9 + 3 + 2). Documented, not enforced |
| Choros enforces 180° + 30° across cuts | TRUE | cinematography_continuity.go:59-83 emits 180_axis and 30_degree. Caveat: eyeline is named in the same comment but lives per-shot in SceneFramingExtras |
| AutoFrameCamera solves a compliant pose in 24 iterations | TRUE as to 24; "solves" overstated | Literal for i := 0; i < 24; i++ at cinematography_autoframe.go:94, pure geometry — but no convergence test and no early exit; it returns the 24th iterate unconditionally |
| Nightly CronJob grew the rulebook 32→38 in three days | MIXED | CronJob is real and production-scheduled (37 3 * * *). Growth timeline is real. But the growth was hand-authored — the job's first commit postdates v1.3, and it never succeeded until 2026-09-02, adding rules 39–41 |
This is the most important correction in the whole package. The honest number is 14 machine-evaluated rules, not 38 — and AutoFrameCamera returns an iterate, not a proven solution. Our competition entry must not repeat the overclaim. We will port the rules that are actually implemented, and we will add a convergence check to the solver. That is a strengthening, and it is exactly the kind of rigour that wins a judged competition.
3.5 The strategic prior art
../client-reports/higgsfield-cinema-studio-4.0/ is a complete adversarial teardown of the product we are asked to rebuild — including the finding that Cinema Studio's headline style-engine claim is contested by a control experiment, and that its three sponsored tutorials show two different builds. That report is our unfair advantage: we know exactly where the incumbent is weak, with receipts.
4. The mapping — our assets onto their cards
| Their card | Our existing asset | Gap to close |
|---|---|---|
| Cinematic Studio | DLO (real optics, metric scene) + Cinematographer (measured corrections) | Port to ComfyUI nodes; build the panel. Comfy's own reference app is prompt-composition only — we can beat it on faithfulness |
| Re-shoot a video | DLO conditioned render + depth/object-ID passes + AutoFrameCamera | Video→scene reconstruction; adopt CrossView-Warp as the steering engine |
| Openjutsu | SCAIL-2 (character replacement, ComfyUI core) already in the model set | Not our play — comfy-openjutsu has 53★ and the ground is taken |
5. Why this wins — corrected for the competitive reality
5.1 The obvious ground is taken
| Competitor | What they have |
|---|---|
| Freefield (MIT) | Reportedly rebuilds ~95 of Higgsfield's ~129 tools as ~117 open-weight ComfyUI workflows (published 10/27) |
D3voz/openjutsu (MIT) | Community Genjutsu alternative on ComfyUI + MiniMax H3 |
Comfy-Org/comfy-openjutsu | 53★, the most popular reference app |
ComfyUI-CameraReference3D | 17 base camera moves + timeline DSL + Plücker export, verified bit-identical to built-ins |
So "we rebuilt Genjutsu" is not a winning pitch. Neither is "we rebuilt re-shoot" — Comfy already ships it. The differentiation has to be a novel node, a better interface, a missing mode, or an ignored category.
5.2 Our differentiation is a novel node — and it is the hardest visible problem
The re-shoot UX's hardest problem is where do I put the camera? The reference app answers it with a hard-coded green/yellow/red zone. We answer it with geometry: a measured pass/fail per rule, a correction lever in metres, and a solver that moves the camera for you.
That is:
- Novel — no framing-evaluation node exists in the ecosystem.
CameraReference3Dgives you 17 presets; we give you a judgement. - Demonstrable in 30 seconds — move the camera, watch the numbers change. It is arithmetic, not a conditioning claim.
- A direct answer to the incumbent's weakness — Cinema Studio has no continuity enforcement at all, and Higgsfield's own consistency guide concedes the problem: "AI video distortion is not a quality problem. It is a memory problem… each new generation is a fresh context." Choros enforces the 180° line deterministically, across cuts.
5.3 The category fit
| Category | Our position |
|---|---|
| Open Weights Only | Strong. LTX + CrossView-Warp + MoGe-2, no MiniMax H3, no closed anchor. We can defend the stack line by line |
| VFX Workflow built with LTX ($2,000 cash) | Strong. LTX is the core video model — CrossView-Warp is an LTX IC-LoRA. Stacks with Open Weights Only |
| Most Innovative | Strong. A novel node pack, with an explicit innovation argument for the video |
| Best Overall | The aggregate bet |
The honest read: we are not going to out-build Freefield's 95 tools in 15 hours. We are going to build one thing nobody has, on a licence-clean stack, and demonstrate it working. That is a category win, not a sweep.
5.4 The risk that dominates everything
Two search syntheses report that Higgsfield served Comfy a cease-and-desist on 10/28 evening (trademark + CEO likeness) and Comfy refused on 10/29. This is REPORTED, NOT INDEPENDENTLY VERIFIED — the primary Reddit disclosure is JS-blocked to our tooling. If true, the event may not proceed as advertised. We should not build anything that depends on Higgsfield's trademarks or likeness, and we should not name the entry after a Higgsfield product. Our working name — ComfyUI-Cinematographer — is deliberately clean.
6. What has to be built in the next ~15 hours
Everything above is Go, in a private monorepo, behind Magieva's auth, billing and org model. None of it is a ComfyUI node pack, and none of it is open-source today.
| # | Deliverable | Effort | Notes |
|---|---|---|---|
| 1 | Repo + OSI licence + README/setup that works | S | Required. Apache-2.0 |
| 2 | ComfyUI-Cinematographer node pack — the DET rules in Python | L | The core novel artefact. Port the 14 actually-implemented rules, not the 38 documented |
| 3 | CinematographerAutoFrame — with a convergence check the original lacks | M | Self-contained geometry, no AI cost |
| 4 | ChorosContinuity — 180° + 30° across a shot list | M | |
| 5 | CameraReliabilityZone — computed, not hard-coded | S | The bridge to comfy-reshoot's UX |
| 6 | A working re-shoot workflow — LTX + CrossView-Warp + MoGe-2 | M | Reuse the verified graph structure from comfyui_workflows/ |
| 7 | The director's panel app | L | Thin Next.js app in the shape of the reference apps |
| 8 | Demo video <2 min | M | Must explain why it is innovative |
| 9 | Submission — repo URL, video URL, #OpenSourceHiggsfield post | S | Needs the Comfy login |
| 10 | Verification lane — does it actually run? | M | Non-negotiable |
Critical path: 2 → 6 → 7 → 8 → 9. Items 1, 3, 4, 5 run in parallel with 2.
The defensible cut. 15 hours is not enough for a full port plus a polished app plus a deployment. The cut is: a genuinely working node pack (the 14 implemented rules + the solver + Choros + the reliability zone), one end-to-end workflow, and a panel that shows the arithmetic. That is a complete, honest, novel contribution. A half-ported pack with no working workflow is not.
What we will NOT do: claim 38 rules when 14 are implemented; claim the solver converges when it does not; depend on MiniMax H3; name anything after a Higgsfield product.
7. Decisions taken
| # | Decision | Answer |
|---|---|---|
| 1 | Credentials | Operator supplied ~/.nathanielhunter-credentials (Magieva login). No ComfyUI login found — see §8 |
| 2 | Entry scope | ✅ Cinematographer node pack + re-shoot-from-any-angle. Confirmed by operator, and reinforced by the competitive analysis |
| 3 | Publication line | ⏸ Decide later — build first. Everything stays in ../higgsfield-comfyui-os/ until we draw the line |
| 4 | Target categories | Open Weights Only + LTX VFX bonus ($2,000 cash) + Most Innovative |
| 5 | Model stack | LTX-core, no MiniMax H3 |
8. The one thing still blocking
There is no ComfyUI login in the credentials file. It contains only the Magieva login (nathaniel@magieva.com + password). I need one of:
- the comfy.org credentials (if the account exists), or
- confirmation that
nathaniel@magieva.comis the comfy.org account and I should sign in with the same password, or - you handle the comfy.org signup + submission yourself, and I hand you a finished repo, video and submission text.
This does not block the build. It blocks only steps 8–9. I am proceeding with the build.
9. What happens next
- Now — freeze the node-pack API in
01-SPEC.md - Then — parallel build: node pack · workflow graphs · panel app · repo scaffolding
- Then — a verification lane that actually runs the thing end to end
- Then — demo video + submission
Package contents
| Path | What it is |
|---|---|
00-BREAKDOWN.md | This document |
01-SPEC.md | The frozen entry spec and node-pack API |
analysis/10-comfyui-substrate-inventory.md | Every ComfyUI integration point in the monorepo, with paths |
analysis/11-contentstudio-inventory.md | ContentStudio, DLO, Cinematographer — plus verification of the prior claims |
analysis/20-competition-brief.md | Rules, prizes, judging, the three cards decoded, the licence trap |
verification/ | The adversarial lane |
raw-articles/ | Captured sources |
app/ | The competition entry |