ComfyUI Higgsfield Opensource Challenge The plan and the build record ← Build log

The plan and the build record

Higgsfield × ComfyUI — what's being built

Prepared: 2026-10-29 · v2 (corrected after the competition-brief lane landed) Event: Comfy Org — Open Source Higgsfield · LA Build Night, in partnership with AI On The Lot Submission deadline: 2026-10-30, 09:00 PT — ≈15.5 hours from this document Status: the breakdown the operator asked for, before building starts.

v2 changes the picture in four places. The event's own premise is false; the "four inspiration prompts" are three cards; the licence landscape contains a trap that disqualifies the obvious build; and the obvious rebuilds are already taken. §1 and §5 are rewritten. The entry thesis survives all four — and is sharper for it.


0. The 60-second version

Comfy Org is running a competition inviting the community to rebuild Higgsfield's most popular apps with ComfyUI. Three app cards are named: Cinematic Studio, Re-shoot a video, Openjutsu.

We are entering with the one thing nobody else has: a cinematographer that measures.

Magieva has already built, in production Go, a 38-rule machine-applicable cinematography rulebook where each rule returns a measured value, a pass/fail, and the camera lever that fixes it — plus a 24-iteration geometric camera solver, a sequence-level continuity engine (the 180° line, the 30° rule), and a metric 3D scene layer with real optics.

Ported into ComfyUI as a custom node pack, that is a genuinely novel contribution to the ecosystem — which is precisely how the Most Innovative category is defined ("novel custom node packs or uses of ComfyUI").

One-line thesis: Every other re-shoot app asks you to aim a camera and hope. Ours tells you the frame is wrong, by how much, and which way to move — before you spend the render.


1. What Comfy Org is actually asking for — corrected

1.1 The premise is false

The event page says "Higgsfield said they'd open source their platform. Let's help." Higgsfield never open-sourced its platform, and no source shows it ever promised to. The September 2026 viral claim conflated higgsfield-ai/higgsfield — an Apache-2.0 LLM-training/GPU-orchestration framework, now archived — with the commercial studio. Higgsfield's eleven public repos are thin API clients or unrelated. The "open source Higgsfield studio" repos dominating search are third-party.

This matters for positioning, not for legality: there is nothing to fork. Cinema Studio 4.0 runs on Seedance 2.5 (ByteDance) per Higgsfield's own help centre; every other Higgsfield-branded model is closed with unpublished architecture. A rebuild is a from-scratch reimplementation of the product surface on open models.

1.2 The "four inspiration prompts" are three cards — and they are Comfy's own apps

The event page presents three app cards, each a Comfy-Org MIT reference app, not a Higgsfield product:

Card Tagline Reference repo How it actually works
Cinematic Studio"Direct your shot like a film set"Comfy-Org/comfy-cinematic-studio (MIT, ~10★)Next.js over bfl/flux-2-pro via Comfy Router. It does not submit a ComfyUI workflow at all — the "director's panel" is prompt composition, not conditioning
Re-shoot a video"Re-shoot from any angle"Comfy-Org/comfy-reshoot (MIT, ~6★)Next.js over a Comfy deployment. MoGe + CrossView-Warp + MiniMax H3, 96 GB GPU. Orbit/dolly UI with a green/yellow/red reliability zone
Openjutsu"Swap a character in a video"Comfy-Org/comfy-openjutsu (MIT, ~53★)Next.js over a Comfy deployment. Akatz H3 character-swap workflow. Optional anchor calls GPT Image 2.5 through Router — a closed API in the critical path

There is no fourth app. There is no Higgsfield app called "Re-shoot" (that is an unrelated iOS app). The event page's own copy calls these "inspiration prompts for your entry" — build something like this, not rebuild this.

The single most useful fact in this table: Comfy's own Cinematic Studio reference app is prompt composition over a hosted model. It does not condition on camera, lens or light. The prior internal report found Higgsfield's own claim to condition on those panels is contested by a control experiment. Nobody in this competition has actually built the conditioning. That is the gap.

1.3 Rules, prizes, mechanics

DeadlineOct 30, 09:00 PT. Form open. Winners Nov 6
Submission form requiresGitHub repo URL · demo video <2 min · a social post tagging #OpenSourceHiggsfield · category multi-select · optional live link ("5–10 runs for judging")
Repo requirementPublic, OSI licence, workflow files, setup instructions that actually work
JudgesNone published. No rubric, no panel. "One submission per person" is the only entry rule
Category Prize Criterion
Best Overall$10,000Auto-considered
Most InnovativeRTX 5090"novel custom node packs or uses of ComfyUI. Demo video must explain why it is innovative"
Most ViralRTX 5090Delightful, shareable
Open Weights OnlyRTX 5090No closed API model in the critical path. Rented GPUs fine
10 Runner Ups$250 credits—
VFX Workflow built with LTX$2,000 cashLTX must be the core video model
Built with FLUX$1,000/$500/$250 creditsFLUX 3 core
Built with MiniMax H3$1,000 creditsH3 core

The "$25,000" headline is a valuation; cash + credits total $16,500 plus three GPUs.

1.4 The licence trap — the thing almost nobody will check

"Open weights" ≠ "open licence".

Model Licence reality
Wan 2.1 / 2.2Apache-2.0 — the only fully unencumbered open video family. No revenue cap, no territory exclusion
LTX 2.3 / 2.5Free for commercial use under $10M revenue; 2.5 weights gated
MiniMax H3Open weights, but the United States is an excluded territory; local commercial use needs a $5,000/month licence (resold only by Comfy)
FLUX 3 Image KleinImage-only, revenue-capped
InfiniteYouCC-BY-NC despite Apache-2.0 code

Only the LTX cash bonus stacks cleanly with Open Weights Only. MiniMax H3 — which both comfy-reshoot and comfy-openjutsu depend on — is the trap: for a US entrant, the H3 bonus realistically means the hosted API, which forfeits Open Weights Only.


2. What we are building

2.1 The entry

An open-source ComfyUI app for re-shooting a clip from any angle, whose differentiator is a novel custom node pack that makes the camera move verifiable.

Working name: ComfyUI-Cinematographer — the missing DP layer for Comfy's own re-shoot app.

2.2 The node pack — the novel artefact

Nothing in the ComfyUI ecosystem evaluates a frame geometrically and tells you how many metres to move the camera. These nodes do:

Node What it does Ported from
CinematographerEvaluateProjects the subject through the camera pose; returns per rule: measured value, pass/fail, and the camera lever that fixes it — "failed rule_of_thirds by 0.18 → +0.3 m camera-x"internal/contentstudio/cinematography.go (EvaluateFraming)
CinematographerAutoFrameSolves a compliant camera pose — 24 geometric iterations, no AI costcinematography_autoframe.go:94
ChorosContinuitySequence-level: 180° axis rule, 30° rule (flags a jump cut when consecutive shots change angle by <30°), across cutscinematography_continuity.go:59-83
SceneSizeImportanceHitchcock size = story importance; dirty-frame / foreground-occlusion detectioncinematography_scene.go:101-142
CameraReliabilityZoneGrades a requested camera pose against the range the re-shoot LoRA was trained on — green / yellow / rednew; mirrors comfy-reshoot's indicator, but computed rather than hard-coded

The last one is the bridge. comfy-reshoot already shows a green/yellow/red reliability zone. Our node pack computes the framing quality on top of it — so the user sees both "is this inside the model's training range?" and "is this a good frame?", with a number for each.

2.3 The app

A director's panel where the user:

  1. Loads a clip → depth is analysed once (MoGe-2)
  2. Orbits / dollies / keyframes a camera move — the existing re-shoot UX
  3. Presses Evaluate → the panel prints rule-by-rule arithmetic and correction levers
  4. Presses Auto-Frame → the solver moves the camera to a compliant pose
  5. Generates → LTX + CrossView-Warp renders the new angle
  6. Checks continuity → Choros validates the 180° line and the 30° rule across the shot list

Step 3 is the thing no competitor has, and it is the thing that makes the demo watchable: numbers change on screen when the camera moves.

2.4 The model stack — licence-clean

Role Model Licence Why
Core videoLTX 2.3 / 2.5LTX-2.x Community (<$10M rev)Required for the $2,000 LTX bonus; CrossView-Warp is an LTX 2.3 IC-LoRA
View steeringCrossView-Warp (Cseti)Apache-2.0The re-shoot engine. Green ±45° azimuth; steers the viewpoint, does not reproject it
DepthMoGe-2openDepth analysis for the warp guide
Licence-clean fallbackWan 2.2 (VACE / Fun-Control / Fun-Camera)Apache-2.0The defensible path if we want zero licence caveats

Deliberately excluded: MiniMax H3 — US territory exclusion. This is a real differentiator against both reference apps.


3. What we already have — the "TON of ComfyUI functionality"

Full evidence in analysis/10-comfyui-substrate-inventory.md and analysis/11-contentstudio-inventory.md.

3.1 The ComfyUI integration — shipped

Capability Where
Full ComfyUI connector: /system_stats, /object_info, /upload/image, POST /prompt, GET /history/{id}, /view — HTTP polling, no websocketservices/local-resource-manager/connector_comfyui.go:25
Embedded (capability × model) workflow storecomfyui_workflows.go:51-93
Graph-agnostic param injector — injects into any API-format graph by class_type + linkscomfyui_workflow_params.go
Platform-GPU ComfyUI bootstrap on GCEservices/gpu-runner-service/startup_script.sh
Universal local generation path (Capability is the workflow selector)internal/aimrs/local_exec.go:248
ComfyUI video provider (comfyui/wan2.2-ti2v-5b, scail-2, ltx-video, …)internal/task-executor/providers/video/comfyui.go:41
Weight-licence hosting gate, enforced on both hosting pathsinternal/modellicense/validate.go:31
Desktop ComfyUI install (venv + ComfyUI-LTXVideo + ComfyUI-RMBG)web/scripts/setup-comfyui.sh

3.2 Verified workflow graphs — shipped and executed on real GPUs

services/local-resource-manager/comfyui_workflows/ — real ComfyUI API-format graphs, converted from ComfyUI's own templates against a live /object_info:

Graph Nodes Provenance
text_to_video_wan22.json11Wan 2.2 TI2V-5B · verified 2026-08-26 · 121 frames @ 832×480
image_to_video_wan22_ti2v.json12verified 2026-08-28 · NVIDIA L4 · 121 frames, 255 s
image_to_video_wan22_a14b.json15Wan 2.2 I2V-A14B
first_last_frame_to_video_wan22_a14b.json16verified 2026-08-28 · 640×640, 81 frames, 20 m 58 s, peak 21,151 MiB
background_removal.json3LoadImage → RMBG → SaveImage

The _README blocks record provenance, exact template version, deliberate deltas from upstream, and — for the FLF graph — an explicit "deliberately NOT registered: the graph is verified, the request contract is not." That is the quality bar for the competition repo.

3.3 The creative engine — ContentStudio

internal/contentstudio/ — 440 Go files, ~131,000 lines, 47 http_*.go route files.

3.4 The rulebook — the actual asset, verified

docs/features/cinematographer/cinematography-bible.md v1.3, 38 rules, each tagged DET (deterministic — computable from 3D coords + camera projection) or PER (perceptual — vision-model scored), each with a measurement/heuristic and a correction lever.

The prior report's four load-bearing claims, verified against the code:

Claim Verdict Evidence
38 cinematography rulesTRUE, with a caveat38 rows in the bible; but only 14 rule identifiers are machine-evaluated in Go (9 + 3 + 2). Documented, not enforced
Choros enforces 180° + 30° across cutsTRUEcinematography_continuity.go:59-83 emits 180_axis and 30_degree. Caveat: eyeline is named in the same comment but lives per-shot in SceneFramingExtras
AutoFrameCamera solves a compliant pose in 24 iterationsTRUE as to 24; "solves" overstatedLiteral for i := 0; i < 24; i++ at cinematography_autoframe.go:94, pure geometry — but no convergence test and no early exit; it returns the 24th iterate unconditionally
Nightly CronJob grew the rulebook 32→38 in three daysMIXEDCronJob is real and production-scheduled (37 3 * * *). Growth timeline is real. But the growth was hand-authored — the job's first commit postdates v1.3, and it never succeeded until 2026-09-02, adding rules 39–41

This is the most important correction in the whole package. The honest number is 14 machine-evaluated rules, not 38 — and AutoFrameCamera returns an iterate, not a proven solution. Our competition entry must not repeat the overclaim. We will port the rules that are actually implemented, and we will add a convergence check to the solver. That is a strengthening, and it is exactly the kind of rigour that wins a judged competition.

3.5 The strategic prior art

../client-reports/higgsfield-cinema-studio-4.0/ is a complete adversarial teardown of the product we are asked to rebuild — including the finding that Cinema Studio's headline style-engine claim is contested by a control experiment, and that its three sponsored tutorials show two different builds. That report is our unfair advantage: we know exactly where the incumbent is weak, with receipts.


4. The mapping — our assets onto their cards

Their card Our existing asset Gap to close
Cinematic StudioDLO (real optics, metric scene) + Cinematographer (measured corrections)Port to ComfyUI nodes; build the panel. Comfy's own reference app is prompt-composition only — we can beat it on faithfulness
Re-shoot a videoDLO conditioned render + depth/object-ID passes + AutoFrameCameraVideo→scene reconstruction; adopt CrossView-Warp as the steering engine
OpenjutsuSCAIL-2 (character replacement, ComfyUI core) already in the model setNot our play — comfy-openjutsu has 53★ and the ground is taken

5. Why this wins — corrected for the competitive reality

5.1 The obvious ground is taken

Competitor What they have
Freefield (MIT)Reportedly rebuilds ~95 of Higgsfield's ~129 tools as ~117 open-weight ComfyUI workflows (published 10/27)
D3voz/openjutsu (MIT)Community Genjutsu alternative on ComfyUI + MiniMax H3
Comfy-Org/comfy-openjutsu53★, the most popular reference app
ComfyUI-CameraReference3D17 base camera moves + timeline DSL + Plücker export, verified bit-identical to built-ins

So "we rebuilt Genjutsu" is not a winning pitch. Neither is "we rebuilt re-shoot" — Comfy already ships it. The differentiation has to be a novel node, a better interface, a missing mode, or an ignored category.

5.2 Our differentiation is a novel node — and it is the hardest visible problem

The re-shoot UX's hardest problem is where do I put the camera? The reference app answers it with a hard-coded green/yellow/red zone. We answer it with geometry: a measured pass/fail per rule, a correction lever in metres, and a solver that moves the camera for you.

That is:

5.3 The category fit

Category Our position
Open Weights OnlyStrong. LTX + CrossView-Warp + MoGe-2, no MiniMax H3, no closed anchor. We can defend the stack line by line
VFX Workflow built with LTX ($2,000 cash)Strong. LTX is the core video model — CrossView-Warp is an LTX IC-LoRA. Stacks with Open Weights Only
Most InnovativeStrong. A novel node pack, with an explicit innovation argument for the video
Best OverallThe aggregate bet

The honest read: we are not going to out-build Freefield's 95 tools in 15 hours. We are going to build one thing nobody has, on a licence-clean stack, and demonstrate it working. That is a category win, not a sweep.

5.4 The risk that dominates everything

Two search syntheses report that Higgsfield served Comfy a cease-and-desist on 10/28 evening (trademark + CEO likeness) and Comfy refused on 10/29. This is REPORTED, NOT INDEPENDENTLY VERIFIED — the primary Reddit disclosure is JS-blocked to our tooling. If true, the event may not proceed as advertised. We should not build anything that depends on Higgsfield's trademarks or likeness, and we should not name the entry after a Higgsfield product. Our working name — ComfyUI-Cinematographer — is deliberately clean.


6. What has to be built in the next ~15 hours

Everything above is Go, in a private monorepo, behind Magieva's auth, billing and org model. None of it is a ComfyUI node pack, and none of it is open-source today.

# Deliverable Effort Notes
1Repo + OSI licence + README/setup that worksSRequired. Apache-2.0
2ComfyUI-Cinematographer node pack — the DET rules in PythonLThe core novel artefact. Port the 14 actually-implemented rules, not the 38 documented
3CinematographerAutoFrame — with a convergence check the original lacksMSelf-contained geometry, no AI cost
4ChorosContinuity — 180° + 30° across a shot listM
5CameraReliabilityZone — computed, not hard-codedSThe bridge to comfy-reshoot's UX
6A working re-shoot workflow — LTX + CrossView-Warp + MoGe-2MReuse the verified graph structure from comfyui_workflows/
7The director's panel appLThin Next.js app in the shape of the reference apps
8Demo video <2 minMMust explain why it is innovative
9Submission — repo URL, video URL, #OpenSourceHiggsfield postSNeeds the Comfy login
10Verification lane — does it actually run?MNon-negotiable

Critical path: 2 → 6 → 7 → 8 → 9. Items 1, 3, 4, 5 run in parallel with 2.

The defensible cut. 15 hours is not enough for a full port plus a polished app plus a deployment. The cut is: a genuinely working node pack (the 14 implemented rules + the solver + Choros + the reliability zone), one end-to-end workflow, and a panel that shows the arithmetic. That is a complete, honest, novel contribution. A half-ported pack with no working workflow is not.

What we will NOT do: claim 38 rules when 14 are implemented; claim the solver converges when it does not; depend on MiniMax H3; name anything after a Higgsfield product.


7. Decisions taken

# Decision Answer
1CredentialsOperator supplied ~/.nathanielhunter-credentials (Magieva login). No ComfyUI login found — see §8
2Entry scope✅ Cinematographer node pack + re-shoot-from-any-angle. Confirmed by operator, and reinforced by the competitive analysis
3Publication line⏸ Decide later — build first. Everything stays in ../higgsfield-comfyui-os/ until we draw the line
4Target categoriesOpen Weights Only + LTX VFX bonus ($2,000 cash) + Most Innovative
5Model stackLTX-core, no MiniMax H3

8. The one thing still blocking

There is no ComfyUI login in the credentials file. It contains only the Magieva login (nathaniel@magieva.com + password). I need one of:

This does not block the build. It blocks only steps 8–9. I am proceeding with the build.


9. What happens next

  1. Now — freeze the node-pack API in 01-SPEC.md
  2. Then — parallel build: node pack · workflow graphs · panel app · repo scaffolding
  3. Then — a verification lane that actually runs the thing end to end
  4. Then — demo video + submission

Package contents

Path What it is
00-BREAKDOWN.mdThis document
01-SPEC.mdThe frozen entry spec and node-pack API
analysis/10-comfyui-substrate-inventory.mdEvery ComfyUI integration point in the monorepo, with paths
analysis/11-contentstudio-inventory.mdContentStudio, DLO, Cinematographer — plus verification of the prior claims
analysis/20-competition-brief.mdRules, prizes, judging, the three cards decoded, the licence trap
verification/The adversarial lane
raw-articles/Captured sources
app/The competition entry