ComfyUI Higgsfield Opensource Challenge The analysis ← Build log

The analysis

ContentStudio inventory

Scope: internal/contentstudio/ in magieva-core, plus its immediate neighbours (internal/poiesis/, internal/poiesis/compile/, cmd/cinematographer-homework/, services/template-service/, the Ops/Core migrations, and the Helm CronJob chart). Revision: branch fix/residency-fail-closed-round-lesson, commit ef3798123c5 (ef3798123c59d15d4040591286c63e81346e86a3, 2026-10-27 05:30:39 -0700). Size: 440 top-level .go files (263 production + 177 test), 131,457 lines, plus subpackages render/ (20 files), timelinedoc/ (6), titleraster/ (4).

Headline. ContentStudio is a production-management system with a genuine node-graph execution engine bolted on, not a ComfyUI-style diffusion graph editor. The graph (internal/poiesis/compile) compiles a drawn {nodes, edges} document into the platform's existing WorkflowStep vocabulary and dispatches it to two engines — a task-executor "table" engine and a workflow-orchestrator "value" engine — with takes, caching, breakpoints, per-node cancel, admission holds and a signed public share link. It is flag-gated off by default (NODE39_GRAPH_RUNS, enabled only in a workspace overlay). Separately, the package contains a real pure-Go software 3D rasteriser (aov_renderer.go) that emits metric depth, camera-space normals, instance IDs and per-asset mattes; a metric scene layer (the DLO) that unprojects frame-space blocking into metres and feeds both a 16-bit measurement depth pass and an 8-bit ControlNet conditioning map; and a deterministic cinematography rulebook with 38 documented rules of which 14 are machine-evaluated in Go. ComfyUI appears in exactly one place — as a local render façade (POST {COMFYUI_API_URL}/render) for ControlNet-depth pose renders, with fal as fallback — and the code explicitly says it "never has to encode a raw ComfyUI node graph" (character_render_controlnet.go:231-233). The prior report's four headline claims are each partly true and partly misattributed; see the verification table below.


Functional clusters

Counts are exact (440 top-level .go = 263 prod + 177 test). Cluster boundaries are judgement calls; the per-file counts are not.

Cluster prod test What it does Key entry points
Graph engine / Poiesis node graph2219Node-graph compile, run dispatch/materialise/settle, kind catalogue, requirement resolution, automations, engine seams, dep/input hashinggraph_run_dispatch.go, graph_run_materialise.go, graph_live_unified.go, base_graphs.go, compile_catalogue.go
DLO + scene scale/dimensions1311Director's Layout, frame-space→metric blocking, grounding text, depth/object-ID passes, dimension resolver, LOD-1 proxiesdlo.go, dlo_blocking_space.go, dlo_depth_pass.go, dlo_grounding.go, proxy_generator.go
Cinematography / AI-DP116Deterministic framing scorer, auto-framer, sequence continuity, camera registry, DP composition/shaded render, camera persistencecinematography.go, cinematography_autoframe.go, cinematography_continuity.go, dp_shaded_render.go
Rendering + AOV65Software rasteriser (depth/normal/instance/mattes), preview render, panorama compositor/slicer/stitcheraov_renderer.go, render_preview.go, pano_compositor.go
Assembly + animatic + timeline1917Animatic stages (2D + 3D), assembly document/edit state, EDL/OTIO export, video assembly, timeline doc, proxy sheetsvideo_assembly.go, animatic.go, assembly_document.go, edl_export.go, timeline_doc_handlers.go
Providers + models + billing2017fal/Google/OpenAI/ElevenLabs/Tripo/Higgsfield adapters, AIMRS resolve, model caps, quota gate/estimates, compute ledgerprovider_falai.go, provider_google.go, aimrs_resolve.go, quota_gate.go
HTTP surface399Fiber handlers for every subsystem; 394 route registrationshttp_handler.go, api_handler.go, http_graph_runs.go
Productions + scenes + shots + takes2114Production service/bible/delete, script decomposition, entity matching, version forking, shot planner/failure/takes/versions, coverageproduction_service.go, decomposer.go, shot_planner.go, coverage.go
Characters + props + locations + sets2310Character generation/pose/ControlNet, companions, crowds/ensembles/family, performers/casting, wardrobe, likeness, props, locations, set constructioncharacter_pipeline.go, companion_creator.go, location_world_pipeline.go, set_construction.go
Assets design pipeline2110Asset library/pipeline/representations/needs/usage, org assets, CS collections + version services, subject classification, reference readiness, clearanceasset_pipeline.go, cs_version_service.go, asset_library.go
3D assets + rigging + USD export1110USD archive/reader/resolver/library, exporter, layer stack, layout layer, level export, scene graph, rigging rules, impostorsusd_exporter.go, usd_scene_graph.go, usd_asset_resolver.go, rigging_rules.go
Levels / games46Level index/approvals/nav export, nav regions, spawn points, game-level patternslevel_index.go, level_nav_export.go, nav_regions.go
Org kinds + patterns + modules1011Org-defined node kinds (prompt/composite/tool), module registry + ports, workflow patterns, production pattern resolutionorg_kinds_store.go, modules.go, registry_modules.go, workflow_pattern.go
QA critics106Momus tiers/axes/evaluate/DLO-frame/level, rig critic, framing critic, vision validator, image QA, approval gatemomus_evaluate.go, vision_validator.go, framing_critic.go, rig_critic.go
Audio / music / voice109Audio duration/peaks, beats + beat audio, music handlers, voice library, dialogue export/projection/promptbeats.go, music_handlers.go, voice_library.go, dialogue_export.go
VFX breakdown + script43VFX breakdown read/write, script source resolution, text-model completer seamvfx_breakdown.go, vfx_source.go
Universal assets + brands + worlds43Versioned cross-production universal assets, group assets, Mythos world bundles, Kleos brandsuniversal_assets.go, mythos.go, brands.go
Skill handlers + task dispatch93Task handler registry, Methodos handlers, keyframe dispatch/sweeten, scene/shot keyframe, shot video, video assembly/breakdown skillstask_handlers.go, keyframe_dispatch.go, skill_shot_video.go
Infra / plumbing66Logger, org guard, ingest reaper, integrations-degraded, provenance seam, Themis authzlogger.go, org_guard.go, themis_authz.go
Test-only helpers / fixtures02pgx test harness + USD substrate fixturecs_pgtest_test.go, usd_substrate_fixture_test.go
Total263177

Subpackages: render/ 11 prod + 9 test (ffmpeg IR/executor); timelinedoc/ 3 prod + 3 test (v2 timeline document + TypeScript codegen); titleraster/ 2 prod + 2 test (embedded-Inter title rasteriser).


The graph engine

It is a node graph, but not a ComfyUI graph

The graph is a Poiesis production graph: nodes are stages of a film production (ingest a script, break it down, forge assets, lay out a shot, cut an animatic, render), wired by typed ports whose tokens are production nouns — Script, Beat, Shot, Cut, Take, Stage, Proxy, Score (modules.go:46-63). The header is explicit that a module "is deliberately not 'an endpoint'" and that Content Studio "serves several hundred routes, almost all of them reads, approvals and single-field edits that nobody would ever wire into a graph" (modules.go:11-16). There are 28 modules in the registry (grep -c '^\t\tID:' modules.go = 28), spanning categories catStory, catAssets, catLayout, catGeneration, catEdit, catQA, CategoryAssetForge (modules.go:296-730).

Node kinds come from four sources, unioned into one catalogue by CodeCompileCatalogue() (compile_catalogue.go:71):

  1. 28 production modules (modules.go:296+) — e.g. script.ingest, script.parse, vfx.breakdown, asset.character, dlo, keyframe, video, voice, animatic, assembly, qa.momus, forge.set.
  2. Registry endpoint kinds — every live, schema-bearing AEIOUY endpoint becomes a node kind at serve time, prefixed endpoint. with JSON-typed ports (registry_modules.go:14-38, compile_catalogue.go:60+).
  3. Control kinds — start, end, approval, condition, loop, subgraph (compile_catalogue.go:193-198); a control node is folded into existing step fields and "never becomes a step of its own" (compile.go:104-113).
  4. Adapter kinds — adapter.table_ref, adapter.ref_value, which make a crossing between the two data modes explicit (compile_catalogue.go:207-216).
  5. Org-defined kinds — prompt, composite (a subgraph published as one node) and tool (a bundled skills/bundled/<key> Tool with a declared args block), versioned immutably in org_node_kinds (org_kinds_store.go:20-31, org_kinds_tools.go:14-33).

The JSON graph format

The document is versioned and split into program and drawing (internal/poiesis/compile/document.go:9-26):

Compile → materialise → dispatch

The compiler is the single translation from drawing to execution, deliberately "not a fourth engine": every engine already consumes internal/applications/models.WorkflowStep, so the missing piece was a compiler (compile.go:1-30). Three modes: validate (save-time, advice), materialise (run-time, errors refuse), diff (compile.go:54-66).

materialiseRun (graph_run_materialise.go:368) is pure — no DB — and answers which nodes run, which reuse an earlier take, which are cache hits, what each engine receives, and where the seam rows go (graph_run_materialise.go:1-12). It partitions nodes by DataMode:

The dispatcher enters the outside world through a package-level graphRunHooks seam so tests drive the whole path with fakes (graph_run_dispatch.go:44-60). The run row is a workflow_executions row with execution_metadata.engine = 'node39', role = 'run'; a value segment is a second row with role = 'segment' (graph_run_store.go:36-44, infra/supabase/migrations/20260921390700_OPS_graph_runs.sql:3-8). Idempotency is a partial unique index on (organization_id, execution_metadata->>'idempotency_key') (20260921390700_OPS_graph_runs.sql:40-45).

Live model, takes, settle

graph_live_unified.go is "one live read model for every host": it unions task rows (keyed by custom_fields.parameters.node_id), orchestrator workflow_step_executions (keyed by node_id), the run's takes, and spend from ai_token_usage, into one status per node in a fixed vocabulary: pending | blocked | running | completed | failed | cancelled | skipped | awaiting_approval | awaiting_budget | cached (graph_live_unified.go:3-35). buildLive is pure; the handler loads rows (graph_live_unified.go:13-14).

Takes are one attempt at one node, in graph_node_takes (infra/supabase/migrations/20260921390500_OPS_graph_node_takes.sql:26). The orchestrator writes its own; for a table segment they are reconciled from task rows, one take per task row, and "(run, node, take_no) is unique and a new task is a new take" (graph_takes.go:3-14).

Settling is deliberately the only writer of a run's terminal status and hold release, because only that reader sees both engines at once (graph_run_settle_pass.go:3-10). Because a run created by a temporal tick has no reader, a settle pass runs every minute over runs running for >2 minutes, calling the same loadRunRows → buildLive → settleRunStatus path an HTTP GET calls (graph_run_settle_pass.go:34-51, :100-117). It is gated on the same flag (:88-95).

Sharing and portability

Does it call ComfyUI, fal, or both?

Neither, in the graph engine itself. The graph dispatches to Magieva's own task-executor and workflow-orchestrator. Providers are reached by the nodes (which are production modules), not by the graph runtime. ComfyUI appears only in character_render_controlnet.go as a local render node façade, and fal is its fallback (see Rendering). There is no ComfyUI node-graph JSON anywhere in the package.


The DLO / scene layer

DLO = "Director's Layout" (dlo.go:8), not "Deep Layout Object". The thesis it serves: "A block-model human at 1.78m, 3m from a 50mm lens, constrains an image generator more than any prose description of framing" (dlo.go:13-16). Everything is in real metres with real optics: default 50 mm lens, 36 mm full-frame sensor, 3.0 m subject distance, 1.6 m eye height, headroom at 0.92 of subject height (dlo.go:19-41).

Scene representation

BuildDLO(g *USDSceneGraph) assembles a resolved shot graph into a DLOLayer: every entity placed at its real metric size, with a real-optics camera (dlo.go:43-68). Placements are props first then characters so prim names are stable, and Placements is initialised non-nil because a nil slice marshals to JSON null and "took down the Scene Scout page on the first real shot that had no bindings" (dlo.go:58-82). ApplyDLOToGraph folds the layout back onto the graph so the structure passes rasterise from the DLO's camera rather than the pre-DLO one (http_dlo_handlers.go:81-85).

Dimensions have one source. Before dlo_dimensions.go, four places disagreed on how big an adult human is — 0.56×1.80×0.44 m, ~1.78 m, 0.6×1.8×0.4 m, and 1.75 m (dlo_dimensions.go:9-25). ResolveEntityDimensions now resolves every consumer, with provenance rungs explicit | footprint | scale_hint | type_default | fallback and an Estimated flag (dlo_dimensions.go:51-67). CameraBasis is the single right-vector definition, because two call sites had used the negated vector and "the unprojection mirrored every frame-space placement horizontally" (dlo_dimensions.go:27-49).

Frame-space blocking → metric

This is the most interesting measurement in the package. dlo_blocking_space.go:12-38 records that of 1237 production_shot_assets rows, all 1237 had x ∈ [0,1], all had facing as the string "camera", none had z, and shot_planner.go computes x = (i+1)/(n+1) — "2D composition hints for a keyframe prompt. They were never world coordinates." Read as metres, a two-hander became two 0.5 m adults 0.33 m apart, interpenetrating at the world origin (dlo_blocking_space.go:29-31). The fix is to unproject: take the frame-relative hint and put it on the ground plane where the camera would actually see it, so the layout is metric and still agrees with the intended 2D composition (dlo_blocking_space.go:33-38). detectBlockingSpace fails safe toward world, and the authoritative statement is now the NOT NULL production_shot_assets.blocking_space column (dlo_blocking_space.go:65-76).

Passes produced

Two depth encodings exist, for different jobs, and the code is explicit about why (dlo_conditioned_render.go:27-50):

Pass Format Purpose
Metric depth pass16-bit grayscale PNG, linear, metres = value × DepthScale; 0 = no geometry; DepthScale defaults to 0.001 (1 mm quantum, 65.535 m range)Measurement, QA, cross-shot comparison (dlo_depth_pass.go:36-60)
Object-ID mask16-bit grayscale PNG, value − 1 = instance id, 0 = no geometryPer-entity masking (dlo_depth_pass.go:40)
Sidecar manifestJSON: version, dimensions, DepthScale, near/far, id→entity mapAuthoritative description; DepthPassVersion = 1 (dlo_depth_pass.go:55-77)
AOV depth image8-bit, normalised per frame, near = whitePresentation/preview only — "not comparable ACROSS shots" (dlo_depth_pass.go:20-34)
AOV normalRGBA, camera-space n*0.5+0.5Structure conditioning (aov_renderer.go:47-49)
AOV instanceRGBA, deterministic golden-ratio huePresentation; ids are not recoverable from hue (aov_renderer.go:48-60)
AOV mattesmap[int]*image.Gray, white-on-blackPer-asset masking (aov_renderer.go:50)

The rasteriser is pure Go, stdlib only, headless (aov_renderer.go:14-27): world-space triangles (aovTri, :88-92), camera transform (:292-295), perspective project (:297-301), and a z-buffer triangle rasteriser with perspective-correct 1/z interpolation (rasterTri, :417-461). renderAOVWithCamera allocates zbuf []float64 (metres), idbuf []int (−1 empty) and nbuf []vec3 (:335-341). AOVResult keeps both the presentation images and the raw measurable buffers DepthMetres/InstanceIDs (:46-63). An optional RoomBounds emits floor + 4 walls + ceiling as instance 0 so the passes "lock the room" (aov_renderer.go:29-33, :204-213).

How it conditions a render

Three arms, measured (dlo_grounding.go:10-29):

  1. Reference image (canvas snapshot) — pre-existing.
  2. Structure conditioning — the DLO's 8-bit near=white depth image drives a ControlNet. The metric 16-bit pass is deliberately not used: its polarity is inverted for a depth ControlNet (larger = further) and it is not normalised per frame, which is what a ControlNet wants (dlo_conditioned_render.go:27-50). renderPoseControlled "resolves to ComfyUI SD3.5+Depth or fal" (dlo_conditioned_render.go:120).
  3. Structured text — BuildSpatialGrounding emits a camera clause (lens, sensor, distance, height, pitch as a named angle, Dutch) and a placement clause (each entity's real height, real camera-relative distance, side of frame, nearest first, capped at 6 entities) (dlo_grounding.go:39-79, :85-114, :120-168). This arm exists because the platform's default keyframe model, nano-banana-pro, is recorded StructureControl:false and "ignores depth entirely" — so on that model "text is the ONLY channel the DLO has" (dlo_grounding.go:18-23).

HTTP surface (http_dlo_handlers.go:15-28): GET .../shots/:shotId/dlo (JSON), /dlo/layout.usda (USD layout layer), /dlo/depth.png, /dlo/object_id.png, /dlo/depth_pass.json. All read-only, per shot, behind requireProductionEntity (http_dlo_handlers.go:47-62). Pass size defaults 1280×720, capped 4096 (:30-36). The depth endpoints reuse the on-disk filenames so "a consumer that can read a written pass can read a served one without a second code path" (:26-28).


The Cinematographer

The subsystem is named Optika in code (cinematography.go:8, cinematography_autoframe.go:8, cinematography_scene.go:5, cinematography_group.go:5) and Choros for the sequence-level half (cinematography_continuity.go:8). It is the deterministic half of an "AI-DP": "Most 'good framing' is provable geometry from the 3D scene + camera projection we already track — NOT vibes" (cinematography.go:10-16).

Rule count and what is actually evaluated

The bible (docs/features/cinematographer/cinematography-bible.md) is v1.3 and contains a 38-row rule table (:39-89; verified by counting ^\| (\d+) \| = 38 rows, max 38). The same 38 names are embedded as baselineRuleNames in the homework job (cmd/cinematographer-homework/main.go:37-49; verified count 38). The DB table public.cinematographer_rules accumulates learned additions beyond the baseline (infra/supabase/migrations/20260701080106_CORE_cinematographer_rules.sql:1-16).

But the number of rules machine-evaluated in Go is 14, not 38:

The bible's own Impl column marks only 8 rules with ✅ (cinematography-bible.md:42-73), which undercounts the code (it predates #33-35) and overcounts relative to nothing — the honest statement is: 38 documented, 14 implemented, the rest prose or PER-only.

EvaluateFraming

EvaluateFraming(s Subject, cam CamPose, shotType string, intent ...ShotIntent) FramingReport (cinematography.go:130) projects the subject (feet position + height + facing) through the camera pose and scores each rule, returning a FramingReport of FramingScore{Rule, Pass, Value, Target, Weight, Correction} (cinematography.go:48-64). It is scale-invariant — everything is relative to the camera/scene (:16). ShotIntent lets the director's deliberate choices suppress false flags: Center relaxes rule-of-thirds to "centered is correct", DutchDeg makes the level rule honor an intended Dutch, Angle and Side set the register (:37-46, :152-160, :203-210).

Choros

Choros is the sequence-level continuity module: "Single-shot geometry can be perfect yet the CUT can break" (cinematography_continuity.go:8-11). EvaluateContinuity(shots []ContinuityShot) compares each consecutive pair and emits ContinuityIssue{Rule, ShotA, ShotB, Pass, Detail} (:21-28, :59-83):

Continuity is surfaced in the dp-plan response as a continuity field (http_cinematography_handlers.go:332).

AutoFrameCamera and the 24 iterations

AutoFrameCamera(s Subject, shotType string, intent ShotIntent) (CamPose, FramingReport) (cinematography_autoframe.go:64) solves a camera pose from geometry, "no AI cost, just geometry" (:8-13). It fixes a 40° vertical FOV and 16:9 aspect, sets the aim baseline to the subject's eye anchor (E = base.Y + 0.92·height), computes an initial dolly distance from the target screen-height fraction (dist = max(0.3, height·f/(2·targetFrac))), orbits to the front of the subject offset by the intended azimuth, and then runs a fixed 24-iteration damped fixed-point loop (cinematography_autoframe.go:94-115):

for i := 0; i < 24; i++ {
    ... build cam from dist + aim offsets ...
    rep = EvaluateFraming(s, cam, shotType, intent)
    dist *= 1 + 0.6*(frac/targetFrac - 1)          // dolly correction
    offX += 0.6 * (ex - desired) * dist * aspect / f   // pan correction
    offY += 0.6 * (midHead - head) * dist / f          // tilt correction
    offX = clamp(offX, -0.8*dist, 0.8*dist)
    offY = clamp(offY, -0.6*dist, 0.6*dist)
}
return cam, rep

So "24 geometric iterations" is literally true (:94). What it is not: a solver with a convergence criterion. There is no early exit, no residual check, no guarantee the 24th iterate passes — it returns whatever iteration 24 produced. AutoFrameGroup (cinematography_group.go:11) extends it to 2+ subjects by solving the centroid then pulling back until the widest pair fits ~80% of frame width (:30-56). moodToIntent derives intent from mood text and alternates which third the subject sits on for variety (cinematography_autoframe.go:39-60).

The nightly homework CronJob — real

It is real and it is wired. infra/helm/magieva/templates/cronjobs.yaml:2696-2767 renders a CronJob named cinematographer-homework when .Values.cronjobs.cinematographerHomework.enabled. Production sets enabled: true, suspend: false, schedule: "37 3 * * *", timeZone: UTC (infra/helm/magieva/values-production.yaml:2619-2622; template :2710-2712). The disposition register lists it scheduled: [production], manual: [staging, sandbox], with a validation note: "2026-09-02, ws-07, against the real Core DB: 'homework complete proposed=3 added=3' (rules 39-41). Its FIRST success — until the fix in this change it posted the direct api.anthropic.com body to the Vertex URL and got 400 on every call since 2026-07-08" (infra/helm/magieva/cronjob-dispositions.yaml:95-105). The twin rigging-homework job is the same design (cronjobs.yaml:2769+).

The job (cmd/cinematographer-homework/main.go) asks for 1-3 new rules, dedups against the 38-name baseline, and INSERTs into public.cinematographer_rules with source='homework' and ON CONFLICT (lower(name)) DO NOTHING (main.go:135-166). The prompt asks for name, tag (DET/PER/DET+PER), effect, measurement, correction as a strict JSON array (main.go:168-180). The model call goes through Vertex Claude (Workload-Identity ADC, internal/vertex), not the direct Anthropic API (main.go:78-82, :183-200); the shared reply-parsing/retry logic is internal/companion/homeworkrules/propose.go (:1-16, :85). It bills the platform org via billing.BillPlatformModelCall (main.go:275-282).

Important nuance: the job appends to the DB table, not to the markdown bible — despite the bible's own header claiming "The nightly homework loop appends to this file and bumps the version" (cinematography-bible.md:9-10). No code writes the markdown.

Verification of prior claims

Prior report: /Users/nathaniel/companies/magieva/GitHub/client-reports/higgsfield-cinema-studio-4.0/analysis/00-SYNTHESIS.md:174-191.

Claim Verdict Evidence
"38 cinematography rules"TRUE, with a material caveatThe bible has exactly 38 rule rows (docs/features/cinematographer/cinematography-bible.md:39-89, counted 38) and the homework job embeds exactly 38 baseline names (cmd/cinematographer-homework/main.go:37-49, counted 38). Caveat: only 14 rule identifiers are machine-evaluated in Go — 9 in EvaluateFraming (cinematography.go:146-224), 3 in SceneFramingExtras (cinematography_scene.go:101-142), 2 in EvaluateContinuity (cinematography_continuity.go:72,79). The bible's own Impl column marks 8 ✅ (cinematography-bible.md:42-73). "38 rules" is a documented count, not an enforced count.
"Choros enforces the 180° axis rule and the 30° rule across cuts"TRUEcinematography_continuity.go:8 names Choros; EvaluateContinuity (:59-83) compares consecutive shots and emits 180_axis (:64-72) and 30_degree (:74-81). Caveat: the third rule named in the same comment, eyeline, is not in Choros — it is per-shot in SceneFramingExtras (cinematography_scene.go:129-142), and the code says so (cinematography_continuity.go:57-58).
"AutoFrameCamera solves a compliant camera pose in 24 geometric iterations"TRUE as to the 24 iterations; "solves a compliant pose" is overstatedThe loop is literally for i := 0; i < 24; i++ (cinematography_autoframe.go:94), pure geometry, no AI cost (:8-13). But there is no convergence test and no early exit — it returns the 24th iterate's FramingReport unconditionally (:115-116). Nothing in the function guarantees rep.OverallPass. The tests assert specific rule outcomes on hand-built poses (cinematography_test.go:11-40, cinematography_extras_test.go:115-120), not that the auto-framer converges for arbitrary input.
"A nightly CronJob asks Claude for new rules and appends them — the rulebook grew from 32 to 38 rules in three days"MIXED: CronJob TRUE; growth timeline TRUE; causal attribution FALSEThe CronJob is real, production-scheduled at 37 3 * * *, unsuspended, and validated (cronjobs.yaml:2696-2767, values-production.yaml:2619-2622, cronjob-dispositions.yaml:95-105). It asks Claude (via Vertex) and appends to public.cinematographer_rules (cmd/cinematographer-homework/main.go:135-180). The bible did grow 32→35→38 across 2026-06-29/30→07-01 (cinematography-bible.md:91-95; git log: 6bc87e97a04 06-29 v1.0, 6ffe7dcc508 06-30 v1.1 +3, 7c3a1140358 06-30 v1.2, 40faedacb88 07-01 v1.3 +3). But that growth was hand-authored, not CronJob output: the job's first commit is 2026-07-01 08:07 (1b33e12bb40), after the v1.3 commit, and the job never succeeded until 2026-09-02, when it added rules 39-41 — not 33-38 (cronjob-dispositions.yaml:95-105). The embedded baseline has been 38 since the job's first commit (verified by extracting baselineRuleNames from 1b33e12bb40, dcf7c43739c, d51f8c9fba2, 7a5ab01b6a6, ff6a0e27c11 — all 38). So: the rulebook's 32→38 growth is real but was authored by hand in three days; the CronJob's actual contribution to date is 38→41.

Rendering and assembly

The software rasteriser (the strongest reusable asset)

aov_renderer.go is a stdlib-only software 3D rasteriser with a z-buffer and perspective-correct interpolation (rasterTri, :417-461), producing depth (metric + normalised 8-bit), camera-space normals, instance IDs and per-instance mattes (:306-412). It is wired to GET /productions/:id/shots/:shotId/dp-structure (http_handler.go:711 → http_cinematography_handlers.go:397, returning a depth PNG) and to the DLO depth endpoints (http_handler.go:692-694). A second z-buffer rasteriser in dp_shaded_render.go (RenderShaded/RenderShadedPNG, :255-265) emits flat-lit RGB only and is used by dp-preview, set-preview and the 3D animatic.

DP render files

ControlNet pose render — the one ComfyUI touchpoint

character_render_controlnet.go renders pose-locked diffusion images. Primary backend is local ComfyUI when COMFYUI_API_URL is set; any failure or timeout degrades to fal fal-ai/flux-control-lora-depth (:160-181). The ComfyUI contract is a façade: POST {COMFYUI_API_URL}/render with a comfyUIRenderRequest, "a stable façade over the SD3.5 + ControlNet-Depth graph so this service never has to encode a raw ComfyUI node graph" (:231-242). The returned provider is "comfyui" (local) or "fal.ai" (:162). The fal body carries control_lora_image_url, control_lora_strength, num_inference_steps: 30, guidance_scale: 3.5, preprocess_depth, optional seed and LoRAs (:193-210). Routes: POST .../characters/:charId/pose-render (http_handler.go:653) and POST .../dlo-render (:699).

Animatic

An animatic is the cut rendered from held stills (2D) or 3D blocking before paying for video, with dialogue muxed on top (animatic.go:40-76). Build: one ffmpeg segment per shot (-loop 1 -t D, 12 fps, 1920×1080, silent stereo bed), then concat -c copy (:143, :260-340). The 3D path calls RenderShadedPNG and honestly clamps LOD to 1 — "LOD 2/3 geometry not written" (animatic_3d.go:80, :204-210). A USD cut.usda stage is written and re-openable (animatic_stage.go:190, :254, :326). Job model with a 45-minute budget (animatic_job.go:37-88). Routes at animatic.go:427-443.

Assembly and the timeline document

render/ is an ffmpeg IR + executor, not a rasteriser (render/plan.go:1-33): BuildPlan flattens a validated doc into contiguous, frame-authoritative segments (build.go:18-61); BuildSegmentGraph emits one small filter_complex per segment with xfade offset=0 (graph.go:58-97); BuildAudioGraph emits one non-segmented graph with amix=...:normalize=0 (audio_graph.go:15-27, :122); duck.go implements a scheduled duck envelope because WebAudio has no sidechain (:9-19, :47-56); segcache.go keys a segment cache on the ffmpeg argv hash + input content hashes + ffmpeg -version (:59, :345-366); fastpath.go allows the legacy single-command path only for cut-only single-video-track docs (:27-42, :130-134); title_resolver.go rasterises a title to a full-frame transparent PNG (:83-144).

timelinedoc/ is the v2 document model (Version = 2, doc.go:45), with Validate rules E1–E9/W1–W5 (validate.go:119) and TypeScript codegen by reflection into app/lib/api/timeline-doc.ts (tsgen.go:31, :57), with a drift test against the checked-in file.

titleraster/ embeds three Inter TTFs (//go:embed fonts/Inter-{Regular,SemiBold,Bold}.ttf, raster.go:53-60) and rasterises a layout to a full-frame transparent RGBA PNG (:93), pure Go via golang.org/x/image/font.

bulk_render_scope_test.go is a static SQL scanner: any query FROM production_shots narrowing on keyframe_asset_id|video_asset_id IS NULL must mention is_current, else it pays providers for archived scene cuts (six prior occurrences; :16-31).


Providers, models and cost

Routing

All model choice goes through AIMRS (aimrs_resolve.go): a process-wide csRoutingClient (:18-26), resolveCSModel (:31-45), resolveCSModelWithProvider (:62-84, which exists because stripping the provider prefix once sent google/lyria-002 to the ElevenLabs API, :50-58), and normalizeCSModelID (:186-195). It never errors — a routing outage falls back to a hardcoded default (:28-30).

External providers wired

Provider Endpoint Env Capabilities Billing provider string
fal.aiPOST https://fal.run/{model} (provider_falai.go:55)FAL_AI_API_KEYt2i, i2i (multi-ref), t2v/i2vmodel.Provider (often "fal"), but "fal-ai" hardcoded in location_world_pipeline.go:298,439,582,648 and "fal" in http_prop3d_handlers.go:50
Higgsfieldhttps://platform.higgsfield.ai (provider_higgsfield.go:48)HIGGSFIELD_API_KEY_ID/_SECRET + per-org org_provider_api_keys (:95-119)t2i (soul/standard), i2v (dop/*, kling-video/v2.1/pro) (:19-23)"higgsfield"
ElevenLabs TTSPOST https://api.elevenlabs.io/v1/text-to-speech/{voice} (provider_elevenlabs.go:101)ELEVENLABS_API_KEYtts"elevenlabs"
ElevenLabs MusicPOST https://api.elevenlabs.io/v1/music (provider_elevenlabs_music.go:154)ELEVENLABS_API_KEYmusicresult.Provider; CostUSD = seconds × 0.008 (:195)
Google Gemini (text)Vertex-first, direct fallback (provider_google.go:80, :104-110)GOOGLE_API_KEY / Vertex ADCchat"google"
Google Imagen / Nano BananaVertex :predict + gemini-2.5-flash-image (provider_google_images.go:103-105, :89)Vertex ADC / GOOGLE_API_KEYt2i, i2i"google"
Google VeoVertex :predictLongRunning → :fetchPredictOperation (provider_google_video.go:103, :131)Vertex ADCt2v/i2vmodel.Provider
Google LyriaVertex :predict (provider_google_lyria.go:102-104)Vertex ADCmusic"google"; flat $0.06 (:48)
OpenAI imagesPOST https://api.openai.com/v1/images/generations (provider_openai_images.go:73)OPENAI_API_KEYt2i"openai"
prop3d (fal)fal image→3D (provider_prop3d.go:39,69)FAL_AI_API_KEYimage_to_3d"fal"
Tripohttps://api.tripo3d.ai/v2/openapi (provider_tripo.go:25)TRIPO_API_KEY + per-org tripo3dimage_to_model, multiview_to_model, animate_rig, animate_retarget"tripo3d"
SAM 2hosts declared (replicate/fal/roboflow, sam2_provider.go:29-31)——No HTTP call, no billing, no route — test-only/unshipped

Local providers

isLocalProvider returns true for comfyui, local-gemma, local-fk, local-render (compute_operations.go:53-59), and recordComputeOperation writes is_local=true, cost_usd=0 (:22-50). Caveat: ComfyUI is still token-charged in the token ledger (http_character3d_handlers.go:509); $0 applies only to compute_operations.cost_usd. POST /productions/:id/ops lets the client ledger a local op (compute_operations.go:64).

Billing model

Debit functions: billMediaGeneration/billMediaGenerationWithHold (tripo_billing.go:228,242), billLLMGeneration (:349), billVisionUsage (momus_evaluate.go:342), billTripoGeneration (tripo_billing.go:321), recordComputeOperation (compute_operations.go:22). providerNameAliases maps fal/fal.ai/falai (tripo_billing.go:85-92).

Vision critics

vision_validator.go tries Claude first (Vertex, then direct api.anthropic.com, default claude-sonnet-4-5-20250929, :163) and falls back to GPT-4o (:321, :326); callers bill billVisionUsage (task_handlers.go:647, http_character_poses.go:201, compute_operations.go:122, image_qa.go:217). framing_critic.go (the PER half of Optika) posts to OpenAI chat completions (:97) with resolveCSModelForProvider(ctx, "validation", "openai", "gpt-4o") (:83); it is not billed internally — the caller bills "openai","gpt-4o-vision" (http_cinematography_handlers.go:212), a model literal that differs from the model actually sent (gpt-4o). Whether the registry prices gpt-4o-vision is unverified.


HTTP surface

394 route registrations across non-test files (151 GET, 203 POST, 8 PUT, 11 PATCH, 21 DELETE), measured by grep -rn '\.\(Get\|Post\|Put\|Patch\|Delete\)("/' internal/contentstudio/ --include='*.go' | grep -v '_test.go' | wc -l.

Mounting (services/template-service/main.go): the package prefix is /api/v1/content-studio, split across two groups — RegisterPublicRoutes on v1.Group("/content-studio", OptionalAuthMiddleware(), RequireResolvedIdentity()) (main.go:606) and RegisterProtectedRoutes on v1Auth.Group("/content-studio") (main.go:677) — plus one truly anonymous route at /api/v1/public/runs/:token (main.go:614). Split: 220 identity-gated "public" group, 173 AuthMiddleware group, 1 anonymous. The comment at main.go:601 claiming "169 routes" is stale.

Note the "public" group is not public: RequireResolvedIdentity() 401s unless user_uuid or is_service is set (services/template-service/middleware.go:138-148).

Route group Example paths ~count Auth Registered
Production lifecycle/productions/:id, /status, /versions, /fork, /complete, /assemble20public-grouphttp_handler.go:437-467,724-731
Character design/productions/:id/characters/:charId/{design,options,views,voice}22public-grouphttp_handler.go:440-453,563-572
Prop design/productions/:id/props/:propId/{design,options}7public-grouphttp_handler.go:485-491
Location design + 360°/locations/:locId/{options,angles,cube,equirect}21public-grouphttp_handler.go:494-511
Levels / USD/levels, /levels/:sceneId/sign-off, /export/level3public-grouphttp_handler.go:459-467
Task/asset inputs + templates/tasks/:taskId/inputs, /asset-templates, /rigging-rules12public-grouphttp_handler.go:471-482
Voice / camera registry/voices, /cameras2public-grouphttp_handler.go:514,518
Workflow registry/patterns, /productions/:id/pattern2public-grouphttp_handler.go:525-526
VFX breakdown`/vfx-breakdown[/report.html.csv.json]`5public-grouphttp_handler.go:531-535,799-800
Shots / keyframes / video/shots/* (approve, regenerate, takes, attribution, batch, generate-keyframes/videos)~55public-grouphttp_handler.go:537-628,716-723
Assembly / NLE/assemblies, /compile, /export (EDL), /edit-state, /timeline-doc, /music/*16public-grouphttp_handler.go:577-601
3D / rig / pose / DLO / DP/generate-3d, /pose-render, /train-lora, /impostors, /dlo*, /dp-plan, /export/usd, /scene-graph~40public-grouphttp_handler.go:629-711
Universal assets/finalize-universal-asset, `/universal-assets[/promote/import], /photo-to-3d`5public-grouphttp_handler.go:669-673
Manual mode CRUDPOST/DELETE /productions/:id/{characters,locations,props,scenes,shots}21public-grouphttp_handler.go:742-775
Calliope scripts/scripts, /scripts/:id/promote3public-grouphttp_handler.go:790-792
Script editingPATCH /productions/:id/script/{dialogue,scenes,shots}/…8public-grouphttp_handler.go:811-820
Backward derivation/derive-assets-from-keyframes, /derive-script-from-shots, /derive-brief-from-assets3public-grouphttp_handler.go:823-825
Asset library/characters/:charId/save-to-library, /library/characters, /import-from-library3public-grouphttp_handler.go:734-736
Modes/auto-mode, /guided-mode, /manual-mode3public-grouphttp_handler.go:737-739
Production list/create/deleteGET/POST /productions, DELETE /productions/:id, PUT /brief, /approve-phase6protectedhttp_handler.go:867,884,939-965
Org patterns (graph doc)/org-patterns[/:key], /versions, /activate, /withdraw, /reset, /compiled, /resolved, /fork, /pull-base, /export, /import~22protectedhttp_handler.go:903-935; http_graph_portability.go:52-53
Graph runs (NODE39)`/graphs/:key/runs[/:runId][/live/cancel/nodes/:nodeId/:verb/takes]`8protected + flaghttp_graph_runs.go:66-73
Graph share mintPOST /graphs/:key/runs/:runId/share1protected + flaghttp_graph_share.go:52
Automations/graphs/:key/schedule, `/automations[/:id/pause/resume/fire]`5protectedhttp_automations.go:43-47
Org kinds`/modules/org-kinds[/:key][/versions/withdraw]`8protectedorg_kinds_handlers.go:40-47
Module types/compatGET /modules/types, POST /modules/compatibility2protectedhttp_module_types.go:32-33
Module catalogue/liveGET /modules, /modules/search, /productions/:id/graph-live*7protectedhttp_handler.go:954-964,1056
Org assets/org-assets[/:uaId], options, pipeline, clearance, publish, angles, generate-3d, train-lora, versions24protectedorg_assets.go:31-52, org_assets_build.go:37-51
Mythos / brands / placements/mythos*, /brands*, /productions/:id/placements*14protectedmythos.go:44-50, brands.go:35-38, placements.go:37-39
Ensembles / beats / scene versions/ensembles*, /scenes/:sceneId/beats, `/versionsrecut, /keyframe/*`24protectedensembles.go:31-46, beats.go:411-414, scene_versions.go:195-200
Scale / animatic / entity versions/scale-charter*, /animatic*, /:entityKind/:entityId/versions18protectedhttp_scale_handlers.go:39-59, animatic.go:431-442
Performers/casting/reps/outfits/continuity/performers*, /cast, /style-reps, /age-reps, /outfits, /continuity*, /marks, /asset-needs~30protectedhttp_handler.go:976-1026
Group assets / waveform / proxy-sheet/groups/:groupId/assets, /assets/:assetId/{waveform,proxy-sheet}8protectedhttp_handler.go:871-877,971-978
Public run shareGET /public/runs/:token1anonymoushttp_graph_share.go:74

Feature flags / kill switches

Route gates (404 when off):

Behaviour switches (no route removed): NODE39_RESOLVE_ORG_PATTERN (production_pattern_resolution.go:43, default OFF, ws-25 only), COMFYUI_API_URL (character_render_controlnet.go:175), CS_IMAGE_QA_DISABLED (image_qa.go:242), CS_APOSE_RIG (http_character3d_handlers.go:62), CS_MULTIVIEW_RIG (:67), CS_OPENPOSE_TWOPASS (character_render_controlnet.go:76), PER_ORG_DB_ENABLED (http_handler.go:227), COMPANION_STABLE_ASSIGNMENT (http_handler.go:141), NODE39_HOLD_TTL_SECONDS (http_graph_runs.go:55), MAGIEVA_TOOLS_BUNDLE_DIR (org_kinds_tools.go:35).

Auth model

callerScope (org_guard.go:72-81) is the tenancy primitive: service callers are trusted and unscoped; everyone else must resolve to a real user and org. requireProduction (:122-145) asserts the production belongs to the caller's org; requireProductionEntity (:275-295) additionally asserts a child row's production_project_id matches — the fix for the guard-mismatch class where :id was validated but :childId acted on. The org comes from the caller, never the resource, and cross-org access returns 404, not 403 (no existence oracle) (org_guard.go:14-53). sovereignOpsMiddleware (http_handler.go:226-244) stashes the per-org Ops pool on both groups. 54 routes carry billing quota gates (h.gate/gateFunc/gateFuncTTL, quota_gate.go:190-234); service callers bypass (:216).


Reusable in a competition entry

The competition is "rebuild Higgsfield's most popular apps with ComfyUI". The honest mapping:

Directly reusable as-is (pure Go, no Magieva dependencies)

  1. The software rasteriser (aov_renderer.go, 530 lines). Stdlib-only, headless, no cgo, no GPU. It turns a camera + placed boxes/humanoids into depth, normals, instance IDs and mattes. This is the single most portable artifact in the package and is exactly the "structure rig" a ComfyUI depth-ControlNet workflow needs. It has no imports outside image, math, os, sort, strconv.
  2. The metric depth pass encoder (dlo_depth_pass.go, 323 lines). 16-bit Gray16 PNG + JSON sidecar with an explicit quantum. Also stdlib-only.
  3. The cinematography scorer (cinematography.go, cinematography_autoframe.go, cinematography_continuity.go, cinematography_scene.go, cinematography_group.go; ~700 lines total). Pure geometry, no AI, no DB, no HTTP. EvaluateFraming + AutoFrameCamera + EvaluateContinuity would drop into any pipeline that has a camera pose and a subject position. This is the "deterministic DP" differentiator and it is genuinely portable.
  4. The spatial grounding text builder (dlo_grounding.go, 202 lines). Turns metric placement into a prompt clause. Pure functions.
  5. The timeline document + ffmpeg IR (timelinedoc/, render/). The render/ package shells out to ffmpeg, which is fine for an open-source app; the timelinedoc validator and TypeScript codegen are self-contained.
  6. The title rasteriser (titleraster/). Pure Go with embedded fonts.

Reusable with surgery

  1. The graph compiler (internal/poiesis/compile/, ~2,600 lines). The model is excellent and genuinely ComfyUI-adjacent: typed ports, data modes, control kinds, adapters, node aliases, a content digest, validate-vs-materialise modes, and a document envelope with credential stripping. But it compiles to Magieva's WorkflowStep, not to a ComfyUI API-format prompt graph. A competition entry would need a new backend: compile Graph → ComfyUI {prompt: {id: {class_type, inputs}}} instead of → WorkflowStep. The front half (parse, validate, topo-sort, port-type checking, digest, export/import) is reusable unchanged; the back half is a rewrite.
  2. The run/live/take model (graph_run_*.go, graph_live_unified.go, graph_takes.go). The concepts — per-node status vocabulary, takes as immutable attempts, breakpoints, per-node cancel, cache-by-input-digest, admission holds — are a better product surface than ComfyUI's queue. But they are welded to workflow_executions, tasks, graph_node_takes and ai_token_usage in the Ops DB. Porting means re-implementing the store, not the logic.

Would need replacing

  1. Everything provider-shaped. fal, Higgsfield, Tripo, ElevenLabs, Google, OpenAI adapters and the AIMRS router are Magieva-specific. A ComfyUI entry replaces all of them with ComfyUI workflows.
  2. The billing/quota layer (quota_gate.go, quota_estimates.go, tripo_billing.go, compute_operations.go). Irrelevant to an open-source entry.
  3. The HTTP surface. 394 routes bound to Fiber, Magieva auth (callerScope, requireProduction), and Ops/Core/Integrations triple-DB. Nothing here is portable; the route inventory is useful as a product spec.
  4. The ComfyUI integration that exists (character_render_controlnet.go:231-242) is a façade over a hypothetical comfyui-service that does not exist in this repo — there is no services/comfyui-service/ and no ComfyUI graph JSON anywhere. So the entry would be building the thing this code assumes.

The one-line pitch the code supports

The DLO + AOV + Optika stack is a deterministic structure-and-framing layer that sits in front of a diffusion model: metric scene → depth/normal/instance passes → ControlNet conditioning, plus a rule-based DP that scores and auto-corrects the camera before a single GPU-second is spent. Wrapping that around ComfyUI (depth ControlNet + a camera-solve node) is a coherent, differentiated competition entry, and the hardest parts (the rasteriser, the metric pass, the scorer) are already written and portable.


Gaps and unknowns

  1. The graph Run API is dark. NODE39_GRAPH_RUNS defaults off and is enabled only in values-workspace-ws-25.yaml. Every /graphs/:key/runs* route 404s in production, staging and sandbox. The graph engine is therefore built and tested but not shipped — this is the single largest caveat on the whole package.
  2. NODE39_RESOLVE_ORG_PATTERN is also off, so createTaskTree still resolves the Magieva-curated orchestration_patterns row, not the org's own graph. The org-graph path is likewise unshipped.
  3. The CronJob's real contribution is 3 rules (38→41), not 6 (32→38). The dispositions file records one validated run on 2026-09-02; whether it has run nightly since is unverified (no live cluster access).
  4. The bible's markdown is not written by the job, despite its own header claiming so. The DB and the doc can drift; nothing reconciles them.
  5. Only 14 of 38 rules are machine-evaluated. The rest are prose or PER-only (vision model). The PER half (framing_critic.go) is not billed internally and names a model literal (gpt-4o-vision) that differs from the model sent (gpt-4o).
  6. AutoFrameCamera has no convergence guarantee. It is a fixed 24-iteration damped loop; whether it reliably reaches OverallPass on real productions is unverified (the tests assert specific poses, not convergence). The platform-overview doc's "~3/12 to 12/12 DP-passing" claim (docs/platform-overview/86-Content-Studio.md:136) is a doc claim, not a code invariant.
  7. sam2_provider.go is dead code — hosts declared, no HTTP call, no billing, no route, referenced only by its own test.
  8. ComfyUI is a façade, not an integration. COMFYUI_API_URL points at a comfyui-service that does not exist in this repo; the code explicitly avoids encoding a ComfyUI node graph. There is no ComfyUI workflow JSON, no node-class mapping, no /prompt call anywhere in internal/contentstudio/.
  9. render/ requires an ffmpeg binary on the host; the segment cache keys on ffmpeg -version, so a version change invalidates it (segcache.go:345-366).
  10. Provider-string inconsistency in the ledger. fal, fal-ai, fal.ai and falai all appear; providerNameAliases (tripo_billing.go:85-92) covers some but not all call sites (location_world_pipeline.go writes "fal-ai" directly). Whether provider_token_mappings has a row for every string is unverified.
  11. No live route dump or cluster inspection was performed. Flag values in staging/prod are inferred from Helm overlays only. The claim that ws-25 is the only environment with NODE39_GRAPH_RUNS=true is from the overlay files, not from a running cluster.
  12. The timelinedoc TypeScript codegen writes app/lib/api/timeline-doc.ts; whether the checked-in file is currently in sync is unverified (the drift test exists but was not run).
  13. bulk_render_scope_test.go documents six prior incidents of paying providers for superseded scene cuts — a real cost-leak class that a competition entry would inherit if it copies the query patterns without the is_current guard.