Skip to main content

ComfyUI

Node-based Stable Diffusion UI for local GPU image generation. Workflows export as JSON and run over a REST API, so Decree can drive them.

Overview​

ComfyUI runs at https://comfyui.EXIST_DOMAIN (via Caddy) and http://comfyui:8188 (container to container), on the local GPU.

The ComfyUI web frontend reports browser errors to Comfy's Sentry, and upstream offers no setting to turn that off.

Using a ComfyUI you already run​

EXIST_COMFYUI_URL is the one address the stack uses (the comfy machine, the comfyui.EXIST_DOMAIN proxy and exist.test.sh). Point it elsewhere and turn the local one off:

EXIST_COMFYUI_URL=http://bigbox:8188
EXIST_IS_AI_COMFYUI=false

Then ./existential.sh && docker compose up -d. Workflows stay in workspace/.decree/lib/comfy/workflows/ (the comfy plugin's); the checkpoints they name must exist on the other host.

First Run​

After docker compose up -d, open https://comfyui.EXIST_DOMAIN.

Download a checkpoint model using ComfyUI Manager (accessible from the menu in the top-right). Common starting points:

ModelUse case
sd_xl_base_1.0.safetensorsSDXL — general purpose, 1024×1024 native
Flux.1-devHigh quality, requires separate text encoder + VAE

The container's whole working tree is the comfyui_data volume: the image copies ComfyUI out to /root on first boot, so models, custom nodes, generated images and saved workflows are all one mount. On the host that is volumes/comfyui_data/ComfyUI/ — checkpoints in models/checkpoints/ (models/diffusion_models, models/text_encoders, models/vae, models/loras for the rest), generated images in output/, saved workflows and settings in user/.

:::warning No authentication ComfyUI has no login of its own, and nothing is put in front of it. Anyone who can reach https://comfyui.EXIST_DOMAIN can queue workflows, read and write files under the volume, and install custom nodes — which is arbitrary Python executed in the container. Keep it on your network or VPN; do not put it behind a public A record. :::

API​

ComfyUI exposes a REST API on port 8188.

EndpointMethodPurpose
/promptPOSTQueue a workflow for generation
/history/{prompt_id}GETPoll generation status; includes output filenames when done
/viewGETDownload a generated image (?filename=<f>&type=output)
/system_statsGETGPU memory usage and system info

The /prompt body is a workflow exported from the ComfyUI UI as JSON, wrapped in a small envelope:

{
  "prompt": { ...workflow nodes... },
  "client_id": "any-string-to-group-your-requests"
}

Using ComfyUI from Decree​

The scripts are the comfy plugin's (automation/plugins/comfy.md). They call ComfyUI's HTTP API using curl. The pattern is: POST a workflow, poll /history until the job finishes, then retrieve the output filename.

The machine that ships​

One machine, comfy, is already written. Its type parameter names a workflow in comfy/workflows/, which the comfy plugin installs into workspace/.decree/lib/:

typeModelNeeds
image-textFlux2-dev—
image-text-imageFlux2-devinput_image
image-qwen-anglesQwen-Image-Edit 2511input_image, and a list of items (below)
video-i2vWan 2.2 I2V 14Binput_image
video-ltx-textLTX-2.5—
video-ltx-imageLTX-2.5input_image
video-ltx-first-lastLTX-2.5input_image, last_image

Each workflow JSON carries the HuggingFace URL for every model it loads, so the download list is the file itself. The shipped graphs have their prompts blanked to prompt text — you always supply your own, so there is nothing to inherit.

There is a ready-made message per workflow under examples: in the comfyui flow, automation/plugins/flows/comfyui.md. Copy one into a file and edit it:

# the comfy-image-text example, saved as comfy-image-text.md
cp comfy-image-text.md volumes/workspace_data/.decree/inbox/

The message body is the prompt, and output_prefix is the filename prefix ComfyUI writes under. Everything else a workflow exposes is a message param of the same name (under params:) — set one to override the workflow's own value, leave it out to keep it:

ParameterApplies to
width, heightthe Flux, Wan and LTX first/last flows — and an override on the other two LTX flows
aspect_ratio, megapixelsvideo-ltx-text and video-ltx-image, which size themselves through a ResolutionSelector
duration, fpsthe LTX flows — duration is seconds, not frames
lengthvideo-i2v, in frames
seedany flow with a sampler; randomized when unset
enhance_promptthe LTX flows — rewrites the prompt through a local LLM before encoding

image-qwen-angles is the batch flow: one input image, up to eight edits of it in a single queue. Its body is a YAML list instead of prompt text, so a prompt sits next to the name its output gets (the comfyui flow's comfy-image-qwen-angles example):

---
machine: comfy
params:
  type: image-qwen-angles
  input_image: example.png
---
- prompt: prompt text
  output: images/example-1
- prompt: prompt text
  output: images/example-2

Fewer than eight entries prunes the unused branches out of the submitted graph, so they cost no GPU time. There is no size parameter for this flow — the output resolution follows input_image. Long lists can live in a file instead of the body, via items_file.

Every flow is fire-and-forget: the script POSTs and exits on the HTTP code, and ComfyUI writes the result into its own output/. To do something with the file afterwards, poll /history/{id} until outputs appears and fetch it from /view.

Running a batch of prompts​

Every prompt against every reference image, with no new code. There is no separate "working directory" to set up — ComfyUI's own input/ folder is it, and on the host that is a plain bind mount at volumes/comfyui_data/ComfyUI/input/. Anything you drop there, ComfyUI knows by name immediately.

1. Put your reference images somewhere under input/. A subfolder keeps a project together:

mkdir -p volumes/comfyui_data/ComfyUI/input/hero
cp ~/refs/*.png volumes/comfyui_data/ComfyUI/input/hero/

2. Write your prompts, one per line, in a file anywhere — this one is not a decree message, just a list. Blank lines are skipped:

a portrait on the front steps at golden hour
the same subject in heavy fog, wide shot

3. Fan them out. From the repo root, this writes one inbox message per (prompt × image) — the cross product:

DIR=volumes/comfyui_data/ComfyUI/input/hero; STAMP=$(date +%H%M%S); n=0
while IFS= read -r p; do
  [ -z "$p" ] && continue
  for img in "$DIR"/*.png; do
    n=$((n+1))
    cat > "volumes/workspace_data/.decree/inbox/comfy-$STAMP-$n.md" <<EOF
---
machine: comfy
params:
  type: image-text-image
  input_image: "hero/$(basename "$img")"
  output_prefix: hero/out-$n
---
$p
EOF
  done
done < prompts.md

Drop the inner for loop, the input_image line, and $DIR for type: image-text — that flow takes no reference image, so it is one message per prompt.

4. Watch it go. Results land in volumes/comfyui_data/ComfyUI/output/hero/, and the queue is visible in ComfyUI's own UI.

docker logs -f automation

Two things to know before you point this at a large folder. The script POSTs and exits, so twenty messages become twenty ComfyUI queue entries in about a second — the depth is ComfyUI's problem, not decree's, and a few hundred is an unattended GPU night. And nothing here dedupes: re-running the loop queues the whole cross product again, which is usually what you want (seeds are randomized).

Triggering over HTTP​

The comfy plugin adds three endpoints to workspace/.decree/decree-go-rest.yml, so anything that can POST can queue a generation. The request body is the prompt:

# No reference image
curl -X POST https://automation-webhook.EXIST_DOMAIN/comfy/image-text/sunset \
  -H "Authorization: Bearer $SECRET" \
  -d 'a sunset over mountains'

# With one — "ref.png" must already be in ComfyUI's input/
curl -X POST https://automation-webhook.EXIST_DOMAIN/comfy/image-text-image/edit1/ref.png \
  -H "Authorization: Bearer $SECRET" \
  -d 'the same scene at night'

# With two — video-ltx-first-last anchors the first and last frames
curl -X POST https://automation-webhook.EXIST_DOMAIN/comfy/video-ltx-first-last/pan/a.png/b.png \
  -H "Authorization: Bearer $SECRET" \
  -d 'a slow camera push between the two frames'

The secret is AUTOMATION_WEBHOOK_SECRET in services/automation-webhook/.env. Output goes to output/comfy/<name>/.

The shape is dictated by two deliberate properties of the webhook (decree-go-rest's SPEC.md): the body is never parsed — it is opaque bytes copied into the message body verbatim — and the machine and its params are fixed per endpoint. So the prompt is the body and everything else has to be a path parameter. That has two consequences worth internalising:

  • A path parameter cannot contain /, so these routes reach only images sitting directly in input/, never in a subfolder. Keep webhook reference images flat in input/; use the batch loop above for subfolders.
  • There is no optional parameter, which is why each image count is its own route. Every additional knob (width, duration, seed, …) would likewise be another path segment or another endpoint with the value baked in — for anything more elaborate, write the inbox message directly, as the batch loop above does. That applies especially to video-ltx-first-last, which reaches its real settings (duration, fps, width, height) only that way.

Each route sets explicit patterns:. The default parameter charset is [A-Za-z0-9_\-!]+, which has no ., so ref.png would be rejected with a 400 without one. Config edits load without a restart; an invalid edit keeps the old endpoints serving and shows up in /healthz.

Designing Workflows​

Adding a flow to the comfy machine is adding two files to workspace/.decree/lib/comfy/workflows/ — a graph and a binding — then ./existential.sh run plugin export comfy to keep them in the plugin. No shell is written:

  1. Open https://comfyui.EXIST_DOMAIN, build the workflow visually, and run it once so you know it works.
  2. Export it as Workflow → Export (API) — the flat node map /prompt accepts, distinct from the regular save format, which is UI layout. Subgraphs are flattened on the way out, so node ids in the export do not match the ids you see on the canvas. Save it into workflows/. (Copying the browser's POST /api/prompt request body out of devtools works too, and keeps the UI graph so generated files re-open as a working workflow.)
  3. Write workflows/<type>.yml beside it, naming each parameter you want to control and the node and input it writes to. Copy the closest existing binding — video-ltx-text.yml for a plain flow, image-qwen-angles.yml for a batch one. A parameter no existing binding uses also needs a line in the comfy plugin's chart.data: (automation/plugins/comfy.md), or decree rejects messages that set it.
  4. ./existential.sh test unit — test-comfy-workflows.sh checks every binding against its graph.

Node ids are the fragile part: re-exporting a workflow renumbers everything, and a binding that points at a node which no longer exists would otherwise submit the untouched template and return an image of somebody else's prompt. Both patch.ts and the unit test refuse a binding that does not resolve, so a re-export fails loudly instead.

Seeds are randomized by default, so a re-run of the same message gives fresh images. Pass an explicit seed when you want to reproduce one.

Tips​

  • Monitor GPU: curl -s http://comfyui:8188/system_stats | jq .
  • Batch generation: submit multiple /prompt requests — ComfyUI queues them and processes in order, each with its own prompt_id
  • Model persistence: the comfyui_data volume keeps all downloaded models across docker compose down
  • Workflow iteration: small changes to steps (20→30), CFG scale (7→9), or sampler (euler → dpmpp_2m) have meaningful quality impact — iterate in the UI before committing to a machine
  • Image retrieval: SaveImage nodes write to /root/ComfyUI/output/ inside the container; the /view endpoint serves them from there with no additional volume mount needed

Debugging​

docker compose logs comfyui
curl -s http://comfyui:8188/system_stats | jq .