ComfyUI is a node-based GUI for image generation. Build workflows by moving blocks around.

Installation

Portable Version (Easy)

  1. Go to https://github.com/comfyanonymous/ComfyUI
  2. Download "Portable" version
  3. Extract
  4. Run run_nvidia_gpu.bat (Windows) or ./run.sh (Linux/Mac)
  5. Browser: http://localhost:8188

Manual Installation

git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
pip install -r requirements.txt

python main.py

First Workflow

  1. Load Checkpoint: "Load Checkpoint" node → sd15-v2-1.safetensors
  2. Positive Prompt: "A beautiful sunset over mountains, oil painting"
  3. Sampler: Steps: 20, CFG: 7.5
  4. VAE Decode + Save Image
  5. Click Play!

Model Comparison

Model Size Quality Speed
SD 1.5 4 GB Good Fast
SDXL 6.9 GB Very good Medium
Flux.1 24 GB State-of-the-art Slow

Advanced: Multi-Step Workflow

Load Model → Encode Prompts → Sampler → VAE Decode → Upscale → Save

LoRA (Style Control)

Load style LoRAs from Civitai:

Load Checkpoint → Load LoRA (strength 1.0) → Sampler

Prompt: "masterpiece, best quality, ghibli style, detailed"

Batch Processing

# batch_generate.py
import requests
import json

API_URL = "http://localhost:8188"

prompts = [
    "A beautiful sunset over ocean",
    "A cozy cabin in snow",
    "Futuristic city at night"
]

for i, prompt in enumerate(prompts):
    workflow = {...}  # Your workflow JSON
    workflow["2"]["inputs"]["text"] = prompt

    response = requests.post(f"{API_URL}/prompt", json={"prompt": workflow})
    print(f"Image {i+1}: {prompt}")

Generate 100 images automatically!

Performance Tuning

  1. Quantization: "Model precision": "fp8" (2x faster)
  2. Tiling: For large images (prevent OOM)
  3. Lower Steps: 15 instead of 30 (faster but less detail)

Top Issues

Problem: CUDA Out of Memory

  • Reduce resolution (1024 → 512)
  • Enable tiling
  • Use fp8 quantization

Problem: Ugly Images

  • Bad prompt → use "masterpiece, best quality"
  • CFG too high (>15) → use 7-10
  • Try different seeds

Problem: ComfyUI won't start

  • Check port (lsof -i :8188)
  • Try different port: python main.py --port 8189

Advanced: ControlNet and Image2Image

ControlNet lets you guide generation with existing images. Instead of pure text, you show the model what composition/pose you want.

Installation

# Download ControlNet models to models/controlnet/
# Available at: huggingface.co/lllyasviel/ControlNet-v1-1

# Common models:
# control_canny-fp16.safetensors  (edge detection)
# control_openpose-fp16.safetensors  (pose control)
# control_depth-fp16.safetensors  (3D structure)
# control_scribble-fp16.safetensors  (hand-drawn guidance)

Basic ControlNet Workflow

Load Image → Canny Edge Detector
              ↓
         Load Checkpoint + Sampler
              ↓
       ControlNet Loader (strength: 1.0)
              ↓
       VAE Decode → Save

Practical: Sketch a rough composition in Paint, ComfyUI respects that structure while generating high-quality details.

Image-to-Image (Img2Img)

Starting from existing image instead of pure noise:

# img2img_workflow.py
workflow = {
    "1": {"class_type": "LoadImage", "inputs": {"image": "base_photo.png"}},
    "2": {"class_type": "VAEEncode", "inputs": {"pixels": ["1", 0], "vae": ["model", 2]}},
    "3": {"class_type": "AddNoise", "inputs": {"samples": ["2", 0], "strength": 0.5}},  # Control how much change
    "4": {"class_type": "KSampler", "inputs": {"model": ["model", 0], "steps": 20, "cfg": 7.5, "latent": ["3", 0]}},
    "5": {"class_type": "VAEDecode", "inputs": {"samples": ["4", 0], "vae": ["model", 2]}},
    "6": {"class_type": "SaveImage", "inputs": {"images": ["5", 0]}}
}

Strength parameter: 0 = no change, 1.0 = complete regeneration. Use 0.3-0.7 for subtle modifications.

Inpainting: Edit Specific Regions

Mask areas you want to regenerate while preserving surroundings.

Load Image → Mask Editor
    ↓            ↓
  VAEEncode → Set Latent Region (mask)
    ↓
  KSampler (CFG: 10 for inpainting)
    ↓
  VAEDecode → Save

Inpainting Example:

# Apply mask before sampler
# mask = [0, 0, 1, 1, ...] where 1 = paint, 0 = preserve

Multi-Step Workflows: Upscaling

Generate at 512x, upscale to 1024+ for faster generation + better quality.

Load Checkpoint (512) → Sampler (20 steps)
    ↓
  VAEDecode
    ↓
  Upscaler Node (4x ESRGAN)
    ↓
  Optional: Second Pass Sampler at 1024
    ↓
  Save

Time: 2x faster than generating at 1024 directly, often better quality.

Batch Generation with Dynamic Prompts

Generate variations automatically:

# dynamic_batch.py
from typing import List
import requests
import json
import time

def generate_batch(prompts: List[str], model: str, steps: int = 20) -> List[str]:
    API_URL = "http://localhost:8188"
    outputs = []

    for i, prompt in enumerate(prompts):
        workflow = load_workflow_template()
        workflow["prompt_encode"]["inputs"]["text"] = prompt

        # Send to API
        response = requests.post(f"{API_URL}/prompt", json={"prompt": workflow})
        prompt_id = response.json()["prompt_id"]

        # Poll for completion
        while True:
            hist = requests.get(f"{API_URL}/history/{prompt_id}").json()
            if prompt_id in hist:
                output_path = hist[prompt_id]["outputs"]["images"][0]["filename"]
                outputs.append(output_path)
                break
            time.sleep(0.5)

    return outputs

# Usage
prompts = ["a red car", "a blue car", "a green car"]
results = generate_batch(prompts, "sd15", steps=15)
print(f"Generated {len(results)} images")

Performance: VRAM Optimization

1. Half-Precision (FP16)

Reduces VRAM by 50%, negligible quality loss:

Load Model → VAE in FP32, Sampler in FP16

2. Tiling (Huge Images)

Process large images in tiles:

Load Image (4096x4096)
    ↓
Enable Tiling in Sampler (tile_size: 512)
    ↓
Stitch results
    ↓
Save

3. Lower Step Count

20 steps often visually indistinguishable from 50:

Quality comparison:
10 steps: 60% similarity to reference
20 steps: 92% similarity
30 steps: 97% similarity
50 steps: 99% similarity (diminishing returns)

Model Comparison (2026)

Model VRAM Quality Speed Best Use
SD1.5 4GB Good Fast Learning, iteration
SDXL 7GB Very Good Medium Production
Flux.1-dev 24GB Excellent Slow High-quality final
Flux.1-pro 32GB State-of-art Very Slow Commercial assets

Troubleshooting Advanced

Memory Errors at Resolution 1024+

Error: "CUDA out of memory"
Solution:
1. Enable tiling (tile_size=512)
2. Reduce batch_size to 1
3. Lower context length
4. Use fp8 quantization: workflow["ksampler"]["inputs"]["model"] = load_fp8()

Artifacts at Boundaries

Problem: Visible seams between tiles
Solution:
1. Increase overlap between tiles (80% overlap)
2. Use blend_modes: "gaussian"
3. Process with post-processing upscaler

Color Banding (Posterization)

Problem: Smooth gradients show color bands
Solution:
1. Add bit depth node (dither=true)
2. Increase sampler precision
3. Use improved sampler: "dpmpp_2m"

Nodes Reference (Common)

Node Type Function Key Params
CheckpointLoader Load model model_name
KSampler Core generation steps, cfg, seed
CLIPTextEncode Prompt → vector text, clip
VAEDecode Latent → image samples, vae
LoraLoader Load style lora_name, strength
ControlNetLoader Pose/edge guidance control_type
ImageUpscaleWithModel 2x/4x enlargement upscale_model
SaveImage Write to disk images, quality

Workflow Export & Sharing

ComfyUI saves workflows as JSON:

# Export workflow from GUI
# Right-click canvas → Save Workflow

# Share with others
git add my_workflow.json
git commit -m "Add portrait generation workflow"

# Others import
# Drag JSON onto ComfyUI canvas

Community Workflows

Popular open-source workflows:

  • AnimateLCM: Fast animation generation
  • InstantID: Face-swapping with identity preservation
  • PhotoBooth: Realistic portrait generation
  • StyleTransfer: Apply style to images

Find at: https://github.com/comfyanonymous/ComfyUI/discussions

Resources