*** 17/08/2023 – This is a beta test template with new multi-ControlNet nodes. ***
This workflow template is intended as a multi-purpose templates for use on a wide variety of projects. It can be used with any SDXL checkpoint model.
The template is intended for use by advanced users. Please try SDXL Workflow Templates if you are new to ComfyUI or SDXL.
List of Templates
SDXL Workflow for ComfyUI with Multi-ControlNet
beta version
Prerequisites
ComfyUI installation
at least 8GB VRAM is recommended
Installation
download the workflows from the Download button
download the SDXL models
download the SDXL VAE encoder
update ComyUI
install or update the following custom nodes
this includes the new multi-ControlNet nodes
ensure you have at least one upscale model installed
it is recommended to use ComfyUI Manager for installing and updating custom nodes, for downloading upscale models, and for updating ComfyUI
the MileHighStyler node is only currently only available via CivitAI
you may get errors if you have old versions of custom nodes or if your ComfyUI is on an old version
Important: Please run updates on the custom nodes before loading new template versions. This will ensure the template picks up new node versions and not the older versions.
Optional downloads (recommended)
ControlNet
download diffusion_pytorch_model.fp16.safetensors
i suggest renaming to canny-xl1.0.safetensors or something similar
download depth-zoe-xl-v1.0-controlnet.safetensors
download OpenPoseXL2.safetensors
SDXL 1.0 ControlNet softedge-dexined
download controlnet-sd-xl-1.0-softedge-dexined.safetensors
upscale models
LoRAs
On first use
select the XL models and VAE (do not use SD 1.5 models)
select an upscale model
add a default image in each of the Load Image nodes (purple nodes)
add a default image batch in the Load Image Batch node
e.g. E:\Comfy Projects\default batch
it should contain one png image, e.g. image.png
add default LoRAs or set each LoRA to Off and None
set the batch-size parameter in XL Aspect Ratio (e.g. 1, 2, 4)
set the filename_prefix in Save Image to your preferred sub-folder
set control_after_generate in the Seed node to randomize
do a test run
save a copy to use as your template
You can use any SDXL checkpoint model for the Base and Refiner models. Please don’t use SD 1.5 models unless you really know what you are doing.
Template Features
Txt2Img or Img2Img
Img2Img batch
batch size on Txt2Img and Img2Img
image padding on Img2Img
SDXL apect ratio selection
base and refiner models
separate style text prompt (optional)
a large selection of styles
prompt mixer with five presets
SDXL mix sampler
image Upscale
ControlNet batches
multi-ControlNet (with On/Off toggles)
four ControlNet pre-processors
canny
open pose
zoe depth
softedge dexined
nine LoRA slots (with On/Off toggles)
post processing options
Multi-ControlNet methodology
The method used in CR Apply Multi-ControlNet is to chain the conditioning so that the output from the first Controlnet becomes the input to the second.
For an example of this method see this link:
https://comfyanonymous.github.io/ComfyUI_examples/controlnet/#mixing-controlnets
In the diagram below the topmost active ControlNet in the Multi-ControlNet Stack nodes will be applied first. And then each successive active ControlNet working dowards. Inactive ControlNets will be ignored.
Tips
each optional component can be bypassed using logic switches (red nodes)
the default setting on all switches is Off (1)
it is not recommended to use SD1.5 models in the SDXL templates
some models don’t need a refiner so you can set the base_ratio in the sampler to 1 for these
set Preview method: Auto in ComfyUI Manager to see previews on the samplers
CR Apply Multi-ControlNet node can also be used with the Control Net Stacker node in the Efficiency Nodes
Resources
https://upscale.wiki/wiki/Model_Database
https://github.com/RockOfFire/ComfyUI_Comfyroll_CustomNodes
https://github.com/SeargeDP/SeargeSDXL
https://comfyanonymous.github.io/ComfyUI_examples/controlnet/
Change History
none yet
Troubleshooting
red missing nodes
use ComfyUI Manager to find missing nodes and download them
wrong model version
SD1.5 models can be used as a Refiner but will generally give make poor images
SD1.5 models will generate an error if used as a Base model
SDXL Sampler issues
these may be caused by updates to nodes
they can usually be fixed by updating the Searge custom nodes
MileHighStyler node missing
download SDXL Style Mile from CivitAI
place the zip file in custom_nodes and unzip
‘NoneType’ object is not subscriptable
this is usually caused by a missing default batch in Load Image Batch
create a folder on your ComfyUI drive for the default batch and place a single image in it called image.png, then copy the full path of the folder into the path field on the node
sigma_min and sigma_max must not be 0 on SeargeSDXLSampler
error occurs on dpm fast and dpm adaptive
an issue has been raised on Searge’s github
ModuleNotFoundError: No module named ‘matplotlib’
ensure your WAS nodes are up to date
you may need to remove the existing WAS nodes and reinstall via ComfyUI Manager
a fix has been included in the latest nodes, the matplotlib library should automatically download now
missing {‘cond_stage_model.clip_g.transformer.text_model.embeddings.position_ids’}
this warning message can be ignored
Comfyroll_ControlNetStack.controlnet_stacker() missing 1 required positional argument
please ensure all image inputs are connected on the multi-ControlNet stacks
SDXL Prompt Mixer Presets
the prompt presets influence the conditioning applied in the sampler
there are currently 5 presets
it is planned to add more presets in future versions
the detailed preset mappings are provided below
Mappings:
default with no style text
pos_g = prompt_positive
pos_l = prompt_positive
pos_r = prompt_positive
neg_g = prompt_negative
neg_l = prompt_negative
neg_r = prompt_negative
default with style text
pos_g = prompt_positive
pos_l = style_positive
pos_r = prompt_positive
neg_g = prompt_negative
neg_l = style_negative
neg_r = prompt_negative
style boost 1
pos_g = style_positive
pos_l = prompt_positive
pos_r = style_positive
neg_g = style_negative
neg_l = prompt_negative
neg_r = style_negative
style boost 2
pos_g = prompt_positive + style_positive
pos_l = prompt_positive + style_positive
pos_r = prompt_positive + style_positive
neg_g = prompt_negative + style_negative
neg_l = prompt_negative + style_negative
neg_r = prompt_negative + style_negative
style text to refiner
pos_g = prompt_positive
pos_l = prompt_positive
pos_r = style_positive
neg_g = prompt_negative
neg_l = prompt_negative
neg_r = style_negative
Credits
The prompts used for testing the templates were borrowed from pictures shared on the awesome AI Revolution discord sever.
There is a thriving ComfyUI community on AI Revolution.
The templates owe a lot to the great work done by Searge on developing new SDXL nodes and advanced workflows.