Style Transfer (ControlNet+IPA v2)

From v1.3 onward workflow functions for both SD1.5 and SDXL

About a week ago I made a Style Changer based on the cos_edit Model ( https://civitai.com/models/397389/cosxledit-img2img-style-changer ). While this was a good attempt it lacked severe versatility. That is why today I bring you a fully functional Style Transfer Workflow with all the power of Flexibility and versatility. It is tied into ControlNet and IPAdapter v2 and works on any style, artistic, photographic realism, and anything in between (This, as long as you can dig up the Style reference pictures you need.)

How it works:

A basic set Image generation nodes (Loaders, Prompts and KSampler) has been tied to ControlNet and IPAdapter.
The Image Loader tied to ControlNet is your Main Image, it’s the one you’ll be altering.
The Image Loader tied to IPAdapter is your Style Image, it’s the one that you’ll use to fix its style onto your main image.

Important information about the ControlNet Image Loader: It has 2 distict Image Outputs. If you are working with a Monochrome Lineart Image you want to use the Lineart Output. If you use an Image that has no Distict Lines for the Lineart ControlNet to detect you want to link up the Non-Lineart Image Output.

This is because the Images need to be prepared for ControleNt. If you are already using a Lineart Picture it’ll be inverted only. If it’s a different style of image the Node will automatically convert it to a Lineart picture in the background and use that for the ControlNet Adapter.

You can go crazy and try it out both ways, but overal the output quality is better when you stick to the correct Image Outputs.

What is needed:

As always, I suggest you Download ComfyUI Manager by Lt.Dr.Data for easy instalment of any additional nodes you currently lack: https://github.com/ltdrdata/ComfyUI-Manager
For general flexibility I suggest the Proteus Model (Linked Below)
The Checkpoint loader also has a LoRA slot for a Detail Adder. (I’ll link the one I use Below)
You will need the Lineart Controlnet for SDXL (Provided in the Files *.zip)
For Flexibility I also Suggest 2 upscaler Model 1 for Artistic Purposes and 1 for Photographic Outcomes (Both are also in the Files Download)

Where to put the Files from the *.zip:

t2i-adapter-lineart-sdxl-1.0.safetensors goes into the ComfyUI\models\controlnet folder.
The 2 Upscale Models go into the ComfyUI\models\upscale_models folder

VRAM Requirements:

The last node in the Image Generation Group is a hiresfixStyle node. If your system is not capable of running this reliably you can always delete it and use Ultimate SD Upscale instead as it’s way lighter on the VRAM.
Normally any GPU with at least 8Gb VRAM should be able to run this workflow without issues.

As always, I provided more Notes inside the Workflow, but all test images uploaded here are all made with the exact same settings. So, it’s quite forgiving in that department. While there are never single settings for this sort of stuff it seems that the LineArt ControlNet and IPAdapter together make for a very user-friendly workflow.

PS: While everything is designed for SDXL if you install the SD1.5 ControlNet files for Lineart there is no reason it wouldn’t work with that version of ControlNet as well.
PPS: The Lineart pictures used in the previews where all made with my UNET – Stable Cascade Workflow (Linked Below)