ControlNet Mysee – Light and Dark – QR Code Hidden Text Squint Illusions

Generate images which are controlled for light and dark areas.

The idea was inspired by QR Code Monster ControlNet and all the “squint images” which are so hot right now. Although I always wondered why they used stylized QR images for training when we can just convert regular images to light and dark areas using a simple filter and train on them. Then the ability to generate QR codes emerges as a side-effect.

To use it set start_percent=0.1-0.2 and end_percent=0.8-0.9 and strength=0.8-1.2. I generaly found that you have to use lower controlnet strength compared to the QR Code models. Note that you don’t have to use white-gray-black images but can use grayscale or RBG images as well.

You can find some tutorials using the QR Code models here:

• https://takin.ai/learn/generate-images-with-hidden-text-using-stable-diffusion-and-controlnet

• https://www.youtube.com/shorts/JnSkoQruv0o

If you need inspirations look here:

• https://civitai.com/models/111006/qr-code-monster

• https://www.reddit.com/r/StableDiffusion/comments/15p5z05/hidden_memes_squint

• https://www.reddit.com/r/StableDiffusion/comments/152gokg/generate_images_with_hidden_text_using_stable

• https://www.reddit.com/r/StableDiffusion/comments/15wdoqa/been_trying_these_optical_illusions

• https://www.reddit.com/r/StableDiffusion/comments/15xh4yi/more_of_these_optical_illusions

• https://www.reddit.com/r/StableDiffusion/comments/1630zvq/remember_the_hysteria_when_people_saw_jesus_in/

• https://www.reddit.com/r/StableDiffusion/comments/162wy92/trying_this_new_squintpics

The v0.1 model is the result of my attempt to train my own Controlnet without any knowledge. It was trained with laion2B-en-aesthetic65 based on runwayml/stable-diffusion-v1-5 with 12GB VRAM (i.e. train_batch_size=1 gradient_accumulation_steps=4 gradient_checkpointing use_8bit_adam set_grads_to_none following this tutorial https://huggingface.co/docs/diffusers/training/controlnet . I don’t know what the consequences of this are) for 17000 steps (x4 = 68000 images)… until I ran out of disk space. My idea was to train the ControlNet to interpret white as light areas, gray as neutral areas and black as dark areas. Please note that I have no idea if latent diffusion models actually work like these but the ControlNet seems to work 🙂 I’d imagine that using full grayscale images would result in the colorizer ControlNet. If this model is actually better than the Monster QR Code model I don’t know: 17000 steps may be under- or over-fitted, the low VRAM might have effects on the training or the basic idea might be wrong. This is something we have to try out! It might be that luminance would be a better proxy and the remap should use a bigger spread. All conditioning images were converted with Imagemagick CLI to Mysee maps using:

magick convert -size 3x1 xc:black xc:fractal xc:white +append palette.png

magick convert input.jpg -dither None -remap palette.png mysee_map.png

which looks something like this:

A tutorial on how to train your own ControlNet will follow soon. Stay tuned!