sebastiankamph
patreon
Flux - How to for ComfyUI
🕑 Added 2024-08-02 15:07:10 +0000 UTCInstallation steps for Comfy
Minimum Hardware Requirements
VRAM: 8-12GB+ (16-24gb recommended)
Steps to Follow
Update your Comfy UI
If you don't have Comfy, here's how to install it: https://youtu.be/KTPLOqAMR0s
Download Necessary Models
Download t5xxl_fp8_e4m3fn.safetensors and clip_l.safetensors from Hugging Face.
Place these files in the ComfyUI/models/clip/ directory.
If you have high vram and ram, you can download the FP16 version (t5xxl_fp16.safetensors) for better results. For lower memory usage, stick with t5xxl_fp8_e4m3fn.safetensors.
Download VAE Model
Download the VAE model ae.safetensors / ae.sft from https://huggingface.co/black-forest-labs/FLUX.1-schnell/tree/main
Place the file in the ComfyUI/models/vae/ folder.
Download UNet Model
Download flux1-schnell.sft from https://huggingface.co/black-forest-labs/FLUX.1-schnell/tree/main
Place the file in the ComfyUI/models/unet/ folder.
Optional: High vram usage, flux1-dev.sft from https://huggingface.co/black-forest-labs/FLUX.1-dev/tree/main
Workflow
https://comfyanonymous.github.io/ComfyUI_examples/flux/
Resolution table, as posted by Aplakka on Reddit:
2.0 MP (Flux maximum)
1:1 exact 1448 x 1448, rounded 1408 x 1408
3:2 exact 1773 x 1182, rounded 1728 x 1152
4:3 exact 1672 x 1254, rounded 1664 x 1216
16:9 exact 1936 x 1089, rounded 1920 x 1088
21:9 exact 2212 x 948, rounded 2176 x 960
1.0 MP
1:1 exact 1024 x 1024
3:2 exact 1254 x 836, rounded 1216 x 832
4:3 exact 1182 x 887, rounded 1152 x 896
16:9 exact 1365 x 768, rounded 1344 x 768
21:9 exact 1564 x 670, rounded 1536 x 640
Realism LORA Available here:
https://huggingface.co/comfyanonymous/flux_RealismLora_converted_comfyui
LOW VRAM USAGE WITH NF4
Download NF4 v2 model https://huggingface.co/lllyasviel/flux1-dev-bnb-nf4/tree/main
Place in ComfyUI/models/checkpoints/ (not UNET as other Flux models)
Go into your custom nodes folder with cmd/terminal/powershell
git clone https://github.com/comfyanonymous/ComfyUI_bitsandbytes_NF4.git
Go into Custom nodes bitsandbytes folder
pip install -r requirements.txt
Restart comfy
Load the model from CheckpointLoaderNF4 node.
TROUBLESHOOTING - NF4 Not loading after all steps completed:
Double check you have installed bitsandbytes (pip install -r requirements.txt)
Go into the folder for the Manager, edit the config.ini and change the security setting from 'normal' to 'weak'. Run Comfy, go into the manager, change the channel to 'dev' then search for the node 'NF4'.
EXPERIMENTAL:
Get Flux to run on lower memory machines (UNTESTED BY ME): https://gist.github.com/AmericanPresidentJimmyCarter/873985638e1f3541ba8b00137e7dacd9
There's now a ControlNet canny model available for download here:
https://huggingface.co/XLabs-AI/flux-controlnet-canny/tree/main
Comments
Teodoro Morcone
I am getting a black screen image but I think I did everthing correctly, can anybody help_
Jack Jack
Hi Sebastian , can I use the Realism LORA in Forge with Flux ?
Sebastian Kamph
Yes, I know about the horns, but it looks cooler, doesn't it :)
Oliver
Schön das du dich in diesen Zeiten des initiierten Wahnsinns auf deine ethnischen Wurzeln berufst, aber Wikingerhelme mit Tierhörner gab es nicht. Die Helme sind eine reine Fiktion aus dem 19. Jahrhundert.
Adam Tubak
I would love to see what you can get out of flux with controlNet. I was not that successfull so far. And If one can achieve something like the qrcode monster stuff. It was already rather cumbersome on SDXL, curious if it will ever work properly on flux. But if it would, that would be something else!
Mackoak
Mine i got the error "If execution seems broken due to forward ComfyUI changes, you can disable the optimization from rgthree settings in ComfyUI" every time ive tried to run the workflow (any model)
Zoltán István Bíró
Thanks for the great tutorial. I will report back on my experience. Unfortunately, I only have an RTX3060 video card. With the t5xxl_fp8_e4m3fn.safetensors recommended for inferior machines, your workflow generated the sample image in 124.23 seconds, and with the t5xxl_fp16.safetensors recommended for better machines than mine, it took 159.31 seconds - I don't see any significant difference in time or quality. Jimmy Carter's experimental solution is too complicated for me, too many parameters would have to be adjusted to... If you ever have time to demonstrate a workflow for 12 GB video card PCs, I would be very interested.
steamrick
Note: There's fp8 versions with lower hardware requirements: https://huggingface.co/Kijai/flux-fp8/blob/main/README.md I had to update comfy dependencies to get it to run because it didn't recognize the weight type. I think it was a torch update in particular?... Edit: Also my measly 32GB RAM seems insufficient. It keeps overflowing and I've had one bluescreen and one complete system freeze in 3 attempts. Or I might be doing something wrong. Dunno. I guess I'll wait a bit for the code to mature. Edit2: Setting 'prefer no sysmem fallback' in the nvidia panel cuda settings and fp8_e4m3fn in the weight_dtype field seems to have fixed my problems? Eh, whatever. If it works, it works.
