Innovate Futures @ Benji
patreon
AnimateDiff Animation Workflow Ver 8.5 Update
🕑 Added 2024-01-09 16:37:45 +0000 UTCFor Supporters, the workflow version 8.5 update.
For explainer about this update, I will have a video for this later.
In-case you missed any of the following AI models, you can download it in the following links, and the attached file of workflow ver.8.5
Need support about this workflow, feel free to leave a comment below.
All Models Required :
Checkpoint Model
https://civitai.com/models/4201/realistic-vision-v60-b1
Latent Consistency Model (LCM) LoRA: SDXL
https://huggingface.co/latent-consistency/lcm-lora-sdxl
Latent Consistency Model (LCM) LoRA: SDv1-5
https://huggingface.co/latent-consistency/lcm-lora-sdv1-5
VAE : vae-ft-mse-840000-ema-pruned.ckpt
Ultralytics model file (bbox) And (segm)
https://github.com/ultralytics
This comes with ComfyUI Impact Pack, you don't have to download this manually.
In case, your installation missed it, you can download in here.
All ControlNet Model Are Here : https://huggingface.co/lllyasviel
You can download all if you have spare storage, or just download the one often use.
AnimateDiff Motion Models (include SD1.5 V2 , V3, SDXL 1.0 Beta, camera padding motion loras)
https://huggingface.co/guoyww/animatediff/tree/cd71ae134a27ec6008b968d6419952b0c0494cf2
IPAdapter Models (all IPA models are listed on here)
https://github.com/cubiq/ComfyUI_IPAdapter_plus?tab=readme-ov-file#installation
Comments
Benjamin Law
something similar to this one. https://www.patreon.com/posts/animatediff-free-96537203 anyway. it's old version that I did in Jan 10. maybe you should use the latest update workflow
moelle
Hey, first of all thank you for the great content. Would it be possible to attach a screenshot of the 8.5 workflow? As I am just trying to get it to work, it would be helpful to see which adapters or models have be used on certain nodes. As sometimes its a bit tough for me to figure out the proper config if I have to recreate a red node card (as it use to only show up red, without any of the used content of the workflow)
Benjamin Law
It is in the Load model status. It shouldn't be out of memory in normal situation. What graphic card are you using?
Евгений Кашин
Hi! I thought I almost successfully run the pipeline, but got a problem at the very final stage. My comfyui process just interrupts at the final AnimateDiff step, the last output that I have is: [AnimateDiffEvo] - INFO - Sliding context window activated - latents passed in (120) greater than context_length 16. [AnimateDiffEvo] - INFO - Using motion module v3_sd15_mm.ckpt:v3. Requested to load BaseModel Requested to load ControlNet Requested to load AnimateDiffModel Loading 3 new models ^C (I don't press "^C" by myself) Is it GPU out of memory? But I thought it would be a more explicit OOM error. The last green node in UI is "KSampler". Thanks!
Benjamin Law
are you using the latest IPA? If yes, those red nodes are because the IPA developer rewrite the whole IPA custom node package. And now everything need to re-create. Therefore, even you install everything, the old node are still in red, because the new version do not have old node. I know it's crazy that they literally give up all previous IPA custom node in the package and rewrite new one.
Евгений Кашин
Impressive work, thanks for sharing! I installed all missing custom nodes, but still have 4 blocks in red. In the "IPA - FACEID Plut V2" block the bottom right one (plut IMAGE and insightface nodes leading to it) and in the "Regional IPAdapter (Optional For Video)" the bottom right node is red. Missing nodes have "undefined" name so I even not sure what I'm missing. Have you had this issue?
Benjamin Law
Set the denoising higher can help. And for text prompt, you can emphasize the text that you want by putting a bracket, and set a weight behind it. For example here is the format: (beautiful young lady) ,(One The Beach) , (Strong Sunlight:1.5)
Kamil
Hi! My output is too close to the input film. What can I change to be closer to the prompts?
Benjamin Law
You may need to test around with Denoising. So higher value of it are able to let the AI to be more creative to add things from your prompt or IPAdapter style.
Alejandro
Thanks, it works after updating comfyui, how ever on the workflow it produce a video but nothing on changes added on the prompt like dress up works. Any ideas. Thanks.
Benjamin Law
If you have ComfyUI Manager, go tonthe menu, and theres a button for install missing custom nodes. And it will show all custom nodes that are in red.
Alejandro
Hello, impressive video, need help to figured out what I have missed on my comfyui installation because it is missing the BatchPromptSchedule, imageCashapper, ttN text and CR String To Number. I have Animatediff-evolved on it but still get issues
