Innovate Futures @ Benji
patreon
Stable Diffusion ComfyUI And Diffutoon Create Deflickering Videos - Domo AI Alternative?
🕑 Added 2024-06-26 13:00:15 +0000 UTC
Video : https://youtu.be/dGdUNZGKQYQ
Stable Diffusion Animation - ComfyUI And Diffutoon Create Deflickering Videos - Domo AI Alternative?
We're diving into the world of Stable Diffusion's animation and exploring a fascinating new project called Diffutoon. This project takes video-to-video transformations to a whole new level by turning dance videos into anime or cartoon-style videos.
Throughout this video, I'll be explaining how Diffutoon functions as a pipeline, linking up various diffusion models like ControlNet and AnimateDiff Motion model. We'll also discuss the integration of ComfyUI extensions, which allow for smooth video generation and rebranding of images.
I'll guide you through the process of setting up and running Diffutoon in Google Colab, highlighting the required diffusion models, line art, depth maps, and soft edges. We'll also explore the benefits of using ComfyUI's custom nodes like "smooth videos" and "My OpenPose" for creating stunning animations with low VRAM consumption.
Moreover, I'll showcase the DiffSynth pipeline, which transforms original videos into alternative styles using diffusion models. We'll delve into the concept of image-to-image techniques and how they can be utilized in the Diffutoon project.
To provide you with a comprehensive understanding, I'll break down the workflow steps and demonstrate the differences between using the smooth video JSON file and the smooth video with batch size JSON file. We'll also touch upon the significance of ControlNet and K-sampler in achieving desired results.
For Patreon :
Full Video : https://www.patreon.com/posts/106911040/
SD AnimateDiff Workflow Updated Version: https://www.patreon.com/posts/106879132
Resources :
Diffutoon Page: https://ecnu-cilab.github.io/DiffutoonProjectPage/
Github: https://github.com/modelscope/DiffSynth-Studio
For ComfyUI: https://github.com/AInseven/ComfyUI-fastblend
AnimateDiff V2V Workflow : https://www.patreon.com/posts/benji-vid2anime-105079998
In case you don't know where to get workflows mentioned in this video.
You can find it in the post Attached file, and for the smooth video workflow, I added the IPAdapter group just for more effect like Domo AI do.
