Innovate Futures @ Benji
patreon
Skyreel image_to_video 2Sampling + upscale Version 20250220
🕑 Added 2025-02-19 19:32:26 +0000 UTCRelated Post : https://www.patreon.com/posts/122644120/
Video: https://youtu.be/ZGjJG_kT9Ew
This is the workflow I create based on the tutorial of Skyreel image_to_video.
As I have explained the importance of using 2nd sampling for refinement in this AI video model.
The workflow for generating videos with a second pass sampling to enhance quality involves several key steps:
1. *Update ComfyUI*: Ensure your ComfyUI is updated to the latest version by either using the update BAT file in the ComfyUI Portable folder or executing a `git pull` command if you're using a virtual environment.
2. *Download Model Files*: Obtain the Skyreel model files (FP8 or BF16 versions) from KJ’s repositories on Hugging Face and place them into a subfolder within your ComfyUI models directory.
3. *Prepare Workflow*: Create a basic workflow in ComfyUI based on the design of Hanyuan Videos, incorporating custom nodes such as instruct pix-to-pix conditioning, dual clip loader, and latent video nodes. Load your initial image and set text prompts.
4. *First Sampling Pass*:
- Configure settings like frame count (e.g., 49 frames for 2 seconds), scheduler steps (e.g., 20), and CFG scale (e.g., 4).
- Use the diffusion model loader to load Skyreel weights and process through the first sampler.
- Apply VAE decode tiling if limited by VRAM constraints.
5. *Second Sampling Pass*:
- Introduce a second sampler group in the workflow to refine video quality. Utilize set-get nodes to pass along model data and conditioning between samplers.
- Adjust denoise levels (e.g., 0.6 to 0.7) for better results during this pass.
- Experiment with different scheduler methods and sample techniques (like DPMPP2M or Eular).
6. *Upscaling (Optional)*: If satisfied with the second pass results, enable an upscaler (such as flux upscale) to improve resolution further. Fix the seed number to maintain consistency in generated outputs.
7. *Review Output*: Assess the final video output, noting improvements in motion smoothness and stability compared to the first pass. Be prepared to tweak parameters like denoise strength and scheduler types for optimal results.
By following this workflow, users can generate higher-quality AI-driven videos from images, leveraging both primary and secondary sampling passes to refine details and achieve more stable animations.
Resources:
Model for ComfyUI: https://huggingface.co/Kijai/SkyReels-V1-Hunyuan_comfy/tree/main
Full Model Weights For Server GPU: https://github.com/SkyworkAI/SkyReels-V1
Attached the file below, have fun :)
