Innovate Futures @ Benji
patreon
ComfyUI Wan 2.1 Skip Layer Guidance - Fix Bad AI Video Morphing Broken Parts No Problem!
🕑 Added 2025-03-20 14:26:44 +0000 UTC
Tutorial Video : https://youtu.be/xY56o8wxQu0
In this video, we explore advanced techniques to enhance AI-generated videos using Wan 2.1 and ComfyUI. If you've ever struggled with low-quality or broken AI videos—like morphing faces, unclear details, or inconsistent motion—this tutorial is for you. We dive into Skip Layer Guidance and Tile Lora, two powerful tools that can significantly improve video quality without needing to regenerate content from scratch. Whether you're working with AI-generated videos from LTX, Wan 2.1, or other models, these methods can help you achieve crystal-clear results with enhanced details and smoother animations. Perfect for AI video creators, digital artists, and tech enthusiasts, this video demonstrates how to fix and enhance existing videos, saving you time and effort while elevating your projects.
Who is this content suitable for?
This content is ideal for AI video creators, digital artists, and anyone working with AI-generated video content. It’s especially useful for those who encounter issues like morphing faces, broken details, or low-quality outputs in their AI-generated videos and want to improve them without starting from scratch.
Why does it matter?
Enhancing AI-generated videos is crucial for achieving professional-quality results. This video provides practical solutions to common problems like morphing and broken details, allowing creators to salvage and improve their existing work. By using Skip Layer Guidance and Tile Lora, you can enhance video quality, add finer details, and maintain consistency, making your AI-generated content more polished and visually appealing.
The Core Workflow: Skip Layer Guidance and Tile Lora
Tutorial Video : https://youtu.be/xY56o8wxQu0
The workflow revolves around two main techniques: Skip Layer Guidance and Tile Lora. These methods allow us to enhance video quality without starting from scratch. Here's how they work:
1. Skip Layer Guidance
Skip Layer Guidance is a training-free sampling guidance method that improves video quality by selectively skipping certain layers during the generation process. This technique is particularly useful for fixing broken or morphing elements in videos.
How it works:
In ComfyUI, you can load Skip Layer Guidance and adjust settings like "blocks" (e.g., blocks 9 and 10) and start/end percentages (e.g., 20% to 80%). These settings control how much of the original video is retained and how much is enhanced.
By skipping certain layers, the AI focuses on improving the remaining layers, resulting in higher-quality outputs.
For example, setting the end percentage to 0.75 means that blocks 9 and 10 are only partially applied, allowing the AI to work on other layers for better results.
When to use it:
Skip Layer Guidance is ideal for videos with significant issues, such as broken faces or morphing objects. It gives the AI more freedom to regenerate problematic areas while maintaining the overall structure of the video.
2. Tile Lora
Tile Lora is another powerful tool that acts like a Tile ControlNet. It’s particularly useful for adding fine details to already decent-quality videos.
How it works:
Tile Lora receives slightly blurred input frames and generates detailed outputs. It’s trained to maintain the shape and form of objects in the video while enhancing details like textures, colors, and small features.
In ComfyUI, you can load Tile Lora (either Version 0.2 or 1.0 from Hugging Face) and connect it to the workflow. The blurred frames are passed into the InstructPix2Pix Conditioning node, which guides the AI to maintain consistency while adding details.
When to use it:
Tile Lora is best for videos that are already in good shape but need more detail. For example, if you have a video where the character’s face is recognizable but lacks sharpness, Tile Lora can enhance the eyes, skin texture, and other fine details without altering the overall structure.
Key Differences Between Skip Layer Guidance and Tile Lora
Freedom vs. Control:
Skip Layer Guidance gives the AI more freedom to regenerate broken or morphing parts of the video. It’s ideal for fixing severe issues like distorted faces or objects.
Tile Lora focuses on maintaining the original shape and form while adding finer details. It’s better suited for videos that are already decent but need enhancement.
Denoise Settings:
When using Skip Layer Guidance, you can adjust the Denoise parameter to control how much the AI deviates from the original video. Higher Denoise values (e.g., 0.6) allow for more creative freedom, while lower values (e.g., 0.4) keep the video closer to the source.
With Tile Lora, Denoise is typically set to 1.0 because the Tile Lora model already guides the AI to maintain consistency. The focus is on adding details rather than making significant changes.
Output Quality:
Skip Layer Guidance can significantly improve videos with major issues, but it may introduce new elements (e.g., adding a diamond ring or changing facial features) depending on the Denoise setting.
Tile Lora enhances videos by adding details like sharper eyeshadows, clearer textures, and more defined objects, but it keeps the overall structure intact.
Workflow Steps in ComfyUI
Tutorial Video : https://youtu.be/xY56o8wxQu0
Here’s a step-by-step breakdown of how to set up and use this workflow in ComfyUI:
Load Your Video:
Start by loading the video you want to enhance into ComfyUI. This could be an AI-generated video from tools like LTX or Hanyuan, or any low-quality video.
Choose Your Method:
Decide whether to use Skip Layer Guidance (for fixing broken or morphing videos) or Tile Lora (for adding details to already good videos).
Adjust Settings:
For Skip Layer Guidance, set the blocks (e.g., 9 and 10) and adjust the start/end percentages (e.g., 20% to 80%). Experiment with Denoise settings (e.g., 0.4 to 0.6) to control how much the AI changes the video.
For Tile Lora, load the Tile Lora model and connect it to the workflow. Use InstructPix2Pix Conditioning to pass slightly blurred frames into the AI for detail enhancement.
Run the Workflow:
Once everything is set up, run the workflow. The AI will process the video and generate an enhanced version based on your settings.
Review and Refine:
Check the output and make adjustments if needed. For example, if the video still has issues, you can increase the Denoise value or try a different combination of Skip Layer Guidance and Tile Lora.
When to Use Each Method
Use Skip Layer Guidance when:
The video has significant issues like broken faces, morphing objects, or distorted shapes.
You want the AI to have more freedom to regenerate problematic areas.
Use Tile Lora when:
The video is already in good shape but needs more detail.
You want to maintain the original structure while enhancing textures, colors, and fine details.
Conclusion
This workflow in ComfyUI using Wan 2.1’s Skip Layer Guidance and Tile Lora is a powerful way to enhance AI-generated videos. Whether you’re fixing broken faces or adding fine details, these techniques allow you to improve video quality without starting from scratch. By understanding the differences between Skip Layer Guidance and Tile Lora, you can choose the right method for your specific needs and achieve stunning results.
Feel free to experiment with the settings and share your results! If you have any questions or need further clarification, drop a comment below.
Tutorial Video : https://youtu.be/xY56o8wxQu0
I decided to free up this workflow for everyone to download. I hope everyone can get good AI generate videos without morphing or broken part ;) Attached below, have fun :)
Comments
Benjamin Law
basically, this demo video you have, the resolution is too low. And one more thing, this workflow logic is not for upscale. please remind that.
Darwin Giordano
Hello Thanks for sharing this excellent material, it is really very valuable to study. In my case, I fail to see significant changes at the time of final processing. I suspect it could be because the original video has a very low resolution, although I'm not entirely sure! Would it be possible to do a quick test with this link I share with you? It's a very simple video, it just shows a face in low resolution, and I would like to know what kind of final result could be obtained from it. https://drive.google.com/file/d/1tO8bPoxG_eBd443ggA3yOOswvPKh-AJS/view?usp=drive_link I also take this opportunity to tell you that I became a member on Patreon for a year, so I'll be happy to keep learning with your content. Thank you very much in advance for your help! Regards, Darwin
