Innovate Futures @ Benji
patreon
Sonic In ComfyUI - A Truly Awesome Portrait Animation AI Talking Avatar Framework
🕑 Added 2025-02-13 13:00:15 +0000 UTC
Tutorial Video: https://youtu.be/8DTAAOhRx3Q
Discover Sonic AI, an open-source framework that transforms static images into animated avatars using audio input—no subscriptions needed! In this tutorial, learn how to install and use Sonic AI in ComfyUI, leveraging Stable Video Diffusion (SVD) for smooth facial animations, lip-syncing, and head movements. Perfect for content creators and AI enthusiasts, this tool rivals paid services like Hydra and Live Portrait while being fully customizable and free! https://jixiaozhong.github.io/Sonic/
Who is this content suitable for?
AI developers, content creators, freelancers, and tech enthusiasts looking to create professional talking avatars without costly subscriptions. Ideal for ComfyUI users familiar with SVD or those exploring AI-driven video generation.
Why does it matter?
Sonic AI democratizes high-quality avatar animation, offering studio-grade results with minimal setup. Its audio-driven workflow, open-source flexibility, and compatibility with tools like Whisper Tiny and SVD XT 1.1 make it a game-changer for video production, presentations, and social media content. Stay ahead of the curve and cut costs with this cutting-edge AI solution!
ComfyUI Setup Steps:
https://github.com/smthemex/ComfyUI_Sonic/tree/main
Step 1
--------------------------------------------------
git clone https://github.com/smthemex/ComfyUI_Sonic.git
pip install -r requirements.txt
Step 2
--------------------------------------------------
https://drive.google.com/drive/folders/1oe8VTPUy0-MHHW2a_NJ1F8xL-0VN5G7W
https://huggingface.co/openai/whisper-tiny/tree/main
In : ComfyUI/models/sonic/
|-- audio2bucket.pth
|-- audio2token.pth
|-- unet.pth
|-- yoloface_v5m.pt
|-- whisper-tiny/
|--config.json
|--model.safetensors
|--preprocessor_config.json
|-- RIFE/
|--flownet.pkl
Step 3
--------------------------------------------------
https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt-1-1/tree/main
ComfyUI/models/checkpoints
├── svd_xt.safetensors or svd_xt_1_1.safetensors
Attached the basic workflow used in this tutorial:
Comments
Coby Christiansen
SONICTLoader 'NoneType' object has no attribute 'seek'. You can only torch.load from a file that is seekable. Please pre-load the data into a buffer like io.BytesIO and try to load from it instead. Anyone running into this error?
Benjamin Law
looks like you haven't check out the video, and even this post have mentioned.
Coby Christiansen
ImageOnlyCheckpointLoader Model in folder 'checkpoints' with filename 'svd_xt_1_1.safetensors' not found.
