AnimateDiff + RAVE Workflow Update Ver.2

Benjamin Law

Mentioned in video tutorial, go to ComfyUI Manager , Click install missing node.

AIThinker

Hi Benji. When I tried to run the model following error message poped up: When loading the graph, the following node types were not found: GetNode ReActorFaceSwap SetNode Integer IPAdapterApply Nodes that have failed to load will show as red on the graph.

Benjamin Law

All model files for ipa listed in their Github. Just scroll down to Installation section. Yes,things I mentioned in previous video and I don't repeat again. Very basic stuff if you wanna go for advance workflow.

jamie

Where do you get your clip vision model (SD1.5\model.safetensors)? I can't find any tutorials where you download it, and it's been hard for me to search for

Benjamin Law

What do you mean?

jamie

Thanks for the video! Do you publish the videos and images you use anywhere?

Benjamin Law

hi Ali, mostly that cause by your source image too large. Chop it smaller before you load into the Load image node. You don't need a large image anyway, because the final IPA are going to cut it in 512 by 512.

Ali fi

HI Benji, can you help me for this mismatch? Thxx Error occurred when executing IPAdapterApply: Error(s) in loading state_dict for Resampler: size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1664]). File "E:\MyAi\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\MyAi\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\MyAi\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\MyAi\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py", line 769, in apply_ipadapter self.ipadapter = IPAdapter( ^^^^^^^^^^ File "E:\MyAi\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py", line 370, in __init__ self.image_proj_model.load_state_dict(ipadapter_model["image_proj"]) File "E:\MyAi\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 2153, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(

Benjamin Law

https://github.com/mav-rik/facerestore_cf?tab=readme-ov-file#face-restoration-models Here both codeformer and gfpgan. They have a folder in a models subfolder.

Yu Mu

i have question, Where can I download the 'codeformer.pth' for the 'Face_restore_model' in Reactor Face Swap? I would appreciate it if you could provide the path.

Leo Don Bin

i have supporter membership and dont get support. wtf is going on here?

Benjamin Law

Looks like it is missing some attributes in the setting

KEVIN

Hi dude, thanks and all :)) I got this error what can I do? Error occurred when executing SEGSDetailerForAnimateDiff: 'ModuleList' object has no attribute '1' File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\animatediff_nodes.py", line 92, in doit segs, cnet_images = SEGSDetailerForAnimateDiff.do_detail(image_frames, segs, guide_size, guide_size_for, max_size, seed, steps, cfg, sampler_name, File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\animatediff_nodes.py", line 69, in do_detail enhanced_image_tensor, cnet_images = core.enhance_detail_for_animatediff(cropped_image_frames, model, clip, vae, guide_size, guide_size_for, max_size, File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\core.py", line 395, in enhance_detail_for_animatediff refined_latent = impact_sampling.ksampler_wrapper(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\impact_sampling.py", line 125, in ksampler_wrapper refined_latent = nodes.KSampler().sample(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise)[0] File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\nodes.py", line 1375, in sample return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise) File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\nodes.py", line 1345, in common_ksampler samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\sample_error_enhancer.py", line 9, in informative_sample return original_sample(*args, **kwargs) # This code helps interpret error messages that occur within exceptions but does not have any impact on other operations. File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\sampling.py", line 334, in motion_sample latents = wrap_function_to_inject_xformers_bug_info(orig_comfy_sample)(model, noise, *args, **kwargs) File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\utils_model.py", line 272, in wrapped_function return function_to_wrap(*args, **kwargs) File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\sample.py", line 100, in sample samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed) File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 712, in sample return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed) File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 611, in sample pre_run_control(model, negative + positive) File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 448, in pre_run_control x['control'].pre_run(model, percent_to_timestep_function) File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\utils.py", line 441, in pre_run_inject self.base.pre_run(model, percent_to_timestep_function) File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\controlnet.py", line 296, in pre_run comfy.utils.set_attr(self.control_model, k, self.control_weights[k].to(dtype).to(comfy.model_management.get_torch_device())) File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\utils.py", line 279, in set_attr obj = getattr(obj, name) File "D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1695, in __getattr__ raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")

Leo Don Bin

how can i use this workflow without LCM, the right way?

Benjamin Law

can you send me your workflow, and I take a look, see if I can improve it?

Yu Mu

Thanks Benji. I have a question that may not be related to this topic, but may I ask? I am using the 'OneFormer COCO Segmentor' to separate individuals from the background in videos, but the performance is not satisfactory. Particularly, there is an issue where when legs are spread apart or when hands hold the waist, the background is also recognized as part of the person. If there is another segmentor available, could you please introduce it? How about 'segment anything'? Can it also separate individuals and backgrounds in videos? If so, it would be helpful if you could provide examples.

Nelson

Thanks. Benji. Let me try out the new workflow ;)

Benjamin Law

Hi, you will have to update your Controlnet custom nodes. Or just do Update All in Comfy Manager , that is easier. Thanks

Stephen Fletcher