Thanks to visit codestin.com
Credit goes to github.com

Skip to content

No longer able to apply LoRAs #364

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
SkutteOleg opened this issue Aug 24, 2024 · 3 comments
Closed

No longer able to apply LoRAs #364

SkutteOleg opened this issue Aug 24, 2024 · 3 comments

Comments

@SkutteOleg
Copy link
Contributor

When I try to apply LoRA to a quantized model, I get the following error:

sd.exe -m sd_xl_turbo_1.0.q8_0.gguf -p "a lovely cat<lora:res-adapter-v2-sdxl:1>" --taesd taesdxl.q8_0.gguf --steps 1 --cfg-scale 1
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce GTX 1660 SUPER, compute capability 7.5, VMM: yes
[INFO ] stable-diffusion.cpp:184  - loading model from 'sd_xl_turbo_1.0.q8_0.gguf'
[INFO ] model.cpp:786  - load sd_xl_turbo_1.0.q8_0.gguf using gguf format
WARNING: Behavior may be unexpected when allocating 0 bytes for ggml_calloc!
[INFO ] stable-diffusion.cpp:224  - Version: SDXL
[INFO ] stable-diffusion.cpp:255  - Weight type:                 q8_0
[INFO ] stable-diffusion.cpp:256  - Conditioner weight type:     q8_0
[INFO ] stable-diffusion.cpp:257  - Diffsuion model weight type: q8_0
[INFO ] stable-diffusion.cpp:258  - VAE weight type:             f32
[INFO ] tae.hpp:204  - loading taesd from 'taesdxl.q8_0.gguf', decode_only = true
[INFO ] model.cpp:786  - load taesdxl.q8_0.gguf using gguf format
WARNING: Behavior may be unexpected when allocating 0 bytes for ggml_calloc!
[INFO ] tae.hpp:226  - taesd model loaded
[INFO ] stable-diffusion.cpp:486  - total params memory size = 3763.23MB (VRAM 3763.23MB, RAM 0.00MB): clip 835.53MB(VRAM), unet 2925.36MB(VRAM), vae 2.34MB(VRAM), controlnet 0.00MB(VRAM), pmid 0.00MB(VRAM)
[INFO ] stable-diffusion.cpp:490  - loading model from 'sd_xl_turbo_1.0.q8_0.gguf' completed, taking 3.27s
[INFO ] stable-diffusion.cpp:517  - running in eps-prediction mode
[WARN ] stable-diffusion.cpp:617  - In quantized models when applying LoRA, the images have poor quality.
[INFO ] stable-diffusion.cpp:635  - Attempting to apply 1 LoRAs
[INFO ] model.cpp:789  - load res-adapter-v2-sdxl.safetensors using safetensors format
[INFO ] lora.hpp:33   - loading LoRA from 'res-adapter-v2-sdxl.safetensors'
D:\a\stable-diffusion.cpp\stable-diffusion.cpp\ggml\src\ggml-backend.c:224: GGML_ASSERT(buf != NULL && "tensor buffer not set") failed

This worked before. Same issue was encountered here: #291 (comment)
This commit 29ec316 appears to have broken something.

If it helps, here are the LoRAs and models I tried this with:

  • res-adapter LoRA
  • SDXL-Turbo (Q8_0)
  • SDXL-Lightning (Q8_0)
@grauho
Copy link
Contributor

grauho commented Aug 25, 2024

I haven't had time to review all the new flux code but I solved this issue by adding "ggml_build_forward_expand(gf, zero_index);" to the end of build_lora_graph. My logic was that all the other places that use set_backend_tensor_data seem to resolve with that but for whatever reason the lora loading did not. I don't know if this is the intended solution but it did get LoRAs loading again for me.

@leejet
Copy link
Owner

leejet commented Aug 25, 2024

It should be fixed now.

@SkutteOleg
Copy link
Contributor Author

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@leejet @SkutteOleg @grauho