Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Convert kohya/CivitAI LoRA safetensors for Anima#1

Open
ghunkins wants to merge 2 commits into
rmatif:animafrom
ghunkins:feat/ghunkins/anima-kohya-lora-loader
Open

Convert kohya/CivitAI LoRA safetensors for Anima#1
ghunkins wants to merge 2 commits into
rmatif:animafrom
ghunkins:feat/ghunkins/anima-kohya-lora-loader

Conversation

@ghunkins

Copy link
Copy Markdown

Summary

CivitAI Anima LoRAs are trained with kohya-ss/sd-scripts, which writes keys as lora_unet_<munged_name>.lora_{down,up}.weight (+ optional .alpha), with dots in module paths replaced by underscores. The existing AnimaLoraLoaderMixin.lora_state_dict only handled the diffusion_model.* prefix with peft-style lora_A/lora_B suffixes, so kohya-format keys slipped through load_lora_weights silently and no PEFT adapter got registered. The user then sees a confusing Adapter name(s) {...} not in the list of present adapters later from set_adapters.

Changes

  • _convert_kohya_anima_lora_to_diffusers (new) in lora_conversion_utils.py:
    • Pairs .lora_down.weight / .lora_up.weight / .alpha by base key.
    • Routes lora_unet_blocks_{i}_* to transformer.transformer_blocks.{i}.* (full self/cross attn, MLP layer1/layer2, adaLN modulation).
    • Routes lora_unet_llm_adapter_blocks_{i}_* to text_conditioner.blocks.{i}.*. Note: the LLM adapter uses o_proj (not the DiT's output_proj) and mlp.{0,2} (not the DiT's mlp.{layer1,layer2}). Verified against the live circlestone-labs/Anima/anima-base-v1.0.safetensors header.
    • Handles the non-block DiT leaves kohya excludes by default but some trainers include: x_embedder_proj_1, t_embedder_1_linear_{1,2}, final_layer_linear, final_layer_adaln_modulation_{1,2}.
    • Applies kohya alpha / rank scaling, distributing across down/up to avoid overflow (same formula as _convert_kohya_flux_lora_to_diffusers).
    • Skips lora_te_* Qwen3 text-encoder keys with a warning since the Anima loader does not load LoRAs into the text encoder.
  • _convert_non_diffusers_anima_lora_to_diffusers (extended): also accepts kohya .lora_down/.lora_up/.alpha suffixes when the prefix is diffusion_model. (some trainers combine the two).
  • AnimaLoraLoaderMixin.lora_state_dict: dispatches to the kohya converter when any key starts with lora_unet_ or lora_te_.

Test plan

  • test_lora_state_dict_conversion (existing) — still passes.
  • test_load_lora_weights (existing) — still passes.
  • test_kohya_lora_state_dict_conversion (new) — covers DiT, LLM adapter, alpha scaling, and the TE-skip path.
  • test_load_lora_weights_kohya_format (new) — end-to-end PEFT registration with kohya keys.
  • Manual end-to-end against the real anima-base-v1.0.safetensors: 515/516 Linear modules round-trip cleanly; the 1 skipped is llm_adapter.embed (nn.Embedding), which kohya excludes by default anyway.

🤖 Generated with Claude Code

CivitAI Anima LoRAs are trained with kohya-ss/sd-scripts which writes keys
as `lora_unet_<munged_name>.lora_{down,up}.weight` (+ optional `.alpha`),
with dots in module paths replaced by underscores. The existing Anima
LoRA loader only handled the `diffusion_model.*` prefix with peft-style
`lora_A`/`lora_B` suffixes, so kohya keys slipped through `load_lora_weights`
silently. No PEFT adapter was registered for the file, surfacing later as
`Adapter name(s) {...} not in the list of present adapters` from
`set_adapters`.

Adds `_convert_kohya_anima_lora_to_diffusers`, wires it into
`AnimaLoraLoaderMixin.lora_state_dict`, and extends the existing
`diffusion_model.*` converter to also accept kohya `.lora_down`/`.lora_up`/
`.alpha` suffixes (some trainers combine the two). The kohya converter
routes DiT blocks to `transformer.*`, LLM-adapter blocks to
`text_conditioner.*` (preserving `o_proj` and `mlp.{0,2}`, which differ
from the DiT's `output_proj` and `mlp.{layer1,layer2}` — verified against
the live anima-base-v1.0 safetensors header), and skips Qwen3
`lora_te_*` keys with a warning since the Anima loader does not load
LoRAs into the text encoder.

Adds `test_kohya_lora_state_dict_conversion` and
`test_load_lora_weights_kohya_format`.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
`utils/check_support_list.py` requires every LoRA mixin defined in
`src/diffusers/loaders/lora_pipeline.py` to appear in
`docs/source/en/api/loaders/lora.md`. Add the missing `AnimaLoraLoaderMixin`
entry and `[[autodoc]]` block.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label May 27, 2026
@ghunkins

Copy link
Copy Markdown
Author

Still testing, but here is a more robust conversion script:

https://gist.github.com/ghunkins/05839b2e6b0111be3a98ba2af13e3dda

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation lora tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant