…16 (Nerogar#147…"

This reverts commit 574ec55.

Several model savers (Ernie, Flux2, Z-Image, ...) duplicate the same
deepcopy + tokenizer __deepcopy__ workaround to produce a dtype-converted
copy of a diffusers pipeline for saving. Extract it into a shared helper
so new savers can reuse it.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

Adds a tokenizer_attrs parameter (default ("tokenizer",)) so savers with
extra/different tokenizer attributes (Flux's tokenizer_2, SD3's
tokenizer_3, HiDream's tokenizer_3/tokenizer_4) can use the same helper.
Replaces the duplicated deepcopy + tokenizer __deepcopy__ workaround in
Chroma, Ernie, Flux, Flux2, HiDream, HunyuanVideo, PixArtAlpha, Qwen,
Sana, StableDiffusion3 and Z-Image with calls to the shared helper.
No behavior change.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

Move the per-callsite checkpointing_or_offloading_enabled() guard into
enable_checkpointing() itself, so every Base*Setup can call
enable_checkpointing_for_* unconditionally. Also extend the central gate
to allow a compile-only path (no checkpointing/offloading, but still
per-layer torch.compile wrapping) when config.compile is set.

Three direct diffusers enable_gradient_checkpointing() calls (SD/SDXL
unet, Wuerstchen v2 prior) keep their explicit guard since they bypass
this central mechanism.

Ports the Ideogram 4 image generation model into OneTrainer, including
model loading/saving, data loading, sampling, training setups for LoRA
and Fine Tune, and corresponding UI and preset additions.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

# Conflicts:
#	modules/modelSetup/BaseErnieSetup.py
#	modules/modelSetup/BaseWuerstchenSetup.py
#	modules/util/checkpointing_util.py
#	training_presets/#flux2 LoRA 8GB.json

# Conflicts:
#	modules/ui/TimestepDistributionWindow.py
#	modules/ui/TrainingTab.py

Mirrors upstream commit 75a44d2, which converted the rest of the
codebase from the trailing factory.register() call to the @factory.register
decorator form.

Wire attention backend selection into Ideogram setup (both transformer and unconditional_transformer, mask=False) and pass controller through the Ideogram UI base2 frame.

FusedModuleGroup._leaf_forward_N called the inner peft module's forward
once per leaf, and that forward's orig_forward is the synthetic fused
base recomputing all N leaves' base output every time -- N real base
computes per leaf, N^2 total for a group of N. Add PeftBase.delta_forward,
an optional hook returning just the adapter's own contribution (the term
added to orig_forward(x)) without touching the base. LoRAModule and
LoHaModule implement it (their deltas never read the base weight);
FusedModuleGroup uses it, when available, to add each leaf's real,
unfused base once instead. DoRA/OFT/LoKr keep the slower generic path
(delta_forward defaults to None) since their forwards recompose the base
weight itself and aren't expressible this way.

Also replace the functools.partial(self._leaf_forward, leaf_index) hook
with four fixed _leaf_forward_0..3 methods: torch.compile guards on
leaf.forward's function identity, and a partial rebuilt on every
hook_to_module() call was a fresh object each time, forcing a recompile
on every hook/unhook cycle.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

FusedModuleGroup._leaf_forward_N called the inner peft module's forward
once per leaf, and that forward's orig_forward is the synthetic fused
base recomputing all N leaves' base output every time -- N real base
computes per leaf, N^2 total for a group of N. Add PeftBase.delta_forward,
an optional hook returning just the adapter's own contribution (the term
added to orig_forward(x)) without touching the base. LoRAModule and
LoHaModule implement it (their deltas never read the base weight);
FusedModuleGroup uses it, when available, to add each leaf's real,
unfused base once instead. DoRA/OFT/LoKr keep the slower generic path
(delta_forward defaults to None) since their forwards recompose the base
weight itself and aren't expressible this way.

Also replace the functools.partial(self._leaf_forward, leaf_index) hook
with four fixed _leaf_forward_0..3 methods: torch.compile guards on
leaf.forward's function identity, and a partial rebuilt on every
hook_to_module() call was a fresh object each time, forcing a recompile
on every hook/unhook cycle.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

Remove non-explanatory/out-of-place comments, drop the untrue Cascade
bracket note on LEGACY_LORA, and revert unnecessary learning_rate
notation-only changes in a few presets.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

…uess

Mirrors the saver's _convert_legacy opt-in default: a model gets LEGACY
load support only by explicitly declaring its historical layout, not by
inheriting a generic reconstruction that happens to be right for most
models but silently wrong for any that isn't. Flux/Flux2/HunyuanVideo/SD3
opt in via the extracted _mixture_legacy_conversion() helper; Sana's now-
redundant None override is removed.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

…ansformer isn't loaded

Instead of refusing cfg_scale > 1.0 without the unconditional transformer, encode an empty ("") prompt and run it through the conditional transformer as the negative branch, like standard CFG.

Add a ModelType.supported_training_methods() that enumerates every
model type explicitly, raising on an unknown type rather than
defaulting. Collapse ModelTab's per-type __setup_*_ui methods into one
build_content that derives the has_* widget flags from
ModelType.model_parts(), and collapse TopBar's per-type training-method
dispatch to build its dropdown from supported_training_methods().

Co-Authored-By: Claude Sonnet 5 <[email protected]>

Resolves conflicts by keeping split-offload's per-component offload_fraction API
and reconciling it with centralize-model-type's ModelType/ModelTabController
centralization. Also ports the per-component checkpointing API (3-arg
enable_checkpointing_for_* calls, no global gradient_checkpointing.enabled()
guard) to Anima/Krea2, which were merged in as new files and had missed it, and
fixes the Krea2 16GB LoRA preset to the current per-part offload_fraction format.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

BaseAnimaSetup/BaseKrea2Setup were added wholesale by the centralize-model-type
merge and kept the old enabled()-guarded call pattern instead of the new
part-based one applied to Chroma/Qwen during conflict resolution.

# Conflicts:
#	modules/modelSetup/BaseChromaSetup.py
#	modules/modelSetup/BaseFlux2Setup.py
#	modules/modelSetup/BaseQwenSetup.py
#	modules/modelSetup/BaseZImageSetup.py

- default activation_offloading to False for fresh configs (it previously
  only took effect combined with layer offloading, so a fresh fine-tune
  now offloads activations out of the box, which enable_activation_offloading
  never did)
- hide the no-op "Layer Offload Fraction" control for CLIP text encoders,
  whose conductor is discarded in setup_optimizations
- narrow the caching_threads/layer-offloading guard in create_data_loader
  to the text-encoder parts that actually run in the caching dataloader's
  worker threads, via a new ModelType.text_encoder_parts() helper
- drop the dead model.vae.enable_gradient_checkpointing() call: the base
  SD setup only ever decodes the VAE under torch.no_grad(), so checkpointing
  it never had any effect; VAEs are small enough that FINE_TUNE_VAE doesn't
  need it either

…ding from train

- do not gate checkpointing or offloading on a part's train flag: grad can
  flow through a frozen part (embedding training runs grad through the frozen
  denoiser and text encoders), where checkpointing and offloading still save
  VRAM. checkpointing_enabled() -> gradient_checkpointing; offloading_enabled()
  -> offload_fraction > 0 or activation_offloading. Restores pre-branch
  behaviour for embedding training.
- reject offloading with checkpointing disabled (raise NotImplementedError)
  instead of silently forcing checkpointing on: the current offload modes need
  the use_reentrant recompute to service the backward pass.
- CLIP encoder checkpointing passes offload_enabled=False so a migrated
  offload_fraction can never build a self-activating conductor for a
  non-offloadable text encoder.
- hide the dead Layer Offload Fraction / Offload Activations controls for the
  SD/SDXL unet and Wuerstchen prior (no conductor), and gate the SD/SDXL native
  gradient checkpointing on checkpointing_enabled() so a dead offload toggle
  can no longer flip it on.
- derive ModelType.has_multiple_text_encoders() from model_parts() instead of a
  parallel is_*() chain.
- remove the dead LayerOffloadConductor.layer_offload_activated().

Co-Authored-By: Claude Opus 4.8 <[email protected]>

Resolves LoRA-fusion helper conflicts by taking ideogram-base's generalized
convert_util constants/check_fusion_match, removes now-dead LoRAModule
._check_fusion_match, and combines independently-added params/functions from
both branches (BaseTrainingTabView text-encoder frame, checkpointing_util's
per-model transformer checkpointing, ModelTabController's transformer-override
support). The requirements-global.txt mgds pin is left at ideogram-base's
upstream commit for now; it lacks the Ideogram-specific EncodeIdeogramText
module until the mgds branches are combined.

…et cleanup

Squashes: address PR review (revert unrelated merge-in changes), bump mgds
pin for EncodeIdeogramText, enforce checkpointing requirement per-call
instead of at setup, match current model API (conductor rename, drop dead
TE checks), move Ideogram presets into "Ideogram 4" folder, rename Chroma1
folder to Chroma, fail loud at setup for trained parts under offloading.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

BaseIdeogramSetup never set text_encoder_offload_conductor, even though
IdeogramModel has the field and the Qwen3VL encoder helper (added for
Krea2, which shares the same text encoder) already exists.

Co-Authored-By: Claude Sonnet 5 <[email protected]>