Public sync - 2026-07-07#247
Closed
lt-research-bot wants to merge 1 commit into
Closed
Conversation
Copilot started reviewing on behalf of
Michael Kupchick (michaellightricks)
July 7, 2026 16:52
View session
There was a problem hiding this comment.
Pull request overview
This automated sync updates LTX-2’s packaging, docs, and inference/training code, with a major expansion around multi-GPU inference (controller + builders) and the introduction of an optional compiled CUDA kernels package (ltx-kernels). It also centralizes device-specific behavior (CUDA/MPS/CPU) in ltx-core and refactors perturbation masking to support compiled execution without recompilation.
Changes:
- Add multi-GPU inference framework (controller/runner contract, NCCL groups, MGPU builders) and introduce
packages/ltx-kernelsfor required CUDA ops. - Refactor device/allocator handling and perturbation mask handling to improve portability and
torch.compilebehavior. - Refresh README and add/expand documentation for pipelines, training configs, and MGPU internals.
Reviewed changes
Copilot reviewed 155 out of 157 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| README.md | Refresh quickstart + links |
| pyproject.toml | Add opt-in ltx-kernels group; ruff config updates |
| packages/ltx-trainer/src/ltx_trainer/validation_runner.py | Pass perturbation mask build metadata |
| packages/ltx-trainer/pyproject.toml | Bump trainer metadata (version/ruff settings) |
| packages/ltx-trainer/configs/README.md | New configs index + table |
| packages/ltx-trainer/AGENTS.md | Document configs README + link rules |
| packages/ltx-pipelines/src/ltx_pipelines/utils/media_io.py | Audio/video encoding validation + empty-video check |
| packages/ltx-pipelines/src/ltx_pipelines/utils/helpers.py | Use centralized device + memory cleanup helpers |
| packages/ltx-pipelines/src/ltx_pipelines/utils/gpu_model.py | Add allocator trim strategy option |
| packages/ltx-pipelines/src/ltx_pipelines/utils/denoisers.py | Build perturbation config with runtime masks metadata |
| packages/ltx-pipelines/src/ltx_pipelines/utils/constants.py | Add TDP distilled sigma schedule |
| packages/ltx-pipelines/src/ltx_pipelines/utils/args.py | Resolve CLI defaults from checkpoint on argv |
| packages/ltx-pipelines/src/ltx_pipelines/utils/allocator_trim_strategy.py | New enum controlling teardown behavior |
| packages/ltx-pipelines/src/ltx_pipelines/ti2vid_two_stages.py | Use from_checkpoint; add trim strategy; CLI param resolution |
| packages/ltx-pipelines/src/ltx_pipelines/ti2vid_two_stages_hq.py | Same as above for HQ pipeline |
| packages/ltx-pipelines/src/ltx_pipelines/ti2vid_one_stage.py | Use from_checkpoint; add trim strategy; CLI param resolution |
| packages/ltx-pipelines/src/ltx_pipelines/t2a_one_stage.py | Use from_checkpoint; add trim strategy; CLI param resolution |
| packages/ltx-pipelines/src/ltx_pipelines/retake.py | Use from_checkpoint; add trim strategy |
| packages/ltx-pipelines/src/ltx_pipelines/multigpu/vae_builders.py | New distributed decoder builder wrapper |
| packages/ltx-pipelines/src/ltx_pipelines/multigpu/tdp_builder.py | New tiled data-parallel transformer builder |
| packages/ltx-pipelines/src/ltx_pipelines/multigpu/sp_builder.py | New sequence-parallel transformer builder |
| packages/ltx-pipelines/src/ltx_pipelines/multigpu/runner.py | New MGPU runner contract + RunnerError |
| packages/ltx-pipelines/src/ltx_pipelines/multigpu/nccl_groups.py | New NCCL group container + creation helper |
| packages/ltx-pipelines/src/ltx_pipelines/multigpu/delegating_builder.py | New shared builder delegation base |
| packages/ltx-pipelines/src/ltx_pipelines/multigpu/bp_gemma_builder.py | New batch-parallel Gemma builder |
| packages/ltx-pipelines/src/ltx_pipelines/multigpu/_broadcast.py | New store-based broadcast signaling |
| packages/ltx-pipelines/src/ltx_pipelines/multigpu/init.py | Export MGPU public API |
| packages/ltx-pipelines/src/ltx_pipelines/lipdub.py | Use from_checkpoint; add trim strategy; CLI param resolution |
| packages/ltx-pipelines/src/ltx_pipelines/keyframe_interpolation.py | Use from_checkpoint; add trim strategy; CLI param resolution |
| packages/ltx-pipelines/src/ltx_pipelines/ic_lora.py | Use from_checkpoint; add trim strategy; CLI param resolution |
| packages/ltx-pipelines/src/ltx_pipelines/hdr_ic_lora.py | Use from_checkpoint; add trim strategy; device cache cleanup |
| packages/ltx-pipelines/src/ltx_pipelines/distilled.py | Use from_checkpoint; add trim strategy; CLI param resolution |
| packages/ltx-pipelines/src/ltx_pipelines/a2vid_two_stage.py | Use from_checkpoint; add trim strategy |
| packages/ltx-pipelines/src/ltx_pipelines/init.py | Lazy pipeline imports (PEP 562) |
| packages/ltx-pipelines/pyproject.toml | Add cloudpickle dependency; bump metadata |
| packages/ltx-pipelines/docs/pipeline-selection.md | New pipeline selection guide |
| packages/ltx-pipelines/docs/multimodal-guidance.md | New multimodal guidance doc |
| packages/ltx-pipelines/docs/multigpu/sequence-parallel.md | New SP deep-dive doc |
| packages/ltx-pipelines/docs/multigpu/README.md | New MGPU overview + requirements |
| packages/ltx-pipelines/docs/multigpu/pipeline-setup.md | New MGPU setup pattern doc |
| packages/ltx-pipelines/docs/multigpu/gemma.md | New MGPU Gemma doc |
| packages/ltx-pipelines/docs/multigpu/distributed-decoder.md | New distributed decoder doc |
| packages/ltx-pipelines/docs/installation.md | New installation + CLI usage doc |
| packages/ltx-pipelines/docs/conditioning.md | New conditioning overview doc |
| packages/ltx-pipelines/CLAUDE.md | Document from_checkpoint usage and semantics |
| packages/ltx-kernels/src/ltx_kernels/blockwise/init.py | New blockwise kernels public API |
| packages/ltx-kernels/src/ltx_kernels/arch.py | New GPU arch detection helper |
| packages/ltx-kernels/src/ltx_kernels/init.py | New ltx-kernels package init |
| packages/ltx-kernels/README.md | New build/install/testing docs |
| packages/ltx-kernels/pyproject.toml | New kernel package metadata |
| packages/ltx-kernels/MANIFEST.in | Include CUDA/C++ sources in sdist |
| packages/ltx-kernels/csrc/ops/rms_norm_split_rope.cpp | New fused op host entry |
| packages/ltx-kernels/csrc/ops/rms_norm_rope.cpp | New fused op host entry |
| packages/ltx-kernels/csrc/ops/ops_api.cpp | PyBind registrations for ops |
| packages/ltx-kernels/csrc/ops/include/static_switch.h | New static-switch macro header |
| packages/ltx-kernels/csrc/ops/include/fast_hadamard_transform.h | New kernel param structs header |
| packages/ltx-kernels/csrc/ops/fp6_bitpack.cpp | New FP6 pack/unpack bindings |
| packages/ltx-kernels/csrc/include/event.hpp | New CUDA stream/event helpers |
| packages/ltx-kernels/csrc/include/cuda/exceptions.cuh | New CUDA exception/assert macros |
| packages/ltx-kernels/csrc/include/cuda/configs.cuh | New kernel configuration constants |
| packages/ltx-kernels/csrc/blockwise/utils.hpp | New blockwise utility header |
| packages/ltx-kernels/csrc/blockwise/static_switch.hpp | New blockwise static-switch macros |
| packages/ltx-kernels/csrc/blockwise/sm90_fp8_gemm_1d2d_bias.hpp | New SM90 FP8 GEMM wrapper |
| packages/ltx-kernels/csrc/blockwise/sm89_fp8_gemm_1d2d.hpp | New SM89 FP8 GEMM wrapper |
| packages/ltx-kernels/csrc/blockwise/kernels/geforce/static_switch.h | New GeForce static-switch header |
| packages/ltx-kernels/csrc/blockwise/kernels/geforce/mma_traits_sm89_fp16.hpp | New SM89 MMA traits |
| packages/ltx-kernels/csrc/blockwise/kernels/geforce/mma_sm89_fp16.hpp | New SM89 MMA asm wrappers |
| packages/ltx-kernels/csrc/blockwise/kernels/deep_gemm/include/deep_gemm/common/types.hpp | New deep_gemm types |
| packages/ltx-kernels/csrc/blockwise/kernels/deep_gemm/include/deep_gemm/common/reduction.cuh | New deep_gemm reductions |
| packages/ltx-kernels/csrc/blockwise/kernels/deep_gemm/include/deep_gemm/common/epilogue_utils.cuh | New deep_gemm epilogue utils |
| packages/ltx-kernels/csrc/blockwise/kernels/deep_gemm/include/deep_gemm/common/cute_tie.cuh | New tuple tie helper |
| packages/ltx-kernels/csrc/blockwise/exceptions.hpp | New blockwise exceptions/asserts |
| packages/ltx-kernels/csrc/blockwise/config.hpp | New GEMM config structs |
| packages/ltx-kernels/csrc/blockwise/api.cpp | PyBind registrations for blockwise GEMM |
| packages/ltx-kernels/csrc/all2all/cuda/api.cuh | New all2all CUDA API declarations |
| packages/ltx-core/src/ltx_core/tiling.py | Add balanced tile split helper |
| packages/ltx-core/src/ltx_core/text_encoders/gemma/encoders/base_encoder.py | Fix fork_rng device pinning off-CUDA |
| packages/ltx-core/src/ltx_core/quantization/trtllm_scaled_usable.py | Remove TRT-LLM usability probe |
| packages/ltx-core/src/ltx_core/quantization/fp8_scaled_mm.py | Always use torch _scaled_mm path |
| packages/ltx-core/src/ltx_core/quantization/blockwise/init.py | New lazy-import blockwise quantization API |
| packages/ltx-core/src/ltx_core/multigpu/vae/init.py | New MGPU VAE exports |
| packages/ltx-core/src/ltx_core/multigpu/transformer/tiled_data_parallel.py | New TDP model wrapper |
| packages/ltx-core/src/ltx_core/multigpu/transformer/init.py | New transformer MGPU exports |
| packages/ltx-core/src/ltx_core/multigpu/gemma/loader.py | New accelerate device_map Gemma loader |
| packages/ltx-core/src/ltx_core/multigpu/gemma/broadcast_wrapper.py | New broadcast wrapper base |
| packages/ltx-core/src/ltx_core/multigpu/gemma/batch_parallel_wrapper.py | New batch-parallel Gemma wrapper |
| packages/ltx-core/src/ltx_core/multigpu/gemma/accelerate_wrapper.py | New accelerate/sharded Gemma wrapper |
| packages/ltx-core/src/ltx_core/multigpu/gemma/init.py | New Gemma MGPU exports |
| packages/ltx-core/src/ltx_core/multigpu/init.py | New MGPU package exports |
| packages/ltx-core/src/ltx_core/model/transformer/transformer_args.py | Simplify mask API; remove device/dtype plumbing |
| packages/ltx-core/src/ltx_core/model/transformer/model.py | Add num_blocks; optional perturbations; eager mask materialization |
| packages/ltx-core/src/ltx_core/model/transformer/compiling.py | New compiled-block perturbation processor |
| packages/ltx-core/src/ltx_core/model/model_protocol.py | Add num_blocks; perturbations optional in protocol |
| packages/ltx-core/src/ltx_core/model/audio_vae/vocoder.py | Handle MPS fp32 path safely |
| packages/ltx-core/src/ltx_core/modality_tiling.py | Use RuntimeError for invariant violations |
| packages/ltx-core/src/ltx_core/loader/single_gpu_model_builder.py | Expose builder checkpoint property |
| packages/ltx-core/src/ltx_core/loader/primitives.py | Add checkpoint to builder protocol |
| packages/ltx-core/src/ltx_core/loader/fuse_loras.py | Device-aware fuse rule selection + fusion device routing |
| packages/ltx-core/src/ltx_core/devices.py | New centralized device abstraction |
| packages/ltx-core/src/ltx_core/block_streaming/wrapper.py | Delegate sync/event logic to provider |
| packages/ltx-core/src/ltx_core/block_streaming/utils.py | Degrade pin_memory off-CUDA |
| packages/ltx-core/src/ltx_core/block_streaming/source.py | Abstract stream event type |
| packages/ltx-core/src/ltx_core/block_streaming/pool.py | Use generic stream sync events |
| packages/ltx-core/src/ltx_core/block_streaming/disk.py | Support MPS transfers + non_blocking gating |
| packages/ltx-core/src/ltx_core/block_streaming/builder.py | Store cpu_slots_count default; generic stream sync; device sync abstraction |
| packages/ltx-core/src/ltx_core/batch_split.py | Slice perturbations via batch_slice; propagate num_blocks; allow None |
| packages/ltx-core/README.md | Update quantization docs wording |
| packages/ltx-core/pyproject.toml | Bump metadata; add MPS SDPA dependency |
| .gitignore | Adjust LFS golden asset allowlist |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
56
to
58
| [tool.ruff] | ||
| target-version = "1.1.6" | ||
| target-version = "v1.1.7" | ||
| line-length = 120 |
| } while (0) | ||
| #endif | ||
|
|
||
| } // namespace deep_gemm |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request was created automatically.