Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Releases: erfanzar/eformer

0.0.42

18 Jul 15:35
@erfanzar erfanzar
0.0.42
c458ee3

Choose a tag to compare

View all tags
0.0.42 Latest
Latest

Release Notes (0.0.42, 0.0.41, 0.0.40, ...)

Major Features & Enhancements

  • Distributed Execution & Cluster Management

    • Added RayExecutor for executing remote functions with support for multi-slice and resumable executions.
    • Implemented Ray TPU/GPU/CPU Cluster Setup utilities.
    • Enhanced TPU patcher and cluster utility functions, including dynamic patching and improved command-line configuration.

  • Sharding & Partitioning

    • Introduced new dynamic sharding axes and enhanced partition manager functionality.
    • Added flexible sharding strategies: Data Parallelism (DP), Fully Sharded Data Parallel (FSDP), Tensor Parallelism (TP), Expert Parallelism (EP), Sequence Parallelism (SP).
    • Improved partition axis handling, including helper functions and dataclass-based refactors.

  • PyTree & Serialization

    • Added FrozenPyTree and improved PyTree module for better JAX compatibility.
    • Enhanced serialization capabilities for JAX PyTree-compatible dataclasses.
    • Improved error handling and docstrings in state management.

  • Optimized Operations & Quantization

    • Improved Triton call logging and error handling for more consistent output.
    • Enhanced quantization functions and support for float8, float16, bfloat16, and dynamic loss scaling.
    • Added support for 8-bit and NF4 quantization for efficient model deployment.

  • Documentation & Usability

    • Updated and expanded documentation, including project structure, key features, and API references.
    • Improved README and Sphinx documentation structure.
    • Added license headers and improved code readability and maintainability.

  • General Refactoring & Maintenance

    • Refactored codebase for improved clarity, maintainability, and Python 3.10+ compatibility.
    • Updated dependencies and switched from poetry to uv for build management.
    • Removed deprecated and obsolete modules, streamlined imports, and improved module exports.

Notable Fixes

  • Fixed issues with mesh creation for multi-slice environments.
  • Enhanced error handling for Ray command execution in TPU patcher.
  • Fixed Python 3.10 compatibility issues.
  • Improved logging and validation in sharding and partitioning utilities.
Assets 2
Loading

eformer (EasyDel Former)

04 Feb 10:06
@erfanzar erfanzar
0.0.1
0379bfd

Choose a tag to compare

View all tags
eformer (EasyDel Former)

eformer (EasyDel Former) is a utility library designed to simplify and enhance the development of machine learning models using JAX. It provides a collection of tools for sharding, custom PyTrees, quantization, mixed precision training, and optimized operations, making it easier to build and scale models efficiently.

  • Mixed Precision Training (mpric): Advanced mixed precision utilities supporting float8, float16, and bfloat16 with dynamic loss scaling.
  • Sharding Utilities (escale): Tools for efficient sharding and distributed computation in JAX.
  • Custom PyTrees (jaximus): Enhanced utilities for creating custom PyTrees and ArrayValue objects, updated from Equinox.
  • Custom Calling (callib): A tool for custom function calls and direct integration with Triton kernels in JAX.
  • Optimizer Factory: A flexible factory for creating and configuring optimizers like AdamW, Adafactor, Lion, and RMSProp.
  • Custom Operations and Kernels:
    • Flash Attention 2 for GPUs/TPUs (via Triton and Pallas).
    • 8-bit and NF4 quantization for efficient model.
    • Many others to be added.
  • Quantization Support: Tools for 8-bit and NF4 quantization, enabling memory-efficient model deploymen
Loading