Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@tallamjr
Copy link
Owner

@tallamjr tallamjr commented Oct 17, 2025

This modernises the Tonic build system and CI/CD pipeline whilst maintaining compatibility with existing functionality and dependency versions.

Changes

  • Migrated from legacy pbr/setup.py to modern pyproject.toml with setuptools backend
  • Introduced uv for fast, reproducible dependency management with lockfile
  • Extended platform support: Ubuntu, Windows, macOS
  • Extended Python support: 3.10, 3.11, 3.12
  • Modernised pre-commit hooks (Ruff, mypy)
  • Removed all legacy build files

Status

Dependencies pinned to match develop branch (no breaking changes)
Infrastructure-only changes (dependency updates deferred)

See #1 for full design document and implementation details.

Closes #1

Replace legacy pbr/setup.py build system with standards-compliant
pyproject.toml using setuptools backend. This enables compatibility
with modern Python packaging tools (PEP 517/518/621).

Changes:
- Add pyproject.toml with complete project metadata
- Remove pbr dependency from requirements.txt (delete file, now redundant)
- Update tonic/__init__.py to use importlib.metadata for versioning
- Remove docs/requirements.txt dependency on pbr

The package now builds as a universal wheel (py3-none-any) compatible
with all platforms without platform-specific builds.
Introduce uv.lock to ensure reproducible builds and faster dependency
installation (10-100x faster than pip). The lockfile includes all
dependencies with exact versions for development, testing, and documentation.
Update CI/CD workflow to support modern Python versions and additional
platforms:

Platform support:
- Add macOS testing (Intel and ARM64)
- Maintain Ubuntu and Windows support

Python version support:
- Extend to Python 3.10, 3.11, 3.12, 3.13, 3.14
- Previously supported: 3.10, 3.11, 3.12

Performance improvements:
- Replace pip with uv for 10-100x faster dependency installation
- Add concurrency control to cancel redundant workflow runs
- Enable caching for uv environments

Build improvements:
- Use 'uv build' instead of deprecated 'python setup.py bdist_wheel'
- Update codecov action to v5
- Add permissions for OIDC token (trusted publishing ready)
Pre-commit updates:
- Replace Black + isort with Ruff (10-100x faster, unified tooling)
- Update to Ruff v0.14.0 (was using ancient v0.0.253 equivalent)
- Add mypy v1.18.2 for type checking (informational only)
- Update pre-commit-hooks to v6.0.0
- Add security hooks: detect-private-key, check-symlinks

Documentation updates:
- Add uv installation instructions for developers
- Update platform support matrix (now includes macOS Intel/ARM64)
- Document Python 3.10-3.14 support

All pre-commit hooks respect configuration in pyproject.toml for
consistency.
Remove obsolete pbr-based setup files now that pyproject.toml provides
all package metadata and build configuration. Modern build tools
(pip>=21.3, build, uv) use pyproject.toml directly via PEP 517/518.
Pin torch, torchaudio, and torchvision to 2.3.0 to match develop branch
versions. This ensures the CI modernisation branch focuses purely on
infrastructure changes (pyproject.toml, uv, lockfiles) without
introducing dependency-related breaking changes.

torchaudio 2.9+ introduces breaking changes (download_asset removal,
torchcodec requirement) that should be addressed separately in a future
dependency update PR.
Remove Python 3.13 and 3.14 from CI matrix due to PyTorch 2.3.0 wheel
incompatibility. Add --no-build flag to prevent source builds that cause
6-hour timeouts on macOS-14. Improve Windows diagnostics for libsndfile.

Changes:
- Reduce matrix from 20 to 12 jobs (4 OS × 3 Python versions)
- Add --no-build flag to all uv sync commands
- Improve Windows PowerShell diagnostics
- Document Python/PyTorch compatibility constraints

PyTorch 2.3.0 lacks pre-built wheels for Python 3.13+ (requires PyTorch
2.6+) and Python 3.14+ (requires PyTorch 2.9+). The --no-build flag
prevents uv from attempting multi-hour source builds when wheels are
unavailable.

Fixes: https://github.com/tallamjr/tonic/actions/runs/18601347188

References:
pytorch/pytorch#130249
pytorch/pytorch#156856
https://docs.astral.sh/uv/guides/integration/pytorch/
The --no-build flag was preventing the local tonic package from being
installed as an editable package, causing "no binary distribution" errors.

Since we've already removed Python 3.13/3.14 from the matrix, and PyTorch
2.3.0 has pre-built wheels for Python 3.10-3.12 on all platforms, the
--no-build flag is unnecessary. uv will automatically use wheels for all
dependencies.

The matrix change alone is sufficient to prevent dependency installation
failures and timeouts.

Fixes: error: Distribution `tonic==1.6.0 @ editable+.` can't be installed
because it is marked as `--no-build` but has no binary distribution
Replace deprecated macos-13 and macos-14 runners with:
- macos-15-intel: Intel (x86_64) architecture
- macos-15: Apple Silicon (ARM64) architecture

This provides comprehensive macOS testing on both architectures using
the latest runner images, whilst avoiding the macos-13 deprecation
(retiring December 4th, 2025).

Matrix now tests 4 OS configurations × 3 Python versions = 12 jobs:
- Ubuntu (x86_64)
- Windows (x86_64)
- macOS Intel (x86_64)
- macOS ARM (ARM64)

References:
https://github.blog/changelog/2025-09-19-github-actions-macos-13-runner-image-is-closing-down/
actions/runner-images#13046
PyTorch 2.3.0 lacks wheels for Intel macOS (x86_64) on macOS 15.
Available wheels: Linux x86_64/ARM, macOS ARM64, Windows x86_64.

Using macos-latest (ARM64) for macOS coverage only.
PyTorch 2.3.0 fails on Windows with NumPy 2.0+ due to missing NumPy C API
(_ARRAY_API). Errors manifest as 'RuntimeError: Numpy is not available' in
torch.from_numpy() calls.

Constrain numpy to <2.0 to ensure compatibility across all platforms.

Fixes Windows test failures with torch operations requiring numpy interop.
Configure matplotlib to use non-interactive 'Agg' backend when CI=true,
preventing plt.show() from blocking on macOS where GUI frameworks are
partially available.

Remove Windows skip decorators as tests now work on all platforms.

Fixes: https://github.com/tallamjr/tonic/actions/runs/18630080196
The imageio PNG-FI format (FreeImage plugin) is unavailable on macOS,
causing test_optical_flow to fail. Implemented fallback to pypng library
which provides reliable 16-bit PNG support across all platforms.

References:
imageio/imageio#204
https://stackoverflow.com/questions/32622658/read-16-bit-png-image-file-using-python
The documentation job was failing because sphinx-build was not in the
PATH when running make commands. The virtual environment created by uv
sync needs to be activated before running make, as the Makefile spawns
subshells that require sphinx-build to be available.

Other jobs correctly use 'uv run' to auto-activate the venv, but the
documentation build was calling make directly without activation.
Removed pbr (Python Build Reasonableness) which was an obsolete build
system primarily used by OpenStack projects. The version is now managed
via static declaration in pyproject.toml and accessed through
importlib.metadata, eliminating the need for pbr.sphinxext.

Additionally, fixed Python 3.12+ SyntaxWarnings by converting docstrings
containing LaTeX math notation to raw strings (r"""..."""), which
properly handles backslash characters in the math expressions.

Changes:
- Remove pbr.sphinxext from Sphinx extensions in docs/conf.py
- Remove pbr from .binder/requirements.txt
- Convert docstrings to raw strings in spatial_jitter.py and transforms.py

This aligns with the modernisation to uv and pyproject.toml-based packaging.

References:
https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals
Updated astroid dependency from ~=2.15 to >=3.0,<4.0 to fix
sphinx-autoapi crash when building documentation with Python 3.12.

The error "'TreeRebuilder' object has no attribute 'visit_typealias'"
occurred because astroid 2.15 doesn't support Python 3.12's PEP 695
type alias syntax. Astroid 3.x adds full Python 3.12 support including
proper handling of the new type alias features.

Version constraint allows:
- Astroid 3.0+ for Python 3.12 support
- Below 4.0 to avoid breaking changes
- Compatible with project's Python 3.10-3.12 support range

References:
https://github.com/tallamjr/tonic/actions/runs/18631865540/job/53118068873
https://github.com/pylint-dev/astroid/releases
https://peps.python.org/pep-0695/
Updated sphinx-autoapi from unversioned to >=3.0 to fix the error
"'Module' object has no attribute 'doc'" when building documentation.

Astroid 3.0 removed the deprecated 'doc' attribute in favour of
'doc_node', which broke sphinx-autoapi versions < 3.0. The fix was
committed to sphinx-autoapi on July 7, 2023, and version 3.0.0+
properly supports astroid 3.x by using the new 'doc_node' attribute.

This change ensures compatibility with the previously updated
astroid>=3.0,<4.0 dependency.

References:
https://github.com/tallamjr/tonic/actions/runs/18632248069/job/53118664830
readthedocs/sphinx-autoapi#392
https://github.com/readthedocs/sphinx-autoapi/releases
Updated Sphinx from ~=5.3 to >=6.1,<8.0 and constrained sphinx-autoapi
to >=3.0,<3.6 to resolve dependency conflicts.

The dependency chain requires:
- astroid>=3.0,<4.0 (for Python 3.12 support)
- sphinx-autoapi>=3.0,<3.6 (for astroid 3.x support)
- sphinx>=6.1,<8.0 (required by sphinx-autoapi 3.0-3.5)

Sphinx-autoapi 3.6+ requires Sphinx >=7.4, but we use <3.6 to allow
a wider Sphinx version range (6.1-7.x) for better compatibility.

The previous attempt with unversioned sphinx-autoapi failed because:
  "sphinx-autoapi>=3.0.0,<3.6.0 depends on sphinx>=6.1.0
   And because sphinx-autoapi>=3.6.0 depends on sphinx>=7.4.0 and
   tonic[dev] depends on sphinx>=5.3,<6.dev0, we can conclude that
   sphinx-autoapi>=3.0.0 and tonic[dev] are incompatible."

References:
https://github.com/readthedocs/sphinx-autoapi/releases
@tallamjr tallamjr merged commit f5931b8 into develop Oct 19, 2025
23 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI/CD Modernisation: Migrate to pyproject.toml and uv

2 participants