ruff: incremental adoption #5625

📰 Custom Issue

As #5623 has been merged, we're now in the position where we can slowly adopt incremental compliance of the ruff linter rules.

Our ruff configuration is defined in two files:

• pyproject.toml: this contains the main ruff configuration for the linter (and formatter, when adopted) and permanent rule exclusions
• .ruff.toml: this configuration extends the pyproject.toml and contains temporary rule exclusions

The goal of our ruff adoption is two-fold:

1. Tackle individual rule categories defined in the .ruff.toml on a piecemeal basis e.g., "F" or "ANN" etc
2. Drop third-party packages replaced by ruff.

Tackling Rules

We need to agree on a general way forward here.

However, essentially it will involve the following steps:

1. Comment out a rule in the .ruff.toml.
2. Install ruff in your development environment e.g., pip install ruff.
3. Run ruff e.g., ruff .
4. Address the non-compliances:
   • Consider using ruff --fix . to auto-fix.
   • Manually address the other non-compliances.
5. Either:
   • Remove the rule from .ruff.toml, as we are now compliant.
   • Replace the rule category with one or more exact rules that still require to be addressed e.g., replace "F" (Pyflakes), which represents all pyflakes rules, with exact "Fxxx" rule instances that still require to be ignored.

Note that, rules can be ignored across all of iris by using the [tool.ruff.lint.ignore] in the pyproject.toml or the lint.ignore in the .ruff.toml.

Rules can also be ignored on a per file basis:

• Use [tool.ruff.lint.per-file-ignores] in pyproject.toml.
• Use lint.extend-per-file-ignores in the .ruff.toml.

As a result of tackling the temporary rule exceptions in .ruff.toml we may uncover permanent rule exceptions that we want to adopt. These should be discussed, and if agreed, added and documented in the pyproject.toml.

Drop Third-Party Packages

The ruff linter and formatter replace the following packages, which we can now drop:

• black (Replace black with ruff #5634)
  • Adopt ruff formatter
  • Remove tool.black section from pyproject.toml
  • Remove pre-commit hook from .pre-commit-config.yaml
  • Remove badge from the README.md
• isort (Replace isort with ruff #5633)
  • Remove tool.isort section from pyproject.toml
  • Remove pre-commit hook from .pre-commit-config.yaml
  • Remove "I" rule ignore from .ruff.toml
• flake8 (see Remove flake8 #6889)
  • Remove .flake8 configuration file
  • Update MANIFEST.in
  • Remove pre-commit hook from .pre-commit-config.yaml