@tomwhite Just out of curiosity, how did you find this? An extrapolation of https://zarr.readthedocs.io/en/stable/user-guide/extending.html#custom-codecs to pipelines?

Yes, and I saw that it was supported in the zarr-python source at https://github.com/zarr-developers/zarr-python/blob/main/src/zarr/registry.py#L113.

I would also like to test this. I'm not sure how, but I think a good start would be changing the test_pipeline.py to not have the import zarrs as well as removing the imports from zarrs in conftest.py (which don't appear to be serving a purpose?). Does that sound reasonable? For me on main removing these causes things to fail as expected but not with your branch i.e., this mechanism is working.

Absolutely. I had manually tested it using another project that depends on zarrs-python, and removing import zarrs worked fine with this change (but not without it). I have updated this PR to remove the imports from zarrs in conftest.py, but I've left the one in test_pipeline.py as it is used here: https://github.com/zarrs/zarrs-python/blob/main/tests/test_pipeline.py#L279.

The way we test critical stuff like this in scanpy is by precomputing the outcomes at import time of the tests, then testing the data, somthing like the following.

import sys

import zarr

# IMPORTED_MODULES must be determined first
IMPORTED_MODULES = list(sys.modules)
with zarr.config.set({"codec_pipeline.path": "zarrs.ZarrsCodecPipeline"}):
    try:
        zarr.array([])
    except zarr.core.config.BadConfigError:
        IS_REGISTERED = False
    else:
        IS_REGISTERED = True


def test_registerable_when_not_imported():
   assert "zarrs" not in IMPORTED_MODULES
   assert IS_REGISTERED

caveat: there are many ways one can accidentally import a module, so a clean subprocess-based approach could also work (if we do async testing here, it wouldn’t even slow us down as we could run things async):

data/check-registry.py

import json

# imported_modules must be determined first
imported_modules = list(sys.modules)
with zarr.config.set({"codec_pipeline.path": "zarrs.ZarrsCodecPipeline"}):
    try:
        zarr.array([])
    except zarr.core.config.BadConfigError:
        is_registered = False
    else:
        is_registered = True

print(json.dumps(dict(
    imported_modules=imported_modules,
    is_registered=is_registered,
)))

test_registry.py

import sys
import json
from subprocess import run
from pathlib import Path

HERE = Path(__file__).parent

def test_registerable_when_not_imported():
    proc = run([sys.executable, "-I", HERE / "data/check-registry.py"], check=True)
    results = json.loads(proc.stdout)
    assert "zarrs" not in results["imported_modules"]
    assert result["is_registered"]

Thanks @flying-sheep! I've added the test you suggested.

Thanks! Also neat, looks like I only missed a capture_output=True in the code I typed into the GitHub comment box haha.

@LDeakin could you please install the pre-commit.ci GitHub app so we get autofixes?

I think it needs to be installed by an org admin, so only you can do it.

@tomwhite Nice find!

@flying-sheep - pre-commit CI is now enabled