Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@jhamman
Copy link
Member

@jhamman jhamman commented Oct 10, 2024

Previously, Zarr was using a home rolled to_thread function to offload work to an implicit executor. This PR changes this in three main ways:

  1. Use asyncio.to_thread instead of the home rolled version
  2. Setup a configurable ThreadPoolExecutor alongside the IOthread and IOloop. The number of threads can be configured using threading.max_workers.
  3. Provide a teardown function (zarr.core.sync.cleanup_resources) to explicitly cleanup resources created by Zarr at runtime. This function is executed at shutdown (via atexit) and can also be triggered manually (e.g. in the dask test suite)

TODO:

  • Add unit tests and/or doctests in docstrings
  • Add docstrings and API docs for any new/modified user-facing classes and functions
  • New/modified features documented in docs/tutorial.rst
  • Changes documented in docs/release.rst
  • GitHub Actions have all passed
  • Test coverage is 100% (Codecov passes)

Copy link
Contributor

@d-v-b d-v-b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like a nice improvement!

@jhamman jhamman merged commit b8f6cb9 into zarr-developers:v3 Oct 10, 2024
@jhamman jhamman deleted the fix/thread-cleanup branch October 10, 2024 20:09
@jhamman jhamman added the V3 label Oct 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants