Thanks to visit codestin.com
Credit goes to github.com

Skip to content
forked from zarrs/zarrs

A rust library for the Zarr storage format for multidimensional arrays and metadata

License

Notifications You must be signed in to change notification settings

dustinlagoy/zarrs

 
 

Repository files navigation

zarrs

Latest Version zarrs documentation msrv downloads build codecov

zarrs is a Rust library for the Zarr storage format for multidimensional arrays and metadata. It supports:

A changelog can be found here. Correctness issues with past versions are detailed here.

Developed at the Department of Materials Physics, Australian National University, Canberra, Australia.

Getting Started

  • Review the implementation status.
  • View the examples.
  • Read the documentation. array::Array, storage, and metadata are good places to start.
  • Check out zarrs_tools for various tools built upon this crate. Includes:
    • A reencoder that can change codecs, chunk shape, convert Zarr V2 to V3, etc.
    • Create an OME-Zarr hierarchy from a Zarr array.
    • Transform arrays: crop, rescale, downsample, gradient magnitude, gaussian, noise filtering, etc.
    • Benchmarking tools and performance benchmarks of zarrs.

Implementation Status

Zarr Enhancement Proposal Status Zarrs
ZEP0001: Zarr specification version 3 Accepted Full support
ZEP0002: Sharding codec Accepted Full support
Draft ZEP0003: Variable chunking zarr-developers #52 Full support
Draft ZEP0007: Strings zarr-developers/zeps #47 Prototype

Example

use zarrs::array::{ArrayBuilder, DataType, FillValue, ZARR_NAN_F32};
use zarrs::array::codec::GzipCodec; // requires gzip feature
use zarrs::array_subset::ArraySubset;
use zarrs::storage::{ReadableWritableListableStorage, store::FilesystemStore};

// Create a filesystem store
let store_path: PathBuf = "/path/to/store".into();
let store: ReadableWritableListableStorage =
    Arc::new(FilesystemStore::new(&store_path)?);

// Create a new V3 array using the array builder
let array = ArrayBuilder::new(
    vec![3, 4], // array shape
    DataType::Float32,
    vec![2, 2].try_into()?, // regular chunk shape (non-zero elements)
    FillValue::from(ZARR_NAN_F32),
)
.bytes_to_bytes_codecs(vec![
    Box::new(GzipCodec::new(5)?),
])
.dimension_names(["y", "x"].into())
.attributes(serde_json::json!({"Zarr V3": "is great"}).as_object().unwrap().clone())
.build(store.clone(), "/group/array")?; // /path/to/store/group/array

// Store the array metadata
array.store_metadata()?;
println!("{}", serde_json::to_string_pretty(array.metadata())?);
// {
//     "zarr_format": 3,
//     "node_type": "array",
//     ...
// }

// Perform some operations on the chunks
array.store_chunk_elements::<f32>(
    &[0, 1], // chunk index
    &[0.2, 0.3, 1.2, 1.3]
)?;
array.store_array_subset_ndarray::<f32, _>(
    &[1, 1], // array index
    ndarray::array![[-1.1, -1.2], [-2.1, -2.2]]
)?;
array.erase_chunk(&[1, 1])?;

// Retrieve all array elements as an ndarray
let array_subset_all = ArraySubset::new_with_shape(array.shape().to_vec());
let array_ndarray = array.retrieve_array_subset_ndarray::<f32>(&array_subset_all)?;
println!("{array_ndarray:4}");
// [[ NaN,  NaN,  0.2,  0.3],
//  [ NaN, -1.1, -1.2,  1.3],
//  [ NaN, -2.1,  NaN,  NaN]]

zarrs Ecosystem

  • zarrs_tools: Various tools for creating and manipulating Zarr V3 data.
  • zarrs_ffi: A subset of zarrs exposed as a C API.

Licence

zarrs is licensed under either of

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

About

A rust library for the Zarr storage format for multidimensional arrays and metadata

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Rust 99.8%
  • Python 0.2%