Zarr: extract time vector once and for all! #2828
Conversation
| time_vector = self._root.get(f"times_seg{segment_index}", None) | ||
| if time_vector is not None: | ||
| time_kwargs["time_vector"] = time_vector | ||
| time_kwargs["time_vector"] = time_vector[:] |
There was a problem hiding this comment.
Maybe I'm misunderstanding the compression-decompression, but how does switching to a view change this?
There was a problem hiding this comment.
Without the [:] the time_vector is set to a zarr.Array object.
This object performs a decompression/retrieval from file upon slicing, so if I call get_times() 100 times, the data will be accessed from the file (and optionally decompressed) 100 times.
This simple change slices the array, so that time_kwargs["time_vector"] is now a numpy array and not a zarr object anymore.
Hope this is clear :)
|
As discuss with Alessio this morning. |
If the zarr folder has timestamps, they were re-extracted and uncompressed every time the
get_times()was called.This PR loads the zarr array as a numpy array at instantiation to speed things up