Tags: yaak-ai/rbyte
Tags
chore: bump (#95) - pola-rs/polars#25567 fixed in `polars==1.36.0` - `unnest_all` no longer necessary (pola-rs/polars#12353 (comment))
feat(rerun logger): columns <-> indexes decoupling (#90) rerun logger: - decouple component columns from time columns via `indices` (a la sparse tensors) allowing for timestep-wise sparse column logging (useful for `rmind` prediction viz) - some schema refactoring - blueprint specification is back
feat: (mostly) TensorDict dataset state + torchdata dataloader (#87) - update dataset state from a pair of `pl.DataFrame`s to a combo of - `data: TensorDict` for most sample keys (see [blog](https://ppwwyyxx.com/blog/2022/Demystify-RAM-Usage-in-Multiprocess-DataLoader/#Serialize-to-a-torch-Tensor)) - `meta: pl.DataFrame` for non tensor friendly dtypes - `streams: dict` for storing metadata about training-time readers - add dataset saving/loading - add a `torchdata` node-based dataloader (https://meta-pytorch.org/data/beta/migrate_to_nodes_from_utils.html) for thread workers - minor config updates - tests refactoring
oomph (#84) A smorgasbord of changes prompted by memory-related experiments: - sample aggregation from inputs is now part of pipeline - support `return_results=False` (see [`pipefunc.map`](https://pipefunc.readthedocs.io/en/latest/reference/pipefunc.map/)) - replace `pipefunc.helpers.collect_kwargs` with `makefun.wrap_signature` to make functions taking variadic kwargs pipefunc-friendly and avoid an extra kwarg aggregation step - replace duckdb UDFs with native spatial functions - update rerun logger - bump deps
PreviousNext