-
Notifications
You must be signed in to change notification settings - Fork 17
Cast shape, maxshape and total_size to (double) #113
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Testingcreated a file in h5py like this: import h5py
import numpy as np
data = np.zeros((1, 5489389, 395), dtype="float32")
with h5py.File("issue_111.h5", "w") as output:
output.create_dataset("data", data=data, chunks=(1, 1000, 395), compression=5)Then in nodejs: const h5wasm = await import("h5wasm/node");
await h5wasm.ready;
const f = new h5wasm.File("issue_111.h5", "r");
// File {
// path: '/',
// file_id: 72057594037927936n,
// type: 'Group',
// filename: 'issue_111.h5',
// mode: 'r'
// }
f.get("data").metadata;
// {
// signed: false,
// type: 1,
// vlen: false,
// littleEndian: true,
// size: 4,
// total_size: 2168308655,
// shape: [ 1, 5489389, 395 ],
// maxshape: [ 1, 5489389, 395 ],
// chunks: [ 1, 1000, 395 ]
// } |
axelboc
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Neat! That memory allocation error is most likely going to be quite useful.
|
|
||
| function check_malloc(nbytes: bigint | number) { | ||
| const max_memory = Module.MAXIMUM_MEMORY; | ||
| if (nbytes > max_memory) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would convert to safe_nbytes before checking, because > will throw if the two operands aren't both number. Though is this ever called with a bigint?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Comparison operators do an automatic conversion (unlike arithmetic operators), but you're right that it's probably only called with number type
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah yes, sorry!
This PR addresses #111 in a direct way:
In the C library for h5wasm, the values for
shape,maxshapeandtotal_sizeare read from the HDF5 library as typehsize_t, which is an alias foruint64_t(or arrays of this type, forshapeandmaxshape). This PR casts these values to(double)before returning them to the Javascript code in themetadatastructure.The C
(double)precision exactly maps onto the JavascriptNumbertype, which is what we want to use.The PR also introduces a new function
check_malloc(nbytes: number | bigint): number;that returns a pointer after checking: