Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@bmaranville
Copy link
Member

This PR addresses #111 in a direct way:

In the C library for h5wasm, the values for shape, maxshape and total_size are read from the HDF5 library as type hsize_t, which is an alias for uint64_t (or arrays of this type, for shape and maxshape). This PR casts these values to (double) before returning them to the Javascript code in the metadata structure.

The C (double) precision exactly maps onto the Javascript Number type, which is what we want to use.

The PR also introduces a new function check_malloc(nbytes: number | bigint): number; that returns a pointer after checking:

  • that it is not requesting more than the maximum memory available in the heap (often 2GB, depends on compiler settings)
  • that the allocation was successful (malloc returns 0 if it fails)

@bmaranville
Copy link
Member Author

Testing

created a file in h5py like this:

import h5py
import numpy as np

data = np.zeros((1, 5489389, 395), dtype="float32")
with h5py.File("issue_111.h5", "w") as output:
    output.create_dataset("data", data=data, chunks=(1, 1000, 395), compression=5)

Then in nodejs:

const h5wasm = await import("h5wasm/node");
await h5wasm.ready;
const f = new h5wasm.File("issue_111.h5", "r");
// File {
//  path: '/',
//  file_id: 72057594037927936n,
//  type: 'Group',
//  filename: 'issue_111.h5',
//  mode: 'r'
// }
f.get("data").metadata;
// {
//  signed: false,
//  type: 1,
//  vlen: false,
//  littleEndian: true,
//  size: 4,
//  total_size: 2168308655,
//  shape: [ 1, 5489389, 395 ],
//  maxshape: [ 1, 5489389, 395 ],
//  chunks: [ 1, 1000, 395 ]
// }

@bmaranville bmaranville requested a review from axelboc December 16, 2025 20:41
Copy link
Collaborator

@axelboc axelboc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat! That memory allocation error is most likely going to be quite useful.


function check_malloc(nbytes: bigint | number) {
const max_memory = Module.MAXIMUM_MEMORY;
if (nbytes > max_memory) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would convert to safe_nbytes before checking, because > will throw if the two operands aren't both number. Though is this ever called with a bigint?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comparison operators do an automatic conversion (unlike arithmetic operators), but you're right that it's probably only called with number type

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, sorry!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants