This seems to be working, with the exception of GraphFrame.to_hdf in Python 3.9. I'm not sure why that specific CI run is failing.

I've figured out the issue here.

The issue starts with PyTables, the library Pandas uses under the hood for HDF support (and which we indirectly use for the GraphFrame.to_hdf and GraphFrame.from_hdf methods). PyTables did not add NumPy 2.0 support until version 3.10, which coincidentally only supports Python 3.10 or higher. So, when running with Python 3.9 (where the CI error occurs), we are forced to run with a version of PyTables that does not support NumPy 2.0, and, as a result, we end up with an error.

I think this spawns a larger discussion about whether we want to continue supporting GraphFrame.to_hdf and GraphFrame.from_hdf or not. Are we even using it? Especially with Thicket?

After discussing with @michaelmckinsey1, we decided to do the following:

1. Officially deprecate the HDF reader/writer (since no one uses it and it is not supported in Thicket)
2. Catch errors from the reader/writer and, if needed, add context to warn that the issue is likely due to version mismatches between NumPy, Pandas, and/or PyTables