I think natively anything that supports .to_numpy() & agree it might be worth adding xarray to the tests

Broadly this intersects w/ my dissertation work & the NASA RSE work, but basically as far as I can tell the idea is if we can revamp the API enough to decouple the data bits from the artist/drawing bits, then external packages can write adapters against a data interface API. This allows external libraries to define what support means on their terms.

Is there not a NumPy standard compatibility NEP we can say we follow?

I think it looks like a lot of the relevant stuff is open neps?

• https://numpy.org/neps/nep-0022-ndarray-duck-typing-overview.html
• https://numpy.org/neps/nep-0030-duck-array-protocol.html
• https://numpy.org/neps/nep-0047-array-api-standard.html

For completeness, we should add h5py datasets to the above list.

Their way of converting to numpy is dataset[()] (and deprecated: dataset.value), none of which we currently use AFAIK. Still, h5py datasets work because their API is enough numpy-like 😲 .

with h5py dataset slicing we have a bunch of tests which are "do h5py and numpy slice exactly the same" so it is very very close to numpy.

Good to know. OTOH I don’t think we’re slicing in Matplotlib, so I assume we‘re relying on other numpy-like aspects.

polars

Maybe pyarrow (instead of polars specifically), then it would make matplotlib easier to use for anyone in the Arrow ecosystem, even from other languages that are doing interop with python

I didn't really get the relation between pyarrow and polars (just did a quick search), but it seems like both support to_numpy. With #22560 this is more consistently used and should improve things.

Can make sense to add a test for both though and I added it to the list (as well as h5py).

I guess I view this as the other way around - we are not guaranteeing compatibility with anything except numpy. However, as a courtesy, we will take results of your to_numpy and try and do the right thing with it. If you don't provide to_numpy or their to_numpy is broken, that is the other library's bug. Pandas falls into a grey area where we have tried to bend over backwards for them because it is so well-used, but that has entailed a moderately large maintenance burden. I don't think we want to add to it by testing the to_numpy implementations of more libraries?

the relation between pyarrow and polars

Good question - apache arrow is a memory model, so it just tells you what the data should look like in memory but it doesn't provide any functions to manipulate the data. Polars uses apache arrow as its memory model and provides methods to transform data.

One of the motivations for using Arrow is passing data around in a zero-copy way between libraries or even between languages. For example calling polars_df.to_arrow() will not create a copy of the data, instead it will return an object of type pyarrow.Table that points to the same data in memory.

Hope this helps and thank you for considering to support these libs.

The conclusion we came to was to add tests for "all" libraries, but only install them in the weekly event testing the newest version of numpy etc. (And not claim compatibility beyond that "if it has a to_numpy you are quite likely to be able to plot something, if not run the right function yourself".)

In this way, we can get a heads up in case something breaks.

We probably should put some floor on usage (polar has like 4k downloads from pypi a day, pandas has 2-3M, xarray hase 20-40k, and we have 500k-1M) before we worry about testing it.

Compatibility in the ecosystem has evolved. IMHO we should try to accept everything that follows the array API standard.