Thanks! Not sure if we can add an easy test, so I am happy to just put this is in as is. I wonder if we shouldn't just replace this whole code with memset(f, 0, sizeof(*f));? But happy to put it in as is, its not like PyArray_Funcs struct can change...

@octachrome on a more general note, just asking anyone with custom dtypes: Can you share what you are doing with your custom dtypes so I could see if there is anything trickier/special that goes beyond the typical rational example?

True, a memset would avoid the need to remember to update PyArray_InitArrFuncs when adding new fields to the structure.

I can't share our C code because it is a commercial product, but there is some documentation about how we use NumPy here: https://www.fico.com/fico-xpress-optimization/docs/dms2020-03/solver/optimizer/python/HTML/chNumpy.html

Our product is an optimization framework in which users define large models consisting of variables and constraints on those variables. The product finds optimal assignments of the variables which satisfy the constraints. It is convenient for our users to use NumPy when defining their models, for example they can define large numbers of constraints in a single inequality expression, where both the left- and right-hand sides are vectors: a <= b.

There are several examples of how our users might use NumPy in this list: https://www.fico.com/fico-xpress-optimization/docs/dms2020-03/examples/python/GUID-E77AC35C-9488-3F0A-98B4-7F5FD81AFF1D.html

Thanks, that is valuable information. I can't tell from the help whether or not there are potential issues going forward, I doubt it, but I am not sure. There are very few examples of user-dtypes (e.g. rational, quaternion) and only with small breaks such as here I hear of some more but can't closely look at.

As a heads up: 1.20 will have more changes. Hopefully, you will just not notice it, but with large enough changes we have to expect potential issues (and since I don't know the details of your dtype I cannot be quite sure that serious ones are impossible). So if you can, please keep an eye out for the next release especially. If you have the man-power going as far as testing against NumPy master occasionally, which is available at: https://anaconda.org/scipy-wheels-nightly/numpy could be extremely helpful both for us and your users.

OK, lets put this in, I might clean it up later (but this function will hopefully never change, until it gets deleted in some years...)

@octachrome I noticed that I can try around a bit with the community edition. There is one thing that slightly worries me, but right now I still think its probably harmless.

OK, one more issue already. You use a structured dtype with a single object field. I guess that is not an unreasonable, hack to make a typed object dtype. Master currently rejects that, so I have to allow that specifically. I guess it should all be fine, it seems like a very useful current hack (I will be surprised if there are not weird corner cases, but that is another issue).

Thanks for taking the time to check our extension, that's greatly appreciated. I will pass on your comments to the lead dev who knows a lot more about it than I do.

Thanks, current master has problems with xpress. Part of it is just that a new check, not anticipating the way you use an object field. I can undo that. I admit one of the reasons I put the error check was to find out whether this type of thing exists in the wild.

There is another issue that I am changing the NumPy dtypes a lot (hopefully gentle enough), but one step is that each dtype now has its own type(dtype). I can do that on the fly during dtype registration. It is hacky but works, but there I currently check that the registered dtype has Py_TYPE(descr) == &PyArrayDescr_Type and this is failing for xpress.

That should be easy to fix up, but it probably means that users will have to update xpress and the original error is hidden behind a cryptic license error.