Slightly apologetic comment - AOR/pl has recently removed some symbols, and likely AOR/math will at some point soon as well, so the way function names are chosen is not future-proof (and won't work at all if you want any routines from pl/). How are you building this? I don't see any config.mk file in this patch, so interested in how AOR is being configured?

I'm just building the bits I need as a submodule in numpy rather than taking the whole library in, that would require additional effort that doesn't appear to be justified?

Slightly apologetic comment - AOR/pl has recently removed some symbols, and likely AOR/math will at some point soon as well, so the way function names are chosen is not future-proof (and won't work at all if you want any routines from pl/). How are you building this? I don't see any config.mk file in this patch, so interested in how AOR is being configured?

I'm just building the bits I need as a submodule in numpy rather than taking the whole library in, that would require additional effort that doesn't appear to be justified?

So the C files are just compiled independently of the AOR Makefiles? Makes sense, but since AOR is designed to be build with a config file, you may end up missing certain features (or even routines) that are enabled based on settings in the config file, for instance without -DWANT_SIMD_EXCEPT=1 you will not have fp exceptions triggered correctly. My understanding is that numpy requires this, so I'm surprised not to see it mentioned here

Slightly apologetic comment - AOR/pl has recently removed some symbols, and likely AOR/math will at some point soon as well, so the way function names are chosen is not future-proof (and won't work at all if you want any routines from pl/). How are you building this? I don't see any config.mk file in this patch, so interested in how AOR is being configured?

I'm just building the bits I need as a submodule in numpy rather than taking the whole library in, that would require additional effort that doesn't appear to be justified?

So the C files are just compiled independently of the AOR Makefiles? Makes sense, but since AOR is designed to be build with a config file, you may end up missing certain features (or even routines) that are enabled based on settings in the config file, for instance without -DWANT_SIMD_EXCEPT=1 you will not have fp exceptions triggered correctly. My understanding is that numpy requires this, so I'm surprised not to see it mentioned here

Likewise, but if the tests are passing I would consider that sufficient for now, I'm keen not to add too many additional flags if I can avoid it especially given they're not namespaced to Optimized Routines so could cause other software to compile weirdly.

Slightly apologetic comment - AOR/pl has recently removed some symbols, and likely AOR/math will at some point soon as well, so the way function names are chosen is not future-proof (and won't work at all if you want any routines from pl/). How are you building this? I don't see any config.mk file in this patch, so interested in how AOR is being configured?

I'm just building the bits I need as a submodule in numpy rather than taking the whole library in, that would require additional effort that doesn't appear to be justified?

So the C files are just compiled independently of the AOR Makefiles? Makes sense, but since AOR is designed to be build with a config file, you may end up missing certain features (or even routines) that are enabled based on settings in the config file, for instance without -DWANT_SIMD_EXCEPT=1 you will not have fp exceptions triggered correctly. My understanding is that numpy requires this, so I'm surprised not to see it mentioned here

Likewise, but if the tests are passing I would consider that sufficient for now, I'm keen not to add too many additional flags if I can avoid it especially given they're not namespaced to Optimized Routines so could cause other software to compile weirdly.

I understand what you're saying, but I think this could become very difficult to maintain for a couple of reasons. One is that AOR routines are not always completely self-contained in one source file - there may be helper routines or coefficient arrays in a different TU (not the case for sin/cos, so it was fine for this patch). These get moved around from time to time - we don't make any promises about what files will exist, and indeed we are in the process of shuffling some things around. The VFABI-mangled symbols don't come from v_<func>.c at all, they come from vn_<func>.c but again, we are in the process of changing this. The only way to reliably get the symbols you need is by building the whole library.

As well, there are certain flags that at some point you will need in your compiler invocation, for instance you can't build SVE routines without explicitly enabling them, either in the config file or with -DWANT_SVE_MATH=1.

Slightly apologetic comment - AOR/pl has recently removed some symbols, and likely AOR/math will at some point soon as well, so the way function names are chosen is not future-proof (and won't work at all if you want any routines from pl/). How are you building this? I don't see any config.mk file in this patch, so interested in how AOR is being configured?

I'm just building the bits I need as a submodule in numpy rather than taking the whole library in, that would require additional effort that doesn't appear to be justified?

So the C files are just compiled independently of the AOR Makefiles? Makes sense, but since AOR is designed to be build with a config file, you may end up missing certain features (or even routines) that are enabled based on settings in the config file, for instance without -DWANT_SIMD_EXCEPT=1 you will not have fp exceptions triggered correctly. My understanding is that numpy requires this, so I'm surprised not to see it mentioned here

Likewise, but if the tests are passing I would consider that sufficient for now, I'm keen not to add too many additional flags if I can avoid it especially given they're not namespaced to Optimized Routines so could cause other software to compile weirdly.

I understand what you're saying, but I think this could become very difficult to maintain for a couple of reasons. One is that AOR routines are not always completely self-contained in one source file - there may be helper routines or coefficient arrays in a different TU (not the case for sin/cos, so it was fine for this patch). These get moved around from time to time - we don't make any promises about what files will exist, and indeed we are in the process of shuffling some things around. The VFABI-mangled symbols don't come from v_<func>.c at all, they come from vn_<func>.c but again, we are in the process of changing this. The only way to reliably get the symbols you need is by building the whole library.

As well, there are certain flags that at some point you will need in your compiler invocation, for instance you can't build SVE routines without explicitly enabling them, either in the config file or with -DWANT_SVE_MATH=1.

I think that's ok as from the brief discussion with @mattip, the focus should be on moving to universal intrinsics in numpy. AOR is a good intermediate step for the performance boost but we can avoid long-term dependence on it if we can't minimally include it. In the intermediary state we can accept having the files as-is and a slightly painful upgrade path, that'll likely land on me 😸

Perusing the source repo, I came across this file. Is this the implementation of cos? It has this comment:

/* worst-case error is 3.5 ulp.
   abs error: 0x1.be222a58p-53 in [-pi/2, pi/2].  */

I would think that would exceed our tests, so I must be looking at the wrong code.

Perusing the source repo, I came across this file. Is this the implementation of cos? It has this comment:

/* worst-case error is 3.5 ulp.
   abs error: 0x1.be222a58p-53 in [-pi/2, pi/2].  */

I would think that would exceed our tests, so I must be looking at the wrong code.

Interesting as the original SVML implementation is 4ULP (#19478)

We discussed this when adding the validation tests, starting around here in the PR. Here is what cpu features are used in CI. Are the new code paths triggered there?

I've updated both setup.py and meson.build, but I'm unsure which gets triggered when

We are transitioning from setup.py to meson. The aarch64 CI run is still using setup.py.

We discussed this when adding the validation tests, starting around here in the PR. Here is what cpu features are used in CI. Are the new code paths triggered there?

They should be, as I had to add the git submodule update --init line into Cirrus for the cibuildwheel build to work 😸

Adding the submodule means the routines were compiled in. But how do we know they were used in the runtime? The only information we get is that the CPU detection kicked in, which should then choose the correct inner loops. Which features are required for these routines?

Adding the submodule means the routines were compiled in. But how do we know they were used in the runtime? The only information we get is that the CPU detection kicked in, which should then choose the correct inner loops. Which features are required for these routines?

They all trigger from ASIMD, which should be standard on AArch64 machines.

Indeed, ASIMD is in the set of required baseline features. The cirrusCI machines report:

NumPy CPU features:  NEON NEON_FP16 NEON_VFPV4 ASIMD ASIMDHP* ASIMDDP* ASIMDFHM?

Indeed, ASIMD is in the set of required baseline features. The cirrusCI machines report:

NumPy CPU features:  NEON NEON_FP16 NEON_VFPV4 ASIMD ASIMDHP* ASIMDDP* ASIMDFHM?

Hmm, so I'm guessing this is all fine in terms of the tests or are there further concerns from your point of view?

We discussed this when adding the validation tests, starting around here in the PR. Here is what cpu features are used in CI. Are the new code paths triggered there?

Regarding accuracy our design principle is to have a threshold of 3.5ULPs over the entire range of libm routines, like SVML has a 4ULPs threshold. Unfortunately we cannot provide better accuracy than a given reference on a per routine basis, hence why all libraries use a single threshold (glibc's libmvec has 4ULPs too).
Are you using the same error threshold for testing SVML and the regular scalar implementation?

Am I right thinking your test thresholds are based on your evaluation of ULP errors, or are these ULP errors provided by SVML? Evaluating the maximum error for double precision routines is notoriously hard and fairly inaccurate by sampling the whole domain or even a reduced interval randomly (even with billions of points).

This is also a reason why AOR routines might appear to pass the tests even with maximum errors larger than the current threshold. A random set of test inputs will likely not trigger the worst cases even with millions of inputs.

Let me know if this discussion is better suited for this PR.

Hi @mattip !
I have a few naive questions about this PR.
What is going to happen to these routines once ported to numpy intrinsics? How are they going to be used?
Is the point to be able to use them on other architectures than Arm? Or is there more to it?
Are you still going to keep SVML implementations? And use them on AVX512 enabled machines then? Are you going to pick the fastest implementation?

How are they going to be maintained? By who? Since AOR implementations are continuously improving, they might diverge from the last drop in quite significantly. So each update would basically consist in doing a full port again from AOR to numpy intrinsics. The situation would become even much worse if they were also modified from within numpy.

An option to keep maintenance low would be to provide an AOR to Numpy interface. We just got rid of this feature but might be able to re-introduce in a different way, if necessary. The code would still have to be configured (pre-processed) so it has the features Numpy expects.

Numpy intrinsics seems like an interesting concept, just curious about the implications when trying to "upstream" external work into Numpy using this language.

I am not sure why the question is directed at me, but I will try to supply some answers. In general, NumPy accepts contributions from many people, and as time goes by if the code is not maintainable or has outlived its usefulness, we remove it. This is especially true of routines that, like the inner loops of ufuncs, is not exposed to users. Thus we have added and dropped support for different compilers and hardware over the years: notably support for the Apple Accelerate library was removed and then restored when it once again passed the acceptance tests.

The ufunc dispatch framework provides a way to choose the "most appropriate" inner loop, based on runtime detection of supported CPU features, and some heuristics (strides, contiguous data). This PR changes none of that, but the mechanism might need some tweaks if someone takes the time to analyze appropriate heuristics for various ARM processors.

Of course we would prefer all these routines be rewritten in universal intrinsics. But we understand that expertise in them is hard to find. This is true for both SVML and AOR: once we have generic routines we can remove the architecture-specific ones.

Integrating a new version of a vendored dependency is tricky. If the AOR team feels the library is not stable and the interfaces will change drastically, perhaps we should hold off with integration until it stabilizes. That question should be answered by the AOR team.

Personally, while I welcome the contributions, I am not sure the 2x performance increase justifies the additional complexity introduced in this PR. Are there routines or hardware where AOR gives a larger boost?

Closing this in favour of #23399