Couple of style nits, otherwise LGTM.

I'd prefer this to use runtime selection, compile time only options are not very useful for the majority of users.

also interesting that avx finally does something for square root on newer cpus, it used to do nothing as it was just effectively two sequential sse2 operations up to including haswell cpus.
I'll have a look at verifying these benchmarks on some newer cpus at work tomorrow.

hm I didn't have much time but a quick test on an Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz showed no difference in performance for sse2, avx2 and avx512 despite the code definitely running the right code paths.

Also no difference in runtime or cycle count between sse and avx2 on a AMD Ryzen 7 1700X

Can you provide your benchmark please?

Perhaps these different experiences depend on the compiler/hardware. @juliantaylor I assume you are using (recent?) gcc. @r-devulap Are you using the Intel compiler? What hardware?

I am using gcc 7.3.0 on Intel i9-7900X. I built NumPy using -march=native. I am basically measuring TSC cycles the function DOUBLE_sqrt/FLOAT_sqrt takes to execute (by using the cpuid, rdtscand rdtscp, cpuid instructions before and after respectively). I did disable turbo and set intel_pstate governor to "performance" mode (to reduce variance across multiple runs). @juliantaylor How are you building NumPy?

Also a gcc 7.3. The difference must be hardware, I checked the assembly that was produced and it looks fine. I also used rdtscp to check the cycle counts.
The Intel(R) Xeon(R) Gold 6130 is a server cpu not a high end desktop variant but it is of the same generation and has the same amount of fma units. I kind of doubt that something as fundamental as a sqrt would differ between these types of cpus.
But I can be wrong. Though this type of major change must be documented somewhere.

What confuses me on your numbers is that there is almost no impact with increasing array size. sqrt is relatively slow so the memory bandwidth is not too problematic compared to simple addition, but I would still have expected an effect when you go beyond the L3 cache.
But maybe RAM just got faster since I was deep into this stuff :/

While I would like to understand why I cannot reproduce any speedups with avx2/avx512 and do not like the way this is implemented, it does not make things worse and I might just be using wrong hardware (though I have tested alot of different cpus over time ...).
So from my side this can be merged at the more active maintainers discretion.

(and again I want to plug my runtime variant of this in #11113 there are things to finish and improve in that PR but base concept of runtime selected ufuncs works)

I think I'll leave this open for a bit, it will be easy enough to backport if wanted. The main thing seems to be the runtime vs compile time variants, although it would be nice to track down the timing differences. @r-devulap Any chance you could try a lower end cpu in the same series?

My bad, @juliantaylor is right, I do not see any performance benefits either. I was using -O2 for compiling my micro-benchmarks and after debugging I realized gcc does not inline small functions consistently. If I inline them or use -O3 flag, then SSE2/AVX2/AVX512 all perform similarly. Vtune analysis shows that the function is memory bound and sqrt vectorization does not seem to help.

I'll close this then. @r-devulap Thanks for the work and tracking down the cause of the timing discrepancy. @juliantaylor Thanks for the review.