Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Neon detection on M1 needs fixing for 1.23.0 release #21747

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
charris opened this issue Jun 14, 2022 · 6 comments · Fixed by #21749
Closed

Neon detection on M1 needs fixing for 1.23.0 release #21747

charris opened this issue Jun 14, 2022 · 6 comments · Fixed by #21749
Labels
06 - Regression
Milestone
1.23.0 release

Comments

@charris
Copy link
Member

charris commented Jun 14, 2022

See #21705 (comment)
@charris charris added this to the 1.23.0 release milestone Jun 14, 2022
@dhomeier
Copy link
Contributor

dhomeier commented Jun 14, 2022 •
edited
Loading

I noticed one failure compiling cpu_asimd.c with 1.23.0rc3:

INFO: compile options: '-Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/opt/include/python3.10 -Ibuild/src.macosx-12.4-arm64-3.10/numpy/core/src/common -Ibuild/src.macosx-12.4-arm64-3.10/numpy/core/src/npymath -c'
extra options: '-Werror'
INFO: CCompilerOpt.feature_test[1547] : testing feature 'ASIMD' with flags ()
INFO: C compiler: gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wno-nullability-completeness -Wno-expansion-to-defined -mtune=native -I/opt/include

INFO: compile options: '-Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/opt/include/python3.10 -Ibuild/src.macosx-12.4-arm64-3.10/numpy/core/src/common -Ibuild/src.macosx-12.4-arm64-3.10/numpy/core/src/npymath -c'
extra options: '-Werror'
WARN: CCompilerOpt.dist_test[630] : CCompilerOpt._dist_test_spawn[764] : Command (gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wno-nullability-completeness -Wno-expansion-to-defined -mtune=native -I/opt/include -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/opt/include/python3.10 -Ibuild/src.macosx-12.4-arm64-3.10/numpy/core/src/common -Ibuild/src.macosx-12.4-arm64-3.10/numpy/core/src/npymath -c /private/scratch.noindex/fink.build/numpy-py310-1.23.0rc3-1/numpy-1.23.0rc3/numpy/distutils/checks/cpu_asimd.c -o /private/tmp/tmpngjq3hpq/private/scratch.noindex/fink.build/numpy-py310-1.23.0rc3-1/numpy-1.23.0rc3/numpy/distutils/checks/cpu_asimd.o -MMD -MF /private/tmp/tmpngjq3hpq/private/scratch.noindex/fink.build/numpy-py310-1.23.0rc3-1/numpy-1.23.0rc3/numpy/distutils/checks/cpu_asimd.o.d -Werror) failed with exit status 1 output -> 
/private/scratch.noindex/fink.build/numpy-py310-1.23.0rc3-1/numpy-1.23.0rc3/numpy/distutils/checks/cpu_asimd.c:17:17: error: incompatible pointer types initializing 'double *' with an expression of type 'float *' [-Werror,-Wincompatible-pointer-types]
        double *src2 = (float*)argv[argc-1];
                ^      ~~~~~~~~~~~~~~~~~~~~
1 error generated.

WARN: CCompilerOpt.feature_test[1563] : testing failed
INFO: CCompilerOpt.feature_test[1547] : testing feature 'NEON' with flags ()
INFO: C compiler: gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wno-nullability-completeness -Wno-expansion-to-defined -mtune=native -I/opt/include

INFO: compile options: '-Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/opt/include/python3.10 -Ibuild/src.macosx-12.4-arm64-3.10/numpy/core/src/common -Ibuild/src.macosx-12.4-arm64-3.10/numpy/core/src/npymath -c'
extra options: '-Werror'
INFO: CCompilerOpt.__init__[1791] : check requested dispatch-able features
INFO: CCompilerOpt.cc_test_flags[1073] : testing flags (-march=armv8.2-a+fp16)
INFO: C compiler: gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wno-nullability-completeness -Wno-expansion-to-defined -mtune=native -I/opt/include

INFO: compile options: '-Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/opt/include/python3.10 -Ibuild/src.macosx-12.4-arm64-3.10/numpy/core/src/common -Ibuild/src.macosx-12.4-arm64-3.10/numpy/core/src/npymath -c'
extra options: '-march=armv8.2-a+fp16'
INFO: CCompilerOpt.feature_test[1547] : testing feature 'ASIMDHP' with flags (-march=armv8.2-a+fp16)
INFO: C compiler: gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wno-nullability-completeness -Wno-expansion-to-defined -mtune=native -I/opt/include

INFO: compile options: '-Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/opt/include/python3.10 -Ibuild/src.macosx-12.4-arm64-3.10/numpy/core/src/common -Ibuild/src.macosx-12.4-arm64-3.10/numpy/core/src/npymath -c'
extra options: '-march=armv8.2-a+fp16 -Werror'
INFO: CCompilerOpt.__init__[1807] : initialize targets groups
INFO: CCompilerOpt.__init__[1809] : parse target group simd_test
INFO: CCompilerOpt._parse_target_tokens[2020] : skip targets (AVX512_SKX SSE2 XOP VSX3 ASIMD VSX2 SSE42 FMA4 VXE VX VSX VSX4 (AVX2 FMA3) AVX512F VXE2 NEON) not part of baseline or dispatch-able features

which did not show up prior to #21705

INFO: compile options: '-Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/opt/include/python3.10 -Ibuild/src.macosx-12.4-arm64-3.10/numpy/core/src/common -Ibuild/src.macosx-12.4-arm64-3.10/numpy/core/src/npymath -c'
extra options: '-Werror'
INFO: CCompilerOpt.feature_test[1522] : testing feature 'ASIMD' with flags ()
INFO: C compiler: gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wno-nullability-completeness -Wno-expansion-to-defined -mtune=native -I/opt/include

INFO: compile options: '-Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/opt/include/python3.10 -Ibuild/src.macosx-12.4-arm64-3.10/numpy/core/src/common -Ibuild/src.macosx-12.4-arm64-3.10/numpy/core/src/npymath -c'
extra options: '-Werror'
INFO: CCompilerOpt.feature_test[1522] : testing feature 'NEON_VFPV4' with flags ()
INFO: C compiler: gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wno-nullability-completeness -Wno-expansion-to-defined -mtune=native -I/opt/include

[numerous other feature_tests]

INFO: compile options: '-Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/opt/include/python3.10 -Ibuild/src.macosx-12.4-arm64-3.10/numpy/core/src/common -Ibuild/src.macosx-12.4-arm64-3.10/numpy/core/src/npymath -c'
extra options: '-march=armv8.2-a+fp16+fp16fml -Werror'
INFO: CCompilerOpt.__init__[1778] : skip features (NEON_FP16 NEON ASIMD NEON_VFPV4) since its part of baseline
INFO: CCompilerOpt.__init__[1782] : initialize targets groups
INFO: CCompilerOpt.__init__[1784] : parse target group simd_test
INFO: CCompilerOpt._parse_target_tokens[1995] : skip targets (SSE2 AVX512F VSX4 VXE2 SSE42 AVX512_SKX FMA4 VSX3 (FMA3 AVX2) VSX2 VSX VXE XOP VX) not part of baseline or dispatch-able features
INFO: CCompilerOpt._parse_policy_not_keepbase[2107] : skip baseline features (ASIMD)

Perhaps it just requires a recast of src2?

@dhomeier
Copy link
Contributor

Perhaps it just requires a recast of src2?

FWIW changing the assignment in cpu_asimd.c to

        double *src2 = (double*)argv[argc-1];

restores the default to

CPU baseline  : 
  Requested   : 'min'
  Enabled     : NEON NEON_FP16 NEON_VFPV4 ASIMD
  Flags       : none
  Extra checks: none

CPU dispatch  : 
  Requested   : 'max -xop -fma4'
  Enabled     : ASIMDHP ASIMDDP ASIMDFHM
  Generated   : 
              : 
  ASIMDHP     : NEON NEON_FP16 NEON_VFPV4 ASIMD
  Flags       : -march=armv8.2-a+fp16
  Extra checks: none
  Detect      : ASIMD ASIMDHP

and requested to

CPU baseline  : 
  Requested   : 'NEON NEON_FP16 NEON_VFPV4 ASIMD ASIMDHP ASIMDDP'
  Enabled     : NEON NEON_FP16 NEON_VFPV4 ASIMD ASIMDHP ASIMDDP
  Flags       : -march=armv8.2-a+fp16+dotprod
  Extra checks: none

CPU dispatch  : 
  Requested   : 'max -xop -fma4'
  Enabled     : ASIMDFHM
  Generated   : none

@mattip
Copy link
Member

mattip commented Jun 14, 2022

@dhomeier could you open a PR?

@mattip
Copy link
Member

mattip commented Jun 14, 2022

We do not have a way to catch these errors in CI. Maybe we should have a qemu M1 job that just tests SIMD, like we have for armv7.

@seiko2plus seiko2plus mentioned this issue Jun 14, 2022
Merged
1 task
@seiko2plus
Copy link
Member

My bad, the current Travis CI are good enough to show up this bug but for somehow I checked armv7 and x86 builds log without paying attention to aarch64 build, I am so sorry for my neglect. Thank you @dhomeier for catching it.

@seiko2plus
Copy link
Member

@dhomeier, cast fixed by #21749, I doubly checked build logs and everything come back to normality. Thank you once again

@charris charris mentioned this issue Jun 14, 2022
Merged
1 task
seiko2plus added a commit to seiko2plus/numpy that referenced this issue Jun 15, 2022
@seiko2plus
CI: Guard compile-time CPU features tests
a3e6736 
  This guard protects against any sudden unexpected changes that may adversely
  affect the compile-time SIMD features detection which could leave the SIMD code
  inactivated. Until now we have faced two cases:

  1. Hardening the compile-time test files of Neon/ASIMD features without checking
     the sanity of the modification leading to disabling all optimizations on aarch64.
     see numpygh-21747

  2. A sudden compiler upgrades by CI side on s390x that causes conflicts with the
     installed assembler leading to disabling the whole VX/E features, which made us
     merge SIMD code without testing it. Later, it was discovered that this code
     disrupted the NumPy build.
     see numpygh-21750, numpygh-21748
@seiko2plus seiko2plus mentioned this issue Jun 15, 2022
Merged
seiko2plus added a commit to seiko2plus/numpy that referenced this issue Jun 15, 2022
@seiko2plus
CI: Guard compile-time CPU features tests
a8da927 
  This guard protects against any sudden unexpected changes that may adversely
  affect the compile-time SIMD features detection which could leave the SIMD code
  inactivated. Until now we have faced two cases:

  1. Hardening the compile-time test files of Neon/ASIMD features without checking
     the sanity of the modification leading to disabling all optimizations on aarch64.
     see numpygh-21747

  2. A sudden compiler upgrades by CI side on s390x that causes conflicts with the
     installed assembler leading to disabling the whole VX/E features, which made us
     merge SIMD code without testing it. Later, it was discovered that this code
     disrupted the NumPy build.
     see numpygh-21750, numpygh-21748
seiko2plus added a commit to seiko2plus/numpy that referenced this issue Jun 15, 2022
@seiko2plus
CI: Guard compile-time CPU features tests
91faeaa 
  This guard protects against any sudden unexpected changes that may adversely
  affect the compile-time SIMD features detection which could leave the SIMD code
  inactivated. Until now we have faced two cases:

  1. Hardening the compile-time test files of Neon/ASIMD features without checking
     the sanity of the modification leading to disabling all optimizations on aarch64.
     see numpygh-21747

  2. A sudden compiler upgrades by CI side on s390x that causes conflicts with the
     installed assembler leading to disabling the whole VX/E features, which made us
     merge SIMD code without testing it. Later, it was discovered that this code
     disrupted the NumPy build.
     see numpygh-21750, numpygh-21748
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
06 - Regression
Projects
None yet
Milestone
1.23.0 release
Development

Successfully merging a pull request may close this issue.

BUG, SIMD: Fix detecting NEON/ASIMD on aarch64 seiko2plus/numpy
4 participants
@charris @dhomeier @mattip @seiko2plus