Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit b46eef0

Browse files
charrischarris
authored
MAINT: Massive update of files from main for numpy 1.26 (#24308)
Rather than do the meson fixups bit by bit, The following files are simply checked out from the main branch, we can add stuff back if needed. The main missing bit is 32 bit wheels on Windows, we may need to just drop them. The numpy/f2py and numpy/random directories were copied wholesale. Note that the SIMD and 32 bit linux problems have also arrived and will need to be fixed along with the main branch. The test_mem_policy.py::test_new_policy test also fails, but no longer runs because it is so slow. - .circleci/config.yml - .cirrus.star - .github/workflows/build_test.yml - .github/workflows/emscripten.yml - .github/workflows/linux_meson.yml - .github/workflows/linux_musl.yml - .github/workflows/wheels.yml - .github/workflows/windows_meson.yml - .spin/cmds.py - azure-pipelines.yml - azure-steps-windows.yml - build_requirements.txt - meson_options.txt - numpy/_utils/__init__.py - numpy/_utils/_convertions.py - numpy/core/tests/test_mem_policy.py - numpy/core/tests/test_umath.py - numpy/f2py/capi_maps.py - numpy/f2py/cfuncs.py - numpy/f2py/tests/test_return_integer.py - numpy/f2py/tests/test_return_real.py - numpy/f2py/tests/util.py - numpy/meson.build - numpy/random/_examples/cython/setup.py - numpy/random/_generator.pyx - numpy/random/_mt19937.pyx - numpy/random/_pcg64.pyx - numpy/random/_philox.pyx - numpy/random/_sfc64.pyx - numpy/random/bit_generator.pyx - numpy/random/meson.build - numpy/random/mtrand.pyx - numpy/random/src/distributions/distributions.c - numpy/random/src/mt19937/randomkit.h - numpy/random/src/pcg64/pcg64.orig.h - numpy/random/tests/test_extending.py - numpy/random/tests/test_generator_mt19937.py - numpy/random/tests/test_generator_mt19937_regressions.py - numpy/random/tests/test_random.py - numpy/random/tests/test_randomstate.py - pyproject.toml - pyproject.toml.setuppy - test_requirements.txt - tools/ci/cirrus_macosx_arm64.yml - tools/ci/cirrus_wheels.yml - tools/openblas_support.py - tools/travis-before-install.sh - tools/travis-test.sh - tools/wheels/repair_windows.sh
1 parent f2b82bf commit b46eef0

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

49 files changed

+1101
-537
lines changed

‎.circleci/config.yml

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -61,10 +61,9 @@ jobs:
6161
name: build numpy
6262
command: |
6363
. venv/bin/activate
64-
pip install --progress-bar=off --upgrade pip 'setuptools<49.2.0'
6564
pip install --progress-bar=off -r test_requirements.txt
6665
pip install --progress-bar=off -r doc_requirements.txt
67-
pip install .
66+
pip install . --config-settings=setup-args="-Dallow-noblas=true"
6867
6968
- run:
7069
name: create release notes

‎.cirrus.star

Lines changed: 19 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -26,14 +26,26 @@ def main(ctx):
2626
SHA = env.get("CIRRUS_CHANGE_IN_REPO")
2727
url = "https://api.github.com/repos/numpy/numpy/git/commits/" + SHA
2828
dct = http.get(url).json()
29-
# if "[wheel build]" in dct["message"]:
30-
# return fs.read("ci/cirrus_wheels.yml")
3129

32-
if "[skip cirrus]" in dct["message"] or "[skip ci]" in dct["message"]:
30+
commit_msg = dct["message"]
31+
if "[skip cirrus]" in commit_msg or "[skip ci]" in commit_msg:
3332
return []
3433

35-
# add extra jobs to the cirrus run by += adding to config
36-
config = fs.read("tools/ci/cirrus_wheels.yml")
37-
config += fs.read("tools/ci/cirrus_macosx_arm64.yml")
34+
wheel = False
35+
labels = env.get("CIRRUS_PR_LABELS", "")
36+
pr_number = env.get("CIRRUS_PR", "-1")
37+
tag = env.get("CIRRUS_TAG", "")
3838

39-
return config
39+
if "[wheel build]" in commit_msg:
40+
wheel = True
41+
42+
# if int(pr_number) > 0 and ("14 - Release" in labels or "36 - Build" in labels):
43+
# wheel = True
44+
45+
if tag.startswith("v") and "dev0" not in tag:
46+
wheel = True
47+
48+
if wheel:
49+
return fs.read("tools/ci/cirrus_wheels.yml")
50+
51+
return fs.read("tools/ci/cirrus_macosx_arm64.yml")

‎.github/workflows/build_test.yml

Lines changed: 8 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ jobs:
6666
if: github.event_name != 'push'
6767
strategy:
6868
matrix:
69-
python-version: ["3.9", "3.10", "3.11", "pypy3.9-v7.3.11"]
69+
python-version: ["3.9", "3.10", "3.11", "pypy3.9-v7.3.12"]
7070
env:
7171
EXPECT_CPU_FEATURES: "SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 AVX512F AVX512CD AVX512_KNL AVX512_KNM AVX512_SKX AVX512_CLX AVX512_CNL AVX512_ICL"
7272
steps:
@@ -96,7 +96,7 @@ jobs:
9696
# for add-apt-repository
9797
sudo apt install software-properties-common -y
9898
sudo add-apt-repository ppa:deadsnakes/ppa -y
99-
sudo apt install python3.9-dev -y
99+
sudo apt install python3.9-dev ninja-build -y
100100
sudo ln -s /usr/bin/python3.9 /usr/bin/pythonx
101101
pythonx -m pip install --upgrade pip setuptools wheel
102102
pythonx -m pip install -r test_requirements.txt
@@ -195,6 +195,7 @@ jobs:
195195
- uses: actions/setup-python@61a6322f88396a6271a6ee3565807d608ecaddd1 # v4.7.0
196196
with:
197197
python-version: ${{ env.PYTHON_VERSION }}
198+
198199
- uses: ./.github/actions
199200

200201
blas64:
@@ -287,26 +288,6 @@ jobs:
287288
python-version: ${{ env.PYTHON_VERSION }}
288289
- uses: ./.github/actions
289290

290-
numpy2_flag:
291-
needs: [smoke_test]
292-
runs-on: ubuntu-latest
293-
if: github.event_name != 'push'
294-
env:
295-
# Test for numpy-2.0 feature-flagged behavior.
296-
NPY_NUMPY_2_BEHAVIOR: 1
297-
# Using the future "weak" state doesn't pass tests
298-
# currently unfortunately
299-
NPY_PROMOTION_STATE: legacy
300-
steps:
301-
- uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2
302-
with:
303-
submodules: recursive
304-
fetch-depth: 0
305-
- uses: actions/setup-python@bd6b4b6205c4dbad673328db7b31b7fab9e241c0 # v4.6.1
306-
with:
307-
python-version: ${{ env.PYTHON_VERSION }}
308-
- uses: ./.github/actions
309-
310291
no_openblas:
311292
needs: [smoke_test]
312293
runs-on: ubuntu-latest
@@ -361,11 +342,10 @@ jobs:
361342
sudo apt update
362343
sudo apt install -y gcc-arm-linux-gnueabihf g++-arm-linux-gnueabihf gfortran-arm-linux-gnueabihf
363344
364-
# Keep the `test_requirements.txt` dependency-subset synced
365-
docker run --name the_container --interactive -v /:/host arm32v7/ubuntu:22.04 /bin/bash -c "
345+
docker run --name the_container --interactive -v /:/host -v $(pwd):/numpy arm32v7/ubuntu:22.04 /bin/bash -c "
366346
apt update &&
367347
apt install -y git python3 python3-dev python3-pip &&
368-
python3 -m pip install cython==0.29.34 setuptools\<49.2.0 hypothesis==6.23.3 pytest==6.2.5 'typing_extensions>=4.2.0' &&
348+
python3 -m pip install -r /numpy/test_requirements.txt
369349
ln -s /host/lib64 /lib64 &&
370350
ln -s /host/lib/x86_64-linux-gnu /lib/x86_64-linux-gnu &&
371351
ln -s /host/usr/arm-linux-gnueabihf /usr/arm-linux-gnueabihf &&
@@ -443,7 +423,7 @@ jobs:
443423
python-version: ${{ env.PYTHON_VERSION }}
444424
- name: Install Intel SDE
445425
run: |
446-
curl -o /tmp/sde.tar.xz https://downloadmirror.intel.com/751535/sde-external-9.14.0-2022-10-25-lin.tar.xz
426+
curl -o /tmp/sde.tar.xz https://downloadmirror.intel.com/784319/sde-external-9.24.0-2023-07-13-lin.tar.xz
447427
mkdir /tmp/sde && tar -xvf /tmp/sde.tar.xz -C /tmp/sde/
448428
sudo mv /tmp/sde/* /opt/sde && sudo ln -s /opt/sde/sde64 /usr/bin/sde
449429
- name: Install dependencies
@@ -454,14 +434,14 @@ jobs:
454434
run: |
455435
export CC=/usr/bin/gcc-12
456436
export CXX=/usr/bin/g++-12
457-
python -m pip install -e .
437+
python setup.py develop
458438
- name: Show config
459439
run: |
460440
python -c "import numpy as np; np.show_config()"
461441
# Run only a few tests, running everything in an SDE takes a long time
462442
# Using pytest directly, unable to use python runtests.py -n -t ...
463-
# Disabled running in the SDE because of an SDE bug
464443
- name: Run linalg/ufunc/umath tests
465444
run: |
466445
python -m pytest numpy/core/tests/test_umath* numpy/core/tests/test_ufunc.py numpy/linalg/tests/test_*
446+
# Can't run on SDE just yet: see https://github.com/numpy/numpy/issues/23545#issuecomment-1659047365
467447
#sde -spr -- python -m pytest numpy/core/tests/test_umath* numpy/core/tests/test_ufunc.py numpy/linalg/tests/test_*

‎.github/workflows/emscripten.yml

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,11 @@ jobs:
5555
run: pip install "pydantic<2" pyodide-build==$PYODIDE_VERSION
5656

5757
- name: Build
58-
run: CFLAGS=-g2 LDFLAGS=-g2 pyodide build
58+
run: |
59+
# Pyodide is still in the process of adding better/easier support for
60+
# non-setup.py based builds.
61+
cp pyproject.toml.setuppy pyproject.toml
62+
CFLAGS=-g2 LDFLAGS=-g2 pyodide build
5963
6064
- name: set up node
6165
uses: actions/setup-node@e33196f7422957bea03ed53f6fbb155025ffc7b8 # v3.7.0
@@ -67,6 +71,7 @@ jobs:
6771
pyodide venv .venv-pyodide
6872
source .venv-pyodide/bin/activate
6973
pip install dist/*.whl
74+
python -c "import sys; print(sys.platform)"
7075
pip install -r test_requirements.txt
7176
- name: Test
7277
run: |

‎.github/workflows/linux_meson.yml

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,12 @@ jobs:
2424
meson_spin:
2525
if: "github.repository == 'numpy/numpy'"
2626
runs-on: ubuntu-latest
27+
strategy:
28+
matrix:
29+
USE_NIGHTLY_OPENBLAS: [false, true]
30+
env:
31+
USE_NIGHTLY_OPENBLAS: ${{ matrix.USE_NIGHTLY_OPENBLAS }}
32+
name: "Test Linux (${{ matrix.USE_NIGHTLY_OPENBLAS && 'nightly' || 'stable' }} OpenBLAS)"
2733
steps:
2834
- uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3
2935
with:
@@ -35,7 +41,15 @@ jobs:
3541
- name: Install dependencies
3642
run: |
3743
pip install -r build_requirements.txt
38-
sudo apt-get install -y libopenblas-serial-dev
44+
# Install OpenBLAS
45+
set -xe
46+
if [[ $USE_NIGHTLY_OPENBLAS == "true" ]]; then
47+
target=$(python tools/openblas_support.py --nightly)
48+
else
49+
target=$(python tools/openblas_support.py)
50+
fi
51+
sudo cp -r $target/lib/* /usr/lib
52+
sudo cp $target/include/* /usr/include
3953
- name: Build
4054
shell: 'script -q -e -c "bash --noprofile --norc -eo pipefail {0}"'
4155
env:
@@ -52,6 +66,7 @@ jobs:
5266
shell: 'script -q -e -c "bash --noprofile --norc -eo pipefail {0}"'
5367
env:
5468
TERM: xterm-256color
69+
LD_LIBRARY_PATH: "/usr/local/lib/" # to find libopenblas.so.0
5570
run: |
56-
pip install pytest hypothesis typing_extensions
57-
spin test
71+
pip install pytest pytest-xdist hypothesis typing_extensions
72+
spin test -j auto

‎.github/workflows/linux_musl.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -59,8 +59,8 @@ jobs:
5959
RUNNER_OS=Linux sh tools/wheels/cibw_before_build.sh .
6060
6161
pip install -r build_requirements.txt
62-
pip install pytest hypothesis typing_extensions
62+
pip install pytest pytest-xdist hypothesis typing_extensions
6363
6464
# use meson to build and test
6565
spin build
66-
spin test
66+
spin test -j auto

‎.github/workflows/wheels.yml

Lines changed: 12 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ on:
2121
# │ │ │ ┌───────────── month (1 - 12 or JAN-DEC)
2222
# │ │ │ │ ┌───────────── day of the week (0 - 6 or SUN-SAT)
2323
# │ │ │ │ │
24-
- cron: "42 1 * * 4"
24+
- cron: "42 2 * * SUN,WED"
2525
push:
2626
pull_request:
2727
types: [labeled, opened, synchronize, reopened]
@@ -78,8 +78,7 @@ jobs:
7878
- [ubuntu-20.04, musllinux_x86_64]
7979
- [macos-12, macosx_x86_64]
8080
- [windows-2019, win_amd64]
81-
- [windows-2019, win32]
82-
python: ["cp39", "cp310", "cp311", "pp39"]
81+
python: ["cp39", "cp310", "cp311", "cp312"] # "pp39"
8382
exclude:
8483
# Don't build PyPy 32-bit windows
8584
- buildplat: [windows-2019, win32]
@@ -101,25 +100,20 @@ jobs:
101100
# https://github.com/actions/checkout/issues/338
102101
fetch-depth: 0
103102

103+
- name: pkg-config-for-win
104+
run: |
105+
choco install -y --checksum 6004DF17818F5A6DBF19CB335CC92702 pkgconfiglite
106+
if: runner.os == 'windows'
107+
104108
# Used to push the built wheels
105109
- uses: actions/setup-python@61a6322f88396a6271a6ee3565807d608ecaddd1 # v4.7.0
106110
with:
107111
python-version: "3.x"
108112

109-
# We need rtools 4.0 to have 32 bit support on windows
110-
- if: runner.os == 'windows'
111-
uses: r-windows/install-rtools@ca1090c210479e995c03019a22b9798cdf57073a # main
112-
113-
- name: setup rtools for 32-bit
114-
run: |
115-
echo "PLAT=i686" >> $env:GITHUB_ENV
116-
echo "PATH=c:\rtools40\mingw32\bin;$env:PATH" >> $env:GITHUB_ENV
117-
gfortran --version
118-
if: ${{ matrix.buildplat[1] == 'win32' }}
119-
120113
- name: Build wheels
121114
uses: pypa/cibuildwheel@f21bb8376a051ffb6cb5604b28ccaef7b90e8ab7 # v2.14.1
122115
env:
116+
CIBW_PRERELEASE_PYTHONS: True
123117
CIBW_BUILD: ${{ matrix.python }}-${{ matrix.buildplat[1] }}
124118

125119
- uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # v3.1.2
@@ -192,12 +186,14 @@ jobs:
192186
python-version: "3.9"
193187
- name: Build sdist
194188
run: |
195-
python setup.py sdist
189+
python -m pip install -U pip build
190+
python -m build --sdist -Csetup-args=-Dallow-noblas=true
196191
- name: Test the sdist
197192
run: |
198193
# TODO: Don't run test suite, and instead build wheels from sdist
199194
# Depends on pypa/cibuildwheel#1020
200-
python -m pip install dist/*.gz
195+
python -m pip install dist/*.gz -Csetup-args=-Dallow-noblas=true
196+
pip install ninja
201197
pip install -r test_requirements.txt
202198
cd .. # Can't import numpy within numpy src directory
203199
python -c "import numpy, sys; print(numpy.__version__); sys.exit(numpy.test() is False)"

‎.github/workflows/windows_meson.yml

Lines changed: 38 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,10 +17,10 @@ permissions:
1717
contents: read # to fetch code (actions/checkout)
1818

1919
jobs:
20-
meson:
21-
name: Meson windows build/test
20+
msvc_64bit_python_openblas:
21+
name: MSVC, x86-64, LP64 OpenBLAS
2222
runs-on: windows-2019
23-
# if: "github.repository == 'numpy/numpy'"
23+
if: "github.repository == 'numpy/numpy'"
2424
steps:
2525
- name: Checkout
2626
uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3
@@ -86,3 +86,38 @@ jobs:
8686
echo "LASTEXITCODE is '$LASTEXITCODE'"
8787
python -c "import numpy, sys; sys.exit(numpy.test(verbose=3) is False)"
8888
echo "LASTEXITCODE is '$LASTEXITCODE'"
89+
90+
msvc_32bit_python_openblas:
91+
name: MSVC, 32-bit Python, no BLAS
92+
runs-on: windows-2019
93+
if: "github.repository == 'numpy/numpy'"
94+
steps:
95+
- name: Checkout
96+
uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3
97+
with:
98+
submodules: recursive
99+
fetch-depth: 0
100+
101+
- name: Setup Python (32-bit)
102+
uses: actions/setup-python@61a6322f88396a6271a6ee3565807d608ecaddd1 # v4.7.0
103+
with:
104+
python-version: '3.10'
105+
architecture: 'x86'
106+
107+
- name: Setup MSVC (32-bit)
108+
uses: bus1/cabuild/action/msdevshell@e22aba57d6e74891d059d66501b6b5aed8123c4d # v1
109+
with:
110+
architecture: 'x86'
111+
112+
- name: Build and install
113+
run: |
114+
python -m pip install . -v -Ccompile-args="-j2" -Csetup-args="-Dallow-noblas=true"
115+
116+
- name: Install test dependencies
117+
run: |
118+
python -m pip install -r test_requirements.txt
119+
120+
- name: Run test suite (fast)
121+
run: |
122+
cd tools
123+
python -m pytest --pyargs numpy -m "not slow" -n2

0 commit comments

Comments
 (0)