Open
Description
Multiplication of a matrix with its transpose causes segfault for "large" matrices. Multiplication works for "small" matrices and also works if transpose is copied (np.copy()). Specific examples of large/small matrices below.
I do have sufficient amount of memory (>350GB) on the machine, and I observe the same behavior on EC2 instance (both Ubuntu and Amazon Linux) as well as in a Docker container.
Reproducing code example:
This fails:
import numpy as np
matrix = np.random.rand(80_000, 3072)
out = matrix.dot(matrix.T)This also fails:
import numpy as np
matrix = np.random.rand(80_000, 3072)
out = matrix @ matrix.TAll these work:
import numpy as np
matrix = np.random.rand(75_000, 3072)
out = matrix @ matrix.Timport numpy as np
matrix = np.random.rand(75_000, 3072)
out = matrix.dot(matrix.T)import numpy as np
matrix = np.random.rand(80_000, 3072)
out = matrix @ np.copy(matrix.T)Error message:
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `python'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007fc6f108cd49 in dgemm_oncopy_SKYLAKEX ()
from /opt/bitnami/python/lib/python3.8/site-packages/numpy/core/../../numpy.libs/libopenblasp-r0-2d23e62b.3.17.so
[Current thread is 1 (Thread 0x7fc6f2a92f00 (LWP 676))]
(gdb) where
#0 0x00007fc6f108cd49 in dgemm_oncopy_SKYLAKEX ()
from /opt/bitnami/python/lib/python3.8/site-packages/numpy/core/../../numpy.libs/libopenblasp-r0-2d23e62b.3.17.so
#1 0x00007fc6f01a0734 in inner_thread ()
from /opt/bitnami/python/lib/python3.8/site-packages/numpy/core/../../numpy.libs/libopenblasp-r0-2d23e62b.3.17.so
#2 0x00007fc6f02c06f5 in exec_blas ()
from /opt/bitnami/python/lib/python3.8/site-packages/numpy/core/../../numpy.libs/libopenblasp-r0-2d23e62b.3.17.so
#3 0x00007fc6f01a10f3 in dsyrk_thread_LT ()
from /opt/bitnami/python/lib/python3.8/site-packages/numpy/core/../../numpy.libs/libopenblasp-r0-2d23e62b.3.17.so
#4 0x00007fc6f00b08ab in cblas_dsyrk ()
from /opt/bitnami/python/lib/python3.8/site-packages/numpy/core/../../numpy.libs/libopenblasp-r0-2d23e62b.3.17.so
#5 0x00007fc6f21e45cd in DOUBLE_matmul_matrixmatrix.constprop.6 ()
from /opt/bitnami/python/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-x86_64-linux-gnu.so
#6 0x00007fc6f21e86fb in DOUBLE_matmul ()
from /opt/bitnami/python/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-x86_64-linux-gnu.so
#7 0x00007fc6f21f7b49 in PyUFunc_GeneralizedFunctionInternal ()
NumPy/Python version information:
1.21.2 3.8.10 (default, Jun 2 2021, 10:49:15)
[GCC 9.4.0]
Metadata
Metadata
Assignees
Labels
No labels