Description
My colleague @WiseAndy and I noticed when converting from an ElementwiseKernel to a Raw kernel that the data layout between the way the two are handling transposed matrices (ie. matrix.T) is inconsistent. It seems like the ElementwiseKernel is giving the right result - presumably doing something to the array before it is executed.
See below, where we create a matrix, and then copy it directly back out. Notice on the RawKernel transposed one, that the the values are different.
matrix = cp.array([[1, 3], [2, 4]], dtype=cp.float32)
Elementwise Transposed : [[1.0, 2.0], [3.0, 4.0]]
Elementwise Transposed Copied : [[1.0, 2.0], [3.0, 4.0]]
RawKernel Transposed : [[1.0, 3.0], [2.0, 4.0]] *****
RawKernel Transposed Copied : [[1.0, 2.0], [3.0, 4.0]]
To Reproduce
import cupy as cp
import numpy as np
elementwise_kernel = cp.ElementwiseKernel(
'float32 matrix',
'float32 out_matrix',
'out_matrix = matrix;', # Simple 1-to-1 copy
'elementwise_copy_test'
)
raw_kernel_code = '''
extern "C" __global__
void raw_copy_test_kernel(const float* matrix, float* out_matrix, int num_elements) {
int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i >= num_elements) return;
out_matrix[i] = matrix[i];
}
'''
raw_kernel = cp.RawKernel(raw_kernel_code, 'raw_copy_test_kernel')
# Original 2x2 matrix on GPU
matrix = cp.array([[1, 3], [2, 4]], dtype=cp.float32)
d_out = cp.zeros_like(matrix)
num_elements = matrix.size
block = 256
grid = (num_elements + block - 1) // block
# Elementwise with tranposed.
elementwise_kernel(matrix.T, d_out)
print("Elementwise Transposed :", d_out.get().tolist())
# Elementwise with copied tranposed.
elementwise_kernel(matrix.T.copy(), d_out)
print("Elementwise Transposed Copied :", d_out.get().tolist())
# RawKernel with tranposed.
raw_kernel((grid,), (block,), (matrix.T, d_out, num_elements))
print("RawKernel Transposed :", d_out.get().tolist())
# Elementwise with copied tranposed.
raw_kernel((grid,), (block,), (matrix.T.copy(), d_out, num_elements))
print("RawKernel Transposed Copied :", d_out.get().tolist())Installation
Wheel (pip install cupy-***)
Environment
OS : Windows-10-10.0.26100-SP0
Python Version : 3.10.11
CuPy Version : 13.4.1
CuPy Platform : NVIDIA CUDA
NumPy Version : 2.2.6
SciPy Version : 1.15.3
Cython Build Version : 3.0.12
Cython Runtime Version : None
CUDA Root : C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6
nvcc PATH : C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6\bin\nvcc.EXE
CUDA Build Version : 12080
CUDA Driver Version : 12060
CUDA Runtime Version : 12080 (linked to CuPy) / 12060 (locally installed)
CUDA Extra Include Dirs : []
cuBLAS Version : 120604
cuFFT Version : 11300
cuRAND Version : 10307
cuSOLVER Version : (11, 7, 1)
cuSPARSE Version : 12504
NVRTC Version : (12, 6)
Thrust Version : 200800
CUB Build Version : 200800
Jitify Build Version : 1a0ca0e
cuDNN Build Version : None
cuDNN Version : None
NCCL Build Version : None
NCCL Runtime Version : None
cuTENSOR Version : None
cuSPARSELt Build Version : None
Device 0 Name : NVIDIA GeForce RTX 4090 Laptop GPU
Device 0 Compute Capability : 89
Device 0 PCI Bus ID : 0000:01:00.0
Additional Information
No response
Description
My colleague @WiseAndy and I noticed when converting from an ElementwiseKernel to a Raw kernel that the data layout between the way the two are handling transposed matrices (ie.
matrix.T) is inconsistent. It seems like the ElementwiseKernel is giving the right result - presumably doing something to the array before it is executed.See below, where we create a matrix, and then copy it directly back out. Notice on the RawKernel transposed one, that the the values are different.
To Reproduce
Installation
Wheel (
pip install cupy-***)Environment
Additional Information
No response