[Build] Compilation of ArrayFire 3.10.0 fails with errors in ocl_cl2hpp-src #3694
Description
Compilation of ArrayFire 3.10.0 fails with errors in ocl_cl2hpp-src.
Description
While trying to build ArrayFire 3.10.0 for our Canadian HPC systems with GCC 12.3 and CUDA 12.6
I run into the following error:
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:1872:79: error: missing binary operator before token "("
1872 | #if CL_KHR_COMMAND_BUFFER_MUTABLE_DISPATCH_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 2)
| ^
Our build recipe is based on the one we used for version 3.9.0 just with a few updated dependencies, most notable CUDA 12.6 instead of 12.2.
The same error occurs with ArrayFire 3.10.0 and CUDA 12.2 with the same dependencies we used for ArrayFire 3.9.0.
When I turn off OpenCL using -DAF_BUILD_OPENCL=OFF I can successfully build ArrayFire 3.10.0.
Error Log
Full cmake and make output:
arrayfire-3.10.0-make.log
arrayfire-3.10.0-cmake.log
[ 78%] Building CXX object test/CMakeFiles/test_qr_dense_opencl.dir/qr_dense.cpp.o
In file included from /home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/cl2.hpp:17,
from /home/stuekero/scratch/ArrayFire/arrayfire-full-3.10.0/test/interop_opencl_external_context_snippet.cpp:30:
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:1872:79: error: missing binary operator before token "("
1872 | #if CL_KHR_COMMAND_BUFFER_MUTABLE_DISPATCH_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 2)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11569:62: error: missing binary operator before token "("
11569 | #if CL_KHR_COMMAND_BUFFER_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 4)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11603:62: error: missing binary operator before token "("
11603 | #if CL_KHR_COMMAND_BUFFER_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 4)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11646:62: error: missing binary operator before token "("
11646 | #if CL_KHR_COMMAND_BUFFER_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 4)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11689:62: error: missing binary operator before token "("
11689 | #if CL_KHR_COMMAND_BUFFER_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 4)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11728:62: error: missing binary operator before token "("
11728 | #if CL_KHR_COMMAND_BUFFER_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 4)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11767:62: error: missing binary operator before token "("
11767 | #if CL_KHR_COMMAND_BUFFER_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 4)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11806:62: error: missing binary operator before token "("
11806 | #if CL_KHR_COMMAND_BUFFER_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 4)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11844:62: error: missing binary operator before token "("
11844 | #if CL_KHR_COMMAND_BUFFER_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 4)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11864:62: error: missing binary operator before token "("
11864 | #if CL_KHR_COMMAND_BUFFER_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 4)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11907:20: error: missing binary operator before token "("
11907 | CL_MAKE_VERSION(0, 9, 2)
| ^
In file included from /home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/cl2.hpp:17,
from /home/stuekero/scratch/ArrayFire/arrayfire-full-3.10.0/test/ocl_ext_context.cpp:29:
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:1872:79: error: missing binary operator before token "("
1872 | #if CL_KHR_COMMAND_BUFFER_MUTABLE_DISPATCH_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 2)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11569:62: error: missing binary operator before token "("
11569 | #if CL_KHR_COMMAND_BUFFER_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 4)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11603:62: error: missing binary operator before token "("
11603 | #if CL_KHR_COMMAND_BUFFER_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 4)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11646:62: error: missing binary operator before token "("
11646 | #if CL_KHR_COMMAND_BUFFER_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 4)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11689:62: error: missing binary operator before token "("
11689 | #if CL_KHR_COMMAND_BUFFER_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 4)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11728:62: error: missing binary operator before token "("
11728 | #if CL_KHR_COMMAND_BUFFER_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 4)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11767:62: error: missing binary operator before token "("
11767 | #if CL_KHR_COMMAND_BUFFER_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 4)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11806:62: error: missing binary operator before token "("
11806 | #if CL_KHR_COMMAND_BUFFER_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 4)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11844:62: error: missing binary operator before token "("
11844 | #if CL_KHR_COMMAND_BUFFER_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 4)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11864:62: error: missing binary operator before token "("
11864 | #if CL_KHR_COMMAND_BUFFER_EXTENSION_VERSION > CL_MAKE_VERSION(0, 9, 4)
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11907:20: error: missing binary operator before token "("
11907 | CL_MAKE_VERSION(0, 9, 2)
| ^
[ 78%] Built target test_interop_opencl_custom_kernel_snippet_opencl
[ 78%] Building CXX object test/CMakeFiles/test_random_opencl.dir/random.cpp.o
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11919:45: error: ‘cl_command_buffer_update_type_khr’ was not declared in this scope; did you mean ‘cl_command_buffer_structure_type_khr’?
11919 | cl_int updateMutableCommands(std::array<cl_command_buffer_update_type_khr,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| cl_command_buffer_structure_type_khr
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11920:56: error: template argument 1 is invalid
11920 | ArrayLength> &config_types,
| ^
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp: In member function ‘cl_int cl::CommandBufferKhr::updateMutableCommands(int&, std::array<const void*, ArrayLength>&)’:
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/opencl.hpp:11928:57: error: request for member ‘data’ in ‘config_types’, which is of non-class type ‘int’
11928 | config_types.data(), configs.data()),
| ^~~~
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/cl2.hpp: At global scope:
/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include/CL/cl2.hpp:18:151: note: ‘#pragma message: cl2.hpp has been renamed to opencl.hpp to make it clear that it supports all versions of OpenCL. Please include opencl.hpp directly.’
18 | #pragma message("cl2.hpp has been renamed to opencl.hpp to make it clear that it supports all versions of OpenCL. Please include opencl.hpp directly.")
|
Build Environment
Compiler version: GCC 12.3.1; nvcc 12.6.2
Operating system: Rocky Linux release 9.6 (Blue Onyx) but with a custom Gentoo-based user-space
(i.e. almost everything that you would normally expect under /usr is now under /cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/ but our compilers all know about this and we've installed many hundreds of packages like this over the past 8 years)
Build environment:
$ module load StdEnv/2023 cuda/12.6 cmake/3.27.7 boost/1.85.0 fftw/3.3.10 cudnn/9.10.0.56 glfw/3.4 spdlog/1.11.0
$ module list
Currently Loaded Modules:
1) CCconfig 14) cudacompat/.12.6 (H)
2) gentoo/2023 (S) 15) cudacore/.12.6.2 (H,t)
3) gcccore/.12.3 (H) 16) gdrcopy/2.3.1
4) gcc/12.3 (t) 17) ucx-cuda/1.14.1
5) hwloc/2.9.1 18) nccl/2.26.2 (t)
6) ucx/1.14.1 19) ucc-cuda/1.2.0
7) libfabric/1.18.0 20) cuda/12.6 (t)
8) pmix/4.2.4 21) cmake/3.27.7 (t)
9) ucc/1.2.0 22) boost/1.85.0 (t)
10) openmpi/4.1.5 (m) 23) fftw/3.3.10 (math)
11) flexiblas/3.3.1 24) cudnn/9.10.0.56 (math)
12) imkl/2023.2.0 (math) 25) glfw/3.4
13) StdEnv/2023 (S) 26) spdlog/1.11.0
CMake Command:
cmake -DCUDA_architecture_build_targets="7.0;8.0;9.0" \
-DCBLAS_INCLUDE_DIR=$EBROOTFLEXIBLAS/include \
-DCBLAS_LIBRARIES="$EBROOTFLEXIBLAS/lib" \
-DLAPACK_INCLUDE_DIR=$EBROOTFLEXIBLAS/include \
-DLAPACK_LIBRARIES="$EBROOTFLEXIBLAS/lib -llapacke" \
-DAF_BUILD_FORGE=ON \
-DCMAKE_CUDA_COMPILER=$EBROOTCUDACORE/bin/nvcc \
../arrayfire-full-3.10.0 |& tee ../cmake-arrayfire-3.10.0.out
make -j 8
CMake variables:
$ cmake -L
CMake Warning:
No source or binary directory provided. Both will be assumed to be the
same as the current working directory, but note that this warning will
become a fatal error in future CMake releases.
CMake Error: The source directory "/home/stuekero/scratch/ArrayFire/build" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
-- Cache values
AF_BUILD_CPU:BOOL=ON
AF_BUILD_CUDA:BOOL=ON
AF_BUILD_DOCS:BOOL=ON
AF_BUILD_EXAMPLES:BOOL=ON
AF_BUILD_FORGE:BOOL=ON
AF_BUILD_ONEAPI:BOOL=OFF
AF_BUILD_OPENCL:BOOL=ON
AF_BUILD_UNIFIED:BOOL=ON
AF_COMPUTE_LIBRARY:STRING=Intel-MKL
AF_CTEST_SEPARATED:BOOL=OFF
AF_SKIP_UNSUPPORTED_TESTS:BOOL=OFF
AF_STACKTRACE_TYPE:STRING=Basic
AF_WITH_CUDNN:BOOL=ON
AF_WITH_EXTERNAL_PACKAGES_ONLY:BOOL=OFF
AF_WITH_FAST_MATH:BOOL=OFF
AF_WITH_FMT_HEADER_ONLY:BOOL=OFF
AF_WITH_SPDLOG_HEADER_ONLY:BOOL=OFF
AF_WITH_STATIC_CUDA_NUMERIC_LIBS:BOOL=OFF
BOOST_ROOT:PATH=/cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v4/Compiler/gcc12/boost/1.85.0
BUILD_TESTING:BOOL=ON
CLBlast_DIR:PATH=CLBlast_DIR-NOTFOUND
CMAKE_BUILD_TYPE:STRING=Release
CMAKE_INSTALL_PREFIX:PATH=/usr/local
CUDA_CUDA_STUB:FILEPATH=/cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Core/cudacore/12.6.2/lib64/stubs/libcuda.so
ENABLE_BLACS:BOOL=OFF
ENABLE_BLAS95:BOOL=OFF
ENABLE_CDFT:BOOL=OFF
ENABLE_CPARDISO:BOOL=OFF
ENABLE_LAPACK95:BOOL=OFF
ENABLE_OMP_OFFLOAD:BOOL=OFF
ENABLE_SCALAPACK:BOOL=OFF
FETCHCONTENT_UPDATES_DISCONNECTED_SPAN-LITE:BOOL=OFF
GTEST_HAS_ABSL:BOOL=OFF
GTest_DIR:PATH=GTest_DIR-NOTFOUND
MKL_ARCH:STRING=intel64
MKL_DIR:PATH=/cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Core/imkl/2023.2.0/mkl/2023.2.0/lib/cmake/mkl
MKL_H:PATH=/cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Core/imkl/2023.2.0/mkl/2023.2.0/include
MKL_LINK:STRING=dynamic
MKL_THREAD_LAYER:STRING=Intel OpenMP
NVPRUNE:FILEPATH=/cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Core/cudacore/12.6.2/bin/nvprune
OMP_LIBRARY:FILEPATH=/cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Core/imkl/2023.2.0/compiler/latest/linux/compiler/lib/intel64_lin/libiomp5.so
cl2hpp_header_file_path:PATH=cl2hpp_header_file_path-NOTFOUND
cl2hpp_var:PATH=/home/stuekero/scratch/ArrayFire/build/extern/ocl_cl2hpp-src/include
mkl_core_file:FILEPATH=/cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Core/imkl/2023.2.0/mkl/2023.2.0/lib/intel64/libmkl_core.so
mkl_intel_lp64_file:FILEPATH=/cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Core/imkl/2023.2.0/mkl/2023.2.0/lib/intel64/libmkl_intel_lp64.so
mkl_intel_thread_file:FILEPATH=/cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Core/imkl/2023.2.0/mkl/2023.2.0/lib/intel64/libmkl_intel_thread.so