KOKKOS Installation

Dear LAMMPS developer,
I am installing the latest version of LAMMPS (15Jun2023) with KOKKOS package. GPU=RTX 3090, cuda=12.2, ubuntu=22.04.
Without KOKKOS package, the installation is done without any problems. But when adding command
-D Kokkos_ENABLE_CUDA=yes
the following error:

CMake Warning:
Ignoring extra path from command line:

“/opt/lammps-15Jun2023gpukokkos/lammps-15Jun2023/build/[-D”

CMake Warning:
Ignoring extra path from command line:

“]”

CMake Deprecation Warning at CMakeLists.txt:18 (cmake_policy):
The OLD behavior for policy CMP0109 will be removed from a future version
of CMake.

The cmake-policies(7) manual explains that the OLD behaviors of all
policies are deprecated and that a policy should be set to OLD only under
specific short-term circumstances. Projects should be ported to the NEW
behavior and not rely on setting a policy to OLD.

– The CXX compiler identification is GNU 11.3.0
– Detecting CXX compiler ABI info
– Detecting CXX compiler ABI info - done
– Check for working CXX compiler: /usr/bin/c++ - skipped
– Detecting CXX compile features
– Detecting CXX compile features - done
– Found Git: /usr/bin/git (found version “2.34.1”)
– Running check for auto-generated files from make-based build system
– Found MPI_CXX: /usr/local/lib/libmpicxx.so (found version “3.1”)
– Found MPI: TRUE (found version “3.1”)
– Looking for C++ include omp.h
– Looking for C++ include omp.h - found
– Found OpenMP_CXX: -fopenmp (found version “4.5”)
– Found OpenMP: TRUE (found version “4.5”) found components: CXX
– Found JPEG: /usr/lib/x86_64-linux-gnu/libjpeg.so (found version “80”)
– Found PNG: /usr/lib/x86_64-linux-gnu/libpng.so (found version “1.6.37”)
– Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version “1.2.11”)
– Found GZIP: /usr/bin/gzip
– Found FFMPEG: /usr/bin/ffmpeg
– Found PkgConfig: /usr/bin/pkg-config (found version “0.29.2”)
– Checking for module ‘fftw3’
– Found fftw3, version 3.3.4
– Found FFTW3: /usr/local/lib/libfftw3.a
– Looking for C++ include cmath
– Looking for C++ include cmath - found
– KOKKOS: Enabling CUDA LAMBDA function support
– Setting default Kokkos CXX standard to 14
– The project name is: Kokkos
/opt/lammps-15Jun2023gpukokkos/lammps-15Jun2023/lib/kokkos/bin/nvcc_wrapper: line 640: nvcc: command not found
CMake Error at /opt/lammps-15Jun2023gpukokkos/lammps-15Jun2023/lib/kokkos/cmake/kokkos_compiler_id.cmake:12 (STRING):
STRING sub-command REPLACE requires at least four arguments.
Call Stack (most recent call first):
/opt/lammps-15Jun2023gpukokkos/lammps-15Jun2023/lib/kokkos/cmake/kokkos_compiler_id.cmake:53 (kokkos_internal_have_compiler_nvcc)
/opt/lammps-15Jun2023gpukokkos/lammps-15Jun2023/lib/kokkos/cmake/kokkos_tribits.cmake:215 (INCLUDE)
/opt/lammps-15Jun2023gpukokkos/lammps-15Jun2023/lib/kokkos/CMakeLists.txt:189 (KOKKOS_SETUP_BUILD_ENVIRONMENT)

– Using -std=c++14 for C++14 standard as feature
CMake Error at /opt/lammps-15Jun2023gpukokkos/lammps-15Jun2023/lib/kokkos/cmake/kokkos_test_cxx_std.cmake:132 (MESSAGE):
Invalid compiler for CUDA. The compiler must be nvcc_wrapper or Clang or
NVC++ or use kokkos_launch_compiler, but compiler ID was GNU
Call Stack (most recent call first):
/opt/lammps-15Jun2023gpukokkos/lammps-15Jun2023/lib/kokkos/cmake/kokkos_tribits.cmake:242 (INCLUDE)
/opt/lammps-15Jun2023gpukokkos/lammps-15Jun2023/lib/kokkos/CMakeLists.txt:189 (KOKKOS_SETUP_BUILD_ENVIRONMENT)

Please post your full CMake command.

You need to resolve this:

/opt/lammps-15Jun2023gpukokkos/lammps-15Jun2023/lib/kokkos/bin/nvcc_wrapper: line 640: nvcc: command not found

Dear stamoor,
Thank you. My command is:
sudo cmake [-D PKG_QEQ=on -D PKG_REAXFF=on -D PKG_RIGID=on -D PKG_RIGID=on -D PKG_MANYBODY=on -D PKG_BODY=on -D PKG_MOLECULE=on -D PKG_OPT=on -D PKG_KSPACE=yes -D FFT=FFTW3 -D BUILD_MPI=yes -D BUILD_OMP=yes -D LAMMPS_MACHINE=mpi -D PKG_GPU=on -D GPU_API=cuda -D GPU_PREC=mixed -D GPU_ARCH=sm_86 -D CUDPP_OPT=yes -D BIN2C=/usr/local/cuda-12.2/bin/bin2c -D BUILD_OMP=yes -D PKG_KOKKOS=on -D Kokkos_ARCH_ZEN3=yes -D Kokkos_ARCH_AMPERE86=yes -D Kokkos_ENABLE_OPENMP=yes -D Kokkos_ENABLE_CUDA=yes ] …/cmake

What CUDA version? What is the output of which nvcc?

which nvcc:
/usr/local/cuda-12.2/bin/nvcc

nvcc -V:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Jun_13_19:16:58_PDT_2023
Cuda compilation tools, release 12.2, V12.2.91
Build cuda_12.2.r12.2/compiler.32965470_0

Never use “sudo” to compile/configure software. It doesn’t help anything and is a big security risk. One typo in the CMakeLists.txt file or one bad error on the cmake command line and you can destroy your installation since the command will run with superuser privileges. As a regular user, you will just get a permission denied message. Of course, once you use “sudo”, you will have all kinds of problems to repeat as a regular user, since now you have created lots of files/folders that are owned by root (and not you).

What is the purpose of the square brackets “[” here? It is not anywhere in the documentation, but it is likely the reason for the warning/error below which will lead to an incorrect CMake command line (due to ignoring/misinterpretation of the -D.

1 Like

Dear Axel,

  1. Thank you for your advice on the sudo.
  2. The command line was too long and I included the square brackets to avoid confusion. I removed the square brackets and the warning was removed. thanks a lot.

Do you know a solution for the following error?

/opt/lammps-15Jun2023gpukokkos/lammps-15Jun2023/lib/kokkos/bin/nvcc_wrapper: line 640: nvcc: command not found

Does the error still appear if you remove sudo in your command? By default sudo does not inherit environment variables, and since your nvcc is installed in a non-standard location (which you may have made it found by changing PATH etc), it’s probably not found under sudo.

Hi,
When I remove sudo, the error is removed, but after command make -j20, the following error occurs:

[100%] Linking CXX executable lmp_mpi
/usr/bin/ld: cannot find -lcufft: No such file or directory
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/lmp.dir/build.make:116: lmp_mpi] Error 1
make[1]: *** [CMakeFiles/Makefile2:366: CMakeFiles/lmp.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

This is ultimately a problem of your CUDA toolkit installation. You are probably missing the correct setting of the LIBRARY_PATH environment variable (not to be confused with LD_LIBRARY_PATH).

You should be able to work around this by using -DFFT=KISS when configuring LAMMPS.

You are exactly right. By setting -D FFT=KISS, the installation was done successfully.
But the following error occurs during LAMMPS run:

ERROR: Package kokkos command without KOKKOS package enabled (src/input.cpp:1695)
Last command: package kokkos neigh full comm device

Please see my cmake command:

cmake -D PKG_QEQ=on -D PKG_REAXFF=on -D PKG_RIGID=on -D PKG_RIGID=on -D PKG_MANYBODY=on -D PKG_BODY=on -D PKG_MOLECULE=on -D PKG_OPT=on -D PKG_KSPACE=yes -D FFT=KISS -D BUILD_MPI=yes -D BUILD_OMP=yes -D LAMMPS_MACHINE=mpi -D PKG_GPU=on -D GPU_API=cuda -D GPU_PREC=mixed -D GPU_ARCH=sm_86 -D CUDPP_OPT=yes -D BIN2C=/usr/local/cuda-12.2/bin/bin2c -D BUILD_OMP=yes -D PKG_KOKKOS=on -D Kokkos_ARCH_ZEN3=yes -D Kokkos_ARCH_AMPERE86=yes -D Kokkos_ENABLE_OPENMP=yes -D Kokkos_ENABLE_CUDA=on …/cmake

Please see the documentation:
https://docs.lammps.org/Speed_kokkos.html#running-lammps-with-the-kokkos-package
and for GPUs specifically:
https://docs.lammps.org/Speed_kokkos.html#running-on-gpus

1 Like

Thanks a lot. Successfully executed.

1 Like