Error of LAMMPS GPU

Dear all,

I have successful installed and run the lammps without GPU package. But, unfortunately, there is a error appeared when I enable the GPU package. The detail of my installed process is taken as follow:

  1. Install the cuda driver
    rpm -i cuda-repo-rhel7-8-0-local-ga2-8.0.61-1.x86_64.rpm
    yum clean all
    yum install cuda

  2. Install the openmpi, fftw package

  3. Install the lammps
    modified the file of …/lammps/lib/gpu/Makefile.linux
    %make -f Makefile.linux
    modified the file of …/lammps/src/MAKE/Makefile.gpu
    Enter folder of …/lammps/src/
    %make yes-gpu yes-…
    %make gpu

After these steps, the ‘lmp_gpu’ file was successfully build.

But, when I run the command ‘mpirun -np 1 lmp -sf gpu -pk gpu 1 < in.melt’, the error was appeared.

LAMMPS (17 Nov 2016)
Cuda driver error 999 in call at file ‘geryon/nvd_device.h’ in line 273.

Hi Muhan,

can you run nvc_get_devices in lib/gpu and post the output? Also, can you post the modified Makefile.linux?

Best,
-Trung

Dear Trung Nguyen,

Thank you very much for your help. According to your suggestion, I have run the command ‘./nvc_get-devices’. The following results were displayed.

------------------------------------------------------------------------------------

Found 1 platform(s).
Using platform: NVIDIA Corporation NVIDIA CUDA Driver
CUDA Driver Version: 8.0

Device 0: “GeForce GTX TITAN X”
Type of device: GPU
Compute capability: 5.2
Double precision support: Yes
Total amount of global memory: 11.9202 GB
Number of compute units/multiprocessors: 24
Number of cores: 4608
Total amount of constant memory: 65536 bytes
Total amount of local/shared memory per block: 49152 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per block: 1024
Maximum group size (# of threads per block) 1024 x 1024 x 64
Maximum item sizes (# threads for each dim) 2147483647 x 65535 x 65535
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Clock rate: 1.2155 GHz
Run time limit on kernels: Yes
Integrated: No
Support host page-locked memory mapping: Yes
Compute mode: Default
Concurrent kernel execution: Yes
Device has ECC support enabled: No

------------------------------------------------------------------------------------

And my modify Makefile.linux is shown as follow:

------------------------------------------------------------------------------------

/* ----------------------------------------------------------------------

Generic Linux Makefile for CUDA

- Change CUDA_ARCH for your GPU

------------------------------------------------------------------------- */

which file will be copied to Makefile.lammps

EXTRAMAKE = Makefile.lammps.standard

ifeq ($(CUDA_HOME),)
CUDA_HOME = /usr/local/cuda
endif

NVCC = nvcc

Tesla CUDA

CUDA_ARCH = -arch=sm_52

newer CUDA

#CUDA_ARCH = -arch=sm_13

older CUDA

#CUDA_ARCH = -arch=sm_10 -DCUDA_PRE_THREE
#CUDA_ARCH = -arch=sm_35

this setting should match LAMMPS Makefile

one of LAMMPS_SMALLBIG (default), LAMMPS_BIGBIG and LAMMPS_SMALLSMALL

LMP_INC = -DLAMMPS_SMALLBIG

precision for GPU calculations

-D_SINGLE_SINGLE # Single precision for all calculations

-D_DOUBLE_DOUBLE # Double precision for all calculations

-D_SINGLE_DOUBLE # Accumulation of forces, etc. in double

CUDA_PRECISION = -D_SINGLE_DOUBLE

CUDA_INCLUDE = -I$(CUDA_HOME)/include
CUDA_LIB = -L$(CUDA_HOME)/lib64
CUDA_OPTS = -DUNIX -O3 -Xptxas -v --use_fast_math $(LMP_INC)

CUDR_CPP = mpic++ -DMPI_GERYON -DUCL_NO_EXIT -DMPICH_IGNORE_CXX_SEEK -DOMPI_SKIP_MPICXX=1 -fPIC
CUDR_OPTS = -O2 $(LMP_INC) # -xHost -no-prec-div -ansi-alias

BIN_DIR = ./
OBJ_DIR = ./
LIB_DIR = ./
AR = ar
BSH = /bin/sh

CUDPP_OPT = -DUSE_CUDPP -Icudpp_mini

include Nvidia.makefile

------------------------------------------------------------------------------------

That’s all. Thank you for your and all lammps users time. Looking forward to your reply.

Many thanks,

Muhan Wang

Hi Muhan,

the output of nvc_get_devices and and Makefile.linux look normal to me. However, I couldn’t reproduce the issue with the 17Nov16 version (the latest stable version at the moment) using the command you used: mpirun -np 1 …/…/src/lmp_mpi -sf gpu -pk gpu 1 < in.melt, on a GTX 950 card (also sm_52) with CUDA 8.0.

What packages have you installed in addition to the GPU package, i.e. make yes-…? also, were you using examples/melt/in.melt?

Cheers,
-Trung

Dear Trung,

Thank you very much. It is so kind of you.

I enable the package using the following command line:

$make yes-all
$make no-lib
$make no-user
$make yes-gpu

I was using the examples/melt/in.melt and the latest stable version was employed. Could these messages bring some inspiration?

Many thanks,

Muhan

Dear all,

My problem was successfully solved. Thank you very much for Trung Nguyen’s help.

Best regards,

Muhan Wang

Hi Muhan,

actually, you have solved the problem yourself. Would you mind sharing how it got solved?

Cheers,
-Trung