GPU error on Ubuntu

Dear all, I am getting the following error when I try to run lammps on gpu:

Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH

I have compiled libgpu.a and lammps using makefiles provided in the link below, and I was successful with it. But I get the error above when I try to run with my input file. I have searched around, but I haven’t been able to fix my problem. My node is with GTX Titan and 32 CPU cores. I hope somebody can help me fix the problem.
http://lammps.sandia.gov/threads/msg36437.html

Hi Kemal,

did you specify -arch=sm_35 for CUDA_ARCH in the Makefile you used to build libgpu.a? If not, please use that parameter, “make -f Makefile.you_chose clean” and rebuild libgpu.a.

If the rebuild is successful, you then execute the generated binary ./nvc_get_devices in lib/gpu to see if the GPU is detected correctly. Then try the example input scripts in examples/gpu or bench/GPU. If the error persists, please post the whole screen output.

-Trung

Hi Trung,
I did specify -arch=sm_35, and ./nvc_get_devices does detect gpu correctly. I tried running in.gpu.melt.2.5 and it ran successfully. But when I run my input file,the error changed now. Here is the print:

kemalramic@…4882…:~/Desktop/LAMMPS/Cuda$ mpirun -n 32 lmp_ubuntu_gpu_cud
polyethylene_2chains
LAMMPS (1 Feb 2014)
Reading data file …
orthogonal box = (0 0 0) to (7.41 4.93 2.558)
4 by 4 by 2 MPI processor grid
reading atoms …
12 atoms
Replicating atoms …
orthogonal box = (0 0 0) to (29.64 19.72 38.37)
4 by 2 by 4 MPI processor grid
2880 atoms
ERROR: Pair style AIREBO requires newton pair on (…/pair_airebo.cpp:216)
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Attempting to use an MPI routine after finalizing MPICH
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH
Cuda driver error 4 in call at file ‘geryon/nvd_device.h’ in line 124.
Attempting to use an MPI routine after finalizing MPICH

And also, I attached my input file.

in.polyethylene_2chains (528 Bytes)

Hi Trung,
I did specify -arch=sm_35, and ./nvc_get_devices does detect gpu correctly. I tried running in.gpu.melt.2.5 and it ran successfully. But when I run my input file,the error changed now. Here is the print:
kemalramic@…4882…:~/Desktop/LAMMPS/Cuda$ mpirun -n 32 lmp_ubuntu_gpu_cud
polyethylene_2chains
LAMMPS (1 Feb 2014)
Reading data file …
orthogonal box = (0 0 0) to (7.41 4.93 2.558)
4 by 4 by 2 MPI processor grid
reading atoms …
12 atoms
Replicating atoms …
orthogonal box = (0 0 0) to (29.64 19.72 38.37)
4 by 2 by 4 MPI processor grid
2880 atoms
ERROR: Pair style AIREBO requires newton pair on (…/pair_airebo.cpp:216)

This error is self-explanatory.
There is no GPU accelerated airebo style, too. Check the manual.

This might sound dumb, but what are my options then? I just keep running it on CPU?

This might sound dumb, but what are my options then? I just keep running it on CPU?

Or you could program the gpu code yourself.

I am afraid my coding skills are far from being able to do so :frowning: But, thank you for the help.

Also, with so few atoms, you don’t get much acceleration. And running with 32 MPI tasks is plain insanity. Even on the CPU you’ll have a hard time getting good parallel efficiency.

well, I think if you keep practice you will become skillful… implementing a GPU accelerated airebo for LAMMPS would benefit both your coding skill and your study…