Pair_style hybrid dpd srp with gpu acceleration?

LAMMPS LAMMPS General Discussion
1

After working on a cpu- cluster for a while I now have access to gpus. My Simulations use pair_style hybrid dpd srp, so for gpu use I compiled LAMMPS using

cmake
-DPKG_GPU=yes
-DGPU_API=cuda
-DGPU_ARCH=sm_72
-DGPU_PREC=double
-DPKG_USER-MISC=yes
-DPKG_MOLECULE=yes

Here the output after starting the simulation:
LAMMPS (29 Oct 2020)
OMP_NUM_THREADS environment is not set. Defaulting to 1 thread. (src/comm.cpp:94)
using 1 OpenMP thread(s) per MPI task
Reading data file …
orthogonal box = (0.0000000 0.0000000 0.0000000) to (100.00000 100.00000 100.00000)
2 by 2 by 3 MPI processor grid
reading atoms …
3000000 atoms
scanning bonds …
4 = max bonds/atom
scanning angles …
1 = max angles/atom
reading bonds …
17397 bonds
reading angles …
14559 angles
Finding 1-2 1-3 1-4 neighbors …
special bond factors lj: 0.0 0.0 0.0
special bond factors coul: 0.0 0.0 0.0
4 = max # of 1-2 neighbors
4 = max # of 1-3 neighbors
8 = max # of 1-4 neighbors
12 = max # of special neighbors
special bonds CPU = 0.130 seconds
read_data CPU = 2.738 seconds
3000000 atoms in group all
2983405 atoms in group sol
16595 atoms in group net

  • Using acceleration for dpd:
  • with 12 proc(s) per device.

Device 0: GeForce RTX 2080 Ti, 68 CUs, 8.9/11 GB, 1.5 GHZ (Double Precision)

Initializing Device and compiling on process 0…Done.
Initializing Device 0 on core 0…Done.
Initializing Device 0 on core 1…Done.
Initializing Device 0 on core 2…Done.
Initializing Device 0 on core 3…Done.
Initializing Device 0 on core 4…Done.
Initializing Device 0 on core 5…Done.
Initializing Device 0 on core 6…Done.
Initializing Device 0 on core 7…Done.
Initializing Device 0 on core 8…Done.
Initializing Device 0 on core 9…Done.
Initializing Device 0 on core 10…Done.
Initializing Device 0 on core 11…Done.

ERROR: PairSRP: Pair srp requires newton pair on (src/USER-MISC/pair_srp.cpp:462)
Last command: run 1000
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_device.h’ in line 135.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_device.h’ in line 135.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_device.h’ in line 135.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_device.h’ in line 135.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_device.h’ in line 135.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_device.h’ in line 135.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_device.h’ in line 135.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_device.h’ in line 135.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_device.h’ in line 135.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_device.h’ in line 135.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 98.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_timer.h’ in line 99.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_device.h’ in line 135.
Cuda driver error 4 in call at file ‘/home/user/lammps-gpu/lib/gpu/geryon/nvd_device.h’ in line 135.

Primary job terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.

mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

Process name: [[20121,1],5]
Exit code: 1

Of course that is kind of what I expected, I already read in some other topics in this forum that gpu force kernels don’t use newton’s 3rd law, i.e. compute each pair interaction twice to avoid a race
condition when updating forces and the srr pair_style seems to depend on it. Is there any way at all to run hybrid dpd srp with gpu acceleration?

2

Not at the moment. There are efforts to alleviate this, but please consider the following:

  • you are using a consumer grade Nvidia GPU (GeForce) which have a much more limited capacity for double precision floating point operations than the dedicated accelerator devices (formerly called Tesla). that will limit the amount of acceleration available
  • you have a ratio of 12 CPU cores per GPU, which will seriously reduce the possible impact of the GPU (even if there was no overhead, you’d be only be getting about 8% of the acceleration compared to a 1:1 ratio)
  • the dpd pair style has rather low computational intensity, thus its potential for acceleration is lower than, say for pair style tersoff
  • when using a hybrid pair style, you also need to use the neighbor list build on the CPU which further limits your potential for acceleration

In summary, it is quite possible that if you were able to make it run, the GPU would not accelerate your calculation significantly or not at all or even decelerate your simulation.

As an alternative you may want to look into getting the Intel OneAPI compiler and include the INTEL package and then obtain some speedup for the dpd pair style on the CPU from vectorization (key is using the Intel compiler instead of the GNU or Clang compiler, not so much whether you have an Intel CPU).

1 Like