How to dynamically generate granular atoms with KOKKOS enabled?

LAMMPS LAMMPS Beginners
1

Hello,

I’m trying to simulate a set of granular particles falling under gravity into a box with walls. I see that the following commands support KOKKOS acceleration:
• atom_style sphere/kk
• pair_style gran/hooke/history/kk
• fix all nve/sphere/kk

However, I’m stuck on the step where I need to dynamically generate new granular atoms.
In particular, fix pour (because it is from a GRANULAR package?) doesn’t seem to be KOKKOS-compatible.

Question: Is there an alternate way to generate new granular atoms on-the-fly while using the KOKKOS package?

P.S. Based on the documentation, KOKKOS appears to be the only package that supports granular-style atoms (sphere, gran/hooke/history, etc.) supporing GPU acceleration.

Any advice or suggested workarounds would be greatly appreciated.

Regards,

2

You can use fix pour with KOKKOS (or pretty much all other non-KOKKOS fixes).

The reason why many fixes exist in a KOKKOS version is not so much in speeding up the computation in the fix, but to avoid data transfer from GPU to CPU and back. So when using non-KOKKOS fixes in KOKKOS mode, you may see warnings about additional data transfers.

However, when adding new particles, you need to update the data structures on the CPU as well.

That said, the benefits from GPU acceleration for DEM are rather limited in general.

3

Thank you akohlmey for your fast and kind answer,

However I got error when using “fix pour”

==== Error ====

$ mpirun -np 10 ./lmp_KIG -kokkos on gpus 1 -in TK.in
LAMMPS (29 Aug 2024 - Maintenance - stable_29Aug2024_update1-26-g89e442dac1-modified)
KOKKOS mode with Kokkos version 4.3.1 is enabled (src/KOKKOS/kokkos.cpp:72)
  will use up to 1 GPU(s) per node
Created orthogonal box = (-0.033 -0.033 -0.033) to (0.033 0.033 0.1)
  1 by 1 by 10 MPI processor grid
  1 by 1 by 5 core grid within node
ERROR: Cannot yet use fix pour with the KOKKOS package (src/GRANULAR/fix_pour.cpp:59)
Last command: fix              gen_particles all pour 1000 1 29494 region gen_area                  diam poly 6 0.0005 0.2 0.0006 0.2 0.0007 0.2 0.0008 0.2 0.0009 0.1 0.001 0.1
--------------------------------------------------------------------------

==== End of Error ====

==== Information of LAMMPS ====

$ ./lmp_KIG -help

Large-scale Atomic/Molecular Massively Parallel Simulator - 29 Aug 2024 - Maintenance
Git info (maintenance / stable_29Aug2024_update1-26-g89e442dac1-modified)

OS: Linux "Ubuntu 24.04.1 LTS" 6.8.0-51-generic x86_64

Compiler: GNU C++ 13.3.0 with OpenMP not enabled
C++ standard: C++17
MPI v3.1: Open MPI v5.0.6, package: Open MPI hp-z8@HP-Z8 Distribution, ident: 5.0.6, repo rev: v5.0.6, Nov 15, 2024

Accelerator configuration:

KOKKOS package API: CUDA Serial
KOKKOS package precision: double
Kokkos library version: 4.3.1


Installed packages:

GRANULAR KOKKOS VTK

==== End of Information ====

4

I wan’t aware of that. Apparently, there is extra work needed for KOKKOS when adding particles during a run. Perhaps @stamoor can provide some details, KOKKOS support is his specialty.

An alternate approach would be to break down your run into many small chunks in a loop and then add new particles and set their properties using the create_atoms and set commands.

Or you could check how much slower it is to run without KOKKOS and use MPI parallelization.

5

Thank you. Appreciate.