A few KOKKOS development questions

LAMMPS LAMMPS Development
8

Wow good questions:

  1. We need a way to call functions from Kokkos pair styles or fixes in the CommKokkos class without having to cast to every individual style, so we inherit from a common base class. If a fix doesn’t implement communication routines then it doesn’t need to inherit from KokkosBase. These functions are all called from the host CPU, so there is no issue with vtables on the GPU. If you try that with device functions then there are issues with the vtable.

  2. That comment is outdated, it just needs to use Kokkos::deep_copy(d_view,0.0) to zero out the values. This is done by default when a view is reallocated, but reallocating every timestep is expensive, so using the deep_copy would be better.

  3. It is not unified memory, but an “unmanaged” view, which doesn’t do reference counting leading to reduced overhead when copying the view.

  4. You can use -pk kokkos comm/exchange device to force it on on the CPU (e.g. for Serial or OpenMP), see package command — LAMMPS documentation.

  5. Sendlist is a list of atoms that are migrating off the current rank to another rank. Copylist is a list of particles that are copied from one place in the view to another to backfill “holes” left by particles that migrated away, so the array remains compact.

  6. It has to do with the different passes in the unpack. Sometimes you only have 1 pass, sometimes you have more than 1, depending on the neighboring proc grid. “Extra” data is carried by fixes so that needs to be communicated too.

  7. I think I just duplicated the original CPU code which is also inconsistent. I don’t think the static casts are strictly necessary, it will implicit cast. More correct would be to use a union with ubuf as is done in the atom_vec styles, so that no precision is lost for converting a huge 64-bit integer to a double.