No difference in practice, but sometimes “template” is required by c++ and sometimes it is not (I think it depends on if it is a member of that class or another class). If your code won’t compile then try adding it
It has to do with overlapping host/device in verlet_kokkos.cpp. For this reason, all topology styles (bond, angle, etc.) must set DATAMASK_READ, DATAMASK_MODIFY in the constructor and must not use atomKK->sync/modified. This is a gotcha that needed to be better documented.
Fences are only needed in a few special places. So almost always: don’t add any fences to your Kokkos code (it already fences after every kernel launch). I believe DeviceType().fence() only fences that DeviceType, while Kokkos::fence() fences everything (is stronger).
Let us know if you have specific questions or need more info.
(5) what is the purpose of c_x in pair styles ? i cant find anywhere it’s actually used. it’s only declared in .h and initialized in .cpp. compiler optimizes it out maybe ??
(5) why do some KOKKOS fix classes also inherit from KokkosBase (eg. FixShakeKokkos) while others (eg. FixGravityKokkos) do not ? only difference i see are the methods pack_forward_comm_fix_kokkos, unpack_forward_comm_fix_kokkos, pack_exchange_kokkos, unpack_exchange_kokkos.
how is this multiple inheritance not causing problems with vtables on cuda ?
(6) what is a zero functor ?
//if(k_eatom.extent(0)<maxeatom) { // won't work without adding zero functor
(7) why is cuda unified memory being used in FixQEqReaxFFKokkos<DeviceType>::pack_exchange_kokkos() ?
(8) how to test pack_exchange_kokkos() ? when does it get called ?? how to modify an example to force it to happen ???
(9) in pack_exchange_kokkos(), what’s the difference between k_exchange_sendlist and k_copylist ?
(10) in unpack_exchange_kokkos(), what are the differences between nrecv, nrecv1, and nextrarecv1 ?
(11) why is there no static_cast<double> in FixShakeKokkos<DeviceType>::pack_exchange_item() but there is static_cast<tagint> in unpack_exchange_kokkos() ? in other words, why do we need to cast one way but not the other ??
We need a way to call functions from Kokkos pair styles or fixes in the CommKokkos class without having to cast to every individual style, so we inherit from a common base class. If a fix doesn’t implement communication routines then it doesn’t need to inherit from KokkosBase. These functions are all called from the host CPU, so there is no issue with vtables on the GPU. If you try that with device functions then there are issues with the vtable.
That comment is outdated, it just needs to use Kokkos::deep_copy(d_view,0.0) to zero out the values. This is done by default when a view is reallocated, but reallocating every timestep is expensive, so using the deep_copy would be better.
It is not unified memory, but an “unmanaged” view, which doesn’t do reference counting leading to reduced overhead when copying the view.
Sendlist is a list of atoms that are migrating off the current rank to another rank. Copylist is a list of particles that are copied from one place in the view to another to backfill “holes” left by particles that migrated away, so the array remains compact.
It has to do with the different passes in the unpack. Sometimes you only have 1 pass, sometimes you have more than 1, depending on the neighboring proc grid. “Extra” data is carried by fixes so that needs to be communicated too.
I think I just duplicated the original CPU code which is also inconsistent. I don’t think the static casts are strictly necessary, it will implicit cast. More correct would be to use a union with ubuf as is done in the atom_vec styles, so that no precision is lost for converting a huge 64-bit integer to a double.
