Errors in using reax/c/kk

LAMMPS LAMMPS Mailing List Mirror
1

Hi All,

I am trying to run reax/c/kk and I am running into a couple of issues . I am using the release version of July 30 2016 and KOKKOS device is openMP.

Issue 1 : When I run the charge equilibration with kokkos, i.e. fix reax/c/kk followed by fix/qeq/reax/kk, I get an error stating that I need to use fix/qeq. This issue gets resolved if I just run charge equilibration without kk, i.e. reax/c/kk followed by fix/qeq/reax .

Issue 2 : I get the following warning message

WARNING: Fixes cannot send data in Kokkos communication, switching to classic communication (…/comm_kokkos.cpp:365). This is an issue because my computation is much slower, at least by a factor of 3 on the same number of processors if i just user user-omp.

The 2 fixes I am using are fix qeq/reax and fix nve. This issue persists when I use fix qeq/reax and fix nvt/kk.

Any help or ideas would be appreciated.

Thanks

Savio

2

Hi All,

I am trying to run reax/c/kk and I am running into a couple of issues . I am
using the release version of July 30 2016 and KOKKOS device is openMP.

please note that reax/c/kk is *very* new and thus some rough edges are
to be expected.

Issue 1 : When I run the charge equilibration with kokkos, i.e. fix
reax/c/kk followed by fix/qeq/reax/kk, I get an error stating that I need to
use fix/qeq. This issue gets resolved if I just run charge equilibration
without kk, i.e. reax/c/kk followed by fix/qeq/reax .

Issue 2 : I get the following warning message

WARNING: Fixes cannot send data in Kokkos communication, switching to
classic communication (../comm_kokkos.cpp:365). This is an issue because my
computation is much slower, at least by a factor of 3 on the same number of
processors if i just user user-omp.

there is no support for reax/c in USER-OMP

The 2 fixes I am using are fix qeq/reax and fix nve. This issue persists
when I use fix qeq/reax and fix nvt/kk.

it would be best to first run tests with the examples bundled with
LAMMPS, enabling KOKKOS support via command line.

Any help or ideas would be appreciated.

please see: http://lammps.sandia.gov/bug.html

thus you would want to upgrade to the very latest patchlevel of LAMMPS
to take advantage of the recent bugfixes and updates and then try
again.

axel.

3

Hi All,

I am trying to run reax/c/kk and I am running into a couple of issues . I am using the release version of July 30 2016 and KOKKOS device is openMP.

Issue 1 : When I run the charge equilibration with kokkos, i.e. fix reax/c/kk followed by fix/qeq/reax/kk, I get an error stating that I need to use fix/qeq. This issue gets resolved if I just run charge equilibration without kk, i.e. reax/c/kk followed by fix/qeq/reax .

You should first use “pair_style reax/c/kk” and then “fix qeq/reax/kk”. Not the other way around.

Issue 2 : I get the following warning message

WARNING: Fixes cannot send data in Kokkos communication, switching to classic communication (…/comm_kokkos.cpp:365). This is an issue because my computation is much slower, at least by a factor of 3 on the same number of processors if i just user user-omp.

What kind of computer are you running the reax/c/kk on? Conventional multi-core or many-core accelerators (Phi or GPU)? Depending on the architecture, using full or half neighbor lists has a tremendous effect to the communication time. This is all documented on LAMMPS doc page.

I would follow Axel’s advice and run a reaxff example first.

Thanks,
Ray

4

Ray and Axel,

Thanks for your response. I tried all the methods that you suggested and nothing really changed. I still get the following warning.

WARNING: Fixes cannot send data in Kokkos communication, switching to classic communication (…/comm_kokkos.cpp:365)

I am testing on RDX example in LAMMPS. I did not make any changes to that input file or the parameters file. I have also applied all the patches available to date. I am running kokkos through the command line.

export OMP_NUM_THREADS=32 #threads for openmp
mpirun -np 1 lmp_mpi -k on t 32 -sf kk -in in.RDX

To build kokkos and reax/c , I did the following

make yes-user-reaxc

make yes-kokkos

make mpi KOKKOS_DEVICES=OpenMP

Would kokkos have an issue with what MPI library I use ?

Thanks

Savio

5

Ray and Axel,

Thanks for your response. I tried all the methods that you suggested and nothing really changed. I still get the following warning.

WARNING: Fixes cannot send data in Kokkos communication, switching to classic communication (…/comm_kokkos.cpp:365)

That is just a warning telling you that all communications required by the fix (nvt/kk and qeq/reax/kk) are done in a classical way. There is nothing you can do to eliminate this warning – unless you edit the source code. You can disregard this.

I am testing on RDX example in LAMMPS. I did not make any changes to that input file or the parameters file. I have also applied all the patches available to date. I am running kokkos through the command line.

export OMP_NUM_THREADS=32 #threads for openmp
mpirun -np 1 lmp_mpi -k on t 32 -sf kk -in in.RDX

Use “-pk kokkos ” after the “-sf kk” to use different neigh, comm, newton settings to experiment how you can achieve the best performance.

Hope this helps.

Ray

6

There is nothing you can do to eliminate this warning – unless you edit the source code. You can disregard this.

Actually you can use the “comm no” option in the Kokkos package to eliminate this warning. The warning means that your communication routines are running in non-threaded mode. On a GPU it also means that there is memory transfer between host and device. But on a CPU you should use “comm no” as it is usually faster.

This is an issue because my computation is much slower, at least by a factor of 3 on the same number of processors if i just user user-omp

It looks like you are using 32 OpenMP threads x 1 MPI. This most likely isn’t a good idea. MPI-only will almost always beat OpenMP on a CPU unless you run into communication bottlenecks. I would suggest first trying with 32 MPI x 1 OpenMP thread with Kokkos and see if the timings are comparable to without Kokkos.

Stan

7

Thanks everyone for the help. Stan’s suggestion solved the problem.

For reference for other users, I used comm no instead of comm host ; the warning message went away and I get very similar run rate as the omp user package.

8

Hello,

I have updated LAMMPS to newest version and I just wanted to re-do the calculations (because I couldnt use read_restart files from old version) , and what is strange that I got this mistake :

compute myRDF all rdf 100 4 14 3 14 14 15
fix Rdf1 all ave/time 100 100 10000 c_myRDF file tmp.rdf mode vector ave running
ERROR: Fix ave/time compute does not calculate a vector (…/fix_ave_time.cpp:166)

and I could calculate this without problem in old version of LAMMPS , I dont know whats the problem. The only difference in input file is that I have added extra/atom/bonds atoms angles in read_data command but I dont think it should be problem. And I changed pair style from coul/cut to coul/long and kspace pppm , but I dont see why this should be a problem ?

I have atached me input file in email, greetings .

BTMA.in (1.84 KB)

9

savio,

please don't write a new e-mail by replying to AND QUOTING an old one.
you are just sending around stuff to (literally) thousands of
lammps-users subscribers (over 2500 last time i checked), that they
have seen before and has nothing to do with your inquiry. this is
wasteful and annoying. thanks in advance for your consideration.

  Hello,

  I have updated LAMMPS to newest version and I just wanted to re-do the
calculations (because I couldnt use read_restart files from old version) ,
and what is strange that I got this mistake :

compute myRDF all rdf 100 4 14 3 14 14 15
   fix Rdf1 all ave/time 100 100 10000 c_myRDF file tmp.rdf mode vector ave
running
ERROR: Fix ave/time compute does not calculate a vector
(../fix_ave_time.cpp:166)

    and I could calculate this without problem in old version of LAMMPS , I
dont know whats the problem. The only difference in input file is that I

LAMMPS is improved all the time and some of these improvements cannot
be done without breaking backward compatibility.
when installing a new version, it is advisable to go to the page with
the list of changes for the current calendar year:

http://lammps.sandia.gov/bug.html

(and the corresponding pages for previous years, if needed) and check
out what was changes, especially changes flagged with
"BACKWARD COMPATIBILITY:"

if you do so and look at the entry for the 1 Aug 2016 patch, you will
see the explanation. it changes the interface for fix ave/time to
handle array data in a more consistent and flexible manner. so in
cases, where you could use the id of the compute or fix in the past,
you now has to use it with a wildcard.

you should also study the corresponding pages in the manual for more details.

have added extra/atom/bonds atoms angles in read_data command but I dont
think it should be problem. And I changed pair style from coul/cut to
coul/long and kspace pppm , but I dont see why this should be a problem ?

none of this is a problem, but the line that is giving the error. the
fact, that it works in an older version is no reason why it should
work in a new version for the reasons given above.

axel.