Inconsistent short range energy with g++ compiler and intel compiler

LAMMPS LAMMPS Mailing List Mirror
1

Dear Lammps users,

I encountered a problem when I simulate a carbon nanotube with a COO- functioncal group. I have two lammps executables (lmp_g++_openmpi and lmp_intel_cpu_openmpi) compiled with g++ and Intel compiler, respectively. The Lammps version is LAMMPS (7 Aug 2019). In the example file, there are two C-O bonds and one C-C’ bond, C’ is the aromatic carbon atom connecting the COO carbon. The structure is like following and the values in the parentheses are the partial charges chosen arbitrarily just for test.
O (-0.4)
/
C’(0.1) – C (0.7)

O (-0.4)
I am using lj/cut/coul/long to account for the short range interactions. The problem is that the two executables give different short range E_vdw and E_coul, while the long range E_long, and bond and angle energies are consistent. I run 0 step with the attached data file and in file, I obtain following results:

For lmp_g++_openmpi:
Step CPU Temp E_vdwl E_coul E_long E_pair E_bond E_angle E_impro TotEng Press PotEng

0 0 0 291.6445648 24.91085569 -24.92351992 291.6319006 48.533504 3.092334748e-27 0 340.1654046 150.8209546 340.1654046

For lmp_intel_cpu_openmpi:
Step CPU Temp E_vdwl E_coul E_long E_pair E_bond E_angle E_impro TotEng Press PotEng
0 0 0 310.4625205 19.3011671 -24.92351999 304.8401676 48.533504 3.092334748e-27 0 353.3736716 162.5949633 353.3736716

I have tested systems with only water and ions (e.g. Na+, K+), the energies are consistent. Here, if I remove the C’-C bond, I get consistent results:

For lmp_g++_openmpi:
Step CPU Temp E_vdwl E_coul E_long E_pair E_bond E_angle E_impro TotEng Press PotEng
0 0 0 12673.20152 28.96379157 -24.92351992 12677.24179 2.758883244e-26 3.092334748e-27 0 12677.24179 7492.570331 12677.24179

For lmp_intel_cpu_openmpi:
Step CPU Temp E_vdwl E_coul E_long E_pair E_bond E_angle E_impro TotEng Press PotEng
0 0 0 12673.20152 28.96379157 -24.92351999 12677.24179 2.758883244e-26 3.092334748e-27 0 12677.24179 7492.570331 12677.24179

Furthermore, I changed the special_bonds parameters 0.0 0.0 0.5 to 0.0 1.0 0.5 to include the 1-3 interaction, the results are also consistent. It seems that the treatment of 1-3 interaction may be different with the two compilers.

I found the simulations are very stable running with lmp_g++_openmpi, while usually crashed down using lmp_intel_cpu_openmpi due to the functional groups are unstable at certain time steps.

I would like to know your thoughts on this issue. Attached please find the input files for simulation and makefiles for the two excutables.

Thanks and best regards,

Xiang

data.cnt.coo.lammps (48.2 KB)

in.cnt.coo.eq (1.4 KB)

Makefile.intel_cpu_openmpi (3.85 KB)

Makefile.g++_openmpi (3.1 KB)

2

there is some crucial information missing here:

  • what exact version of the intel compiler were you using?
  • what is the exact command line (e.g. are you using USER-INTEL package through the -sf intel flag?)?

You may want to try the latest LAMMPS patch version (9 Jan 2020) as it includes some updates and corrections for Makefiles in the USER-INTEL package and related files that are supposed to work around known problems with some recent versions of the intel compilers.

Axel.

3

Hi Axel,

My intel compiler is Version 19.0.5.281. The command line I am using is “mpirun -np xx lmp_intel_cpu_openmpi < in.cnt.coo.eq”.

I will try the latest version Lammps patch verion as you suggested. Do you mean that this problem only occurs with recent verions of intel compilers, while not for old versions of intel compilers? Besides, in my case, does it mean that I can trust more the results of g++ compiled lammps?

Thanks and best,

Xiang

4

Hi Axel,

My intel compiler is Version 19.0.5.281. The command line I am using is “mpirun -np xx lmp_intel_cpu_openmpi < in.cnt.coo.eq”.

I will try the latest version Lammps patch verion as you suggested. Do you mean that this problem only occurs with recent verions of intel compilers, while not for old versions of intel compilers?

yes. version 19 has known problems generating correct code when certain compiler flags for aggressive optimization are used. this primarily affects the USER-INTEL package, but could apply to others as well. so you should compare the corresponding makefile in the patch release with what you are doing. There are also a few code changes in the code of the USER-INTEL package, but that doesn’t apply in your case.

Besides, in my case, does it mean that I can trust more the results of g++ compiled lammps?

yes. i get the same results as your g++ executable with the clang 9.0 and gcc 9.2 compiler on my desktop machine.

here is the output of a clang++ compiled executable with out enabling USER-INTEL
Step CPU Temp E_vdwl E_coul E_long E_pair E_bond E_angle E_impro TotEng Press PotEng
0 0 0 291.6445648 24.91085566 -24.92351992 291.6319006 48.533504 3.092334748e-27 0 340.1654046 150.8209546 340.1654046

and here is the output of the same executable with -sf intel and using USER-INTEL in mixed precision (for additional vectorization speedup). so you see there are some small differences due to some of the calculation being done with single precision, but it is essentially still comparable and not deviating as much as your intel compiler output.

Step CPU Temp E_vdwl E_coul E_long E_pair E_bond E_angle E_impro TotEng Press PotEng
0 0 0 291.6443799 24.91085744 -24.92402432 291.631213 48.53344345 1.17950105e-12 0 340.1646564 150.8209401 340.1646564

if you don’t use the USER-INTEL package, there should be next to no difference in performance between an up-to-date g++ compiler or the intel compilers. the intel compilers excel in vectorization which is most prominently available (due to significant restructuring and rewriting of the code) in the USER-INTEL package. so if you want to benefit from the extra complication of using the intel compilers, then you should benchmark and compare with the USER-INTEL package code included and enabled.

axel.

5

Thanks Alex for your help. My problem is solved.

Best,

Xiang