Erratic behavior of USER-INTEL package on Intel 5120D co-processor

LAMMPS LAMMPS Mailing List Mirror
1

Hi,
I think I need help! I know such questions are discouraged but i would like to invoke rule 10 of mailing list rules on lammps website “Finally, if in doubt, just post your question. You won’t be the first to have posted a strange message or a question with an obvious answer.”!!

Here we go:
First thing first, i tried compiling latest lammps version (lammps-lammps-icms branch) yesterday using compiler 15.0.1 – same error/bug persists

I tried cross compiling using version 14.0.1 with lammps version 14 may 16
I get following output:
icpc: command line warning #10006: ignoring unknown option ‘-qopenmp’
icpc: command line warning #10006: ignoring unknown option ‘-qoverride-limits’
icpc: command line warning #10006: ignoring unknown option ‘-qoffload-option,mic,compiler,-fp-model fast=2 -mGLOB_default_function_attrs=“gather_scatter_loop_unroll=4”’

Turns out -qflag is deprecated (?) in favor of -flag notation. so i removed ‘q’ from from and now my make file consist of -openmp -override-limits etc
After which above warning dissapeared but i got a new error:
…/fix_intel.cpp(175): error: identifier “omp_get_num_procs” is undefined
offload_cores = omp_get_num_procs();
quick google search yields this result : https://github.com/HPAC/lammps-tersoff-vector/issues/1 [last 2 entries]
which says “This last error message originated outside of my code, so I did contact the original author.
His fix has landed in the latest LAMMPS version, so compilation with ICC 14.0 should be possible now.”

After which I again started compiling lammps-lammps-icms branch using 14.0.1 compilers. Luckily all goes smooth until the very end where i get following error (at the linking stage i presume?)

ld: skipping incompatible /mnt/lustre/ipc3/ipcamit/intel_composer/mic/lib/libmpigc4.so when searching for -lmpigc4
ld: skipping incompatible /mnt/lustre/ipc3/ipcamit/intel_composer/mic/lib/libmpigc4.a when searching for -lmpigc4
ld: cannot find -lmpigc4
ld: skipping incompatible /mnt/lustre/ipc3/ipcamit/intel_composer/mic/lib/libmpigf.so when searching for -lmpigf
ld: skipping incompatible /mnt/lustre/ipc3/ipcamit/intel_composer/mic/lib/libmpigf.a when searching for -lmpigf
ld: cannot find -lmpigf
ld: skipping incompatible /mnt/lustre/ipc3/ipcamit/intel_composer/mic/lib/libmpi_dbg_mt.so when searching for -lmpi_dbg_mt
ld: skipping incompatible /mnt/lustre/ipc3/ipcamit/intel_composer/mic/lib/libmpi_dbg_mt.a when searching for -lmpi_dbg_mt
ld: cannot find -lmpi_dbg_mt
ld: skipping incompatible /mnt/lustre/ipc3/ipcamit/intel_composer/mic/lib/libmpigi.a when searching for -lmpigi
ld: cannot find -lmpigi

Now I have no clue. All the flags were default, am not sure why default mic libraries are conflicting here. Any hints?

Also I am not sure if cross compiling will help my case. As would i not require systems default mpi libraries if i want to run job on multiple nodes?

Again I understand such questions are tangential to LAMMPS mailing list at best and i apologize in advance but I can’t get any clue on what to do next. My only hope now is to wait till System Admin updates compilers again and check if that sorts out the problem :frowning:

Other details:
I unloaded all unnecessary default cray modules: module unload intel cray-libsci PrgEnv-intel cray-mpich cray-shmem fftw craype
I downloaded student’s version of compilers icc and ifort 14.0.1 which is actually composer 2013 sp1.1.106, installed it for user (non sudo) in one of login node. along with intel mpi version 4.1 and sourced both of them as said.
current CCFLAGS look like:
MIC_OPT = -offload-option,mic,compiler,"-fp-model fast=2 -mGLOB_default_function_attrs=“gather_scatter_loop_unroll=4"”
CCFLAGS = -g -O3 -openmp -DLMP_INTEL_OFFLOAD -DLAMMPS_MEMALIGN=64
-fno-alias -ansi-alias -restrict
-override-limits $(MIC_OPT)
current intel_coprocessor file Link flags look like:
LINKFLAGS = -g -O3 -openmp -offload

Thank you

2

Hi Amit,

Sorry that this is still an issue for you.

Turns out -qflag is deprecated (?) in favor of -flag notation. so i removed ‘q’ from from and now my make file consist of -openmp -override-limits etc

In the older compilers, you did not need -q in front of options starting with an ‘o’. That is deprecated and now it is required.

Also I am not sure if cross compiling will help my case. As would i not require systems default mpi libraries if i want to run job on multiple nodes?

Does the Cray system you are using support Intel MPI? If so, you can either copy the libraries to a directory accessible from the compute nodes and set LD_LIBRARY_PATH or try to get a static build (this is generally preferred on a lot of the Cray machines) [see the static options available in the intel documentation; there are multiple]. However, your approach of installing the compiler locally and building there seems best; although, you might need to use the Cray environment for whatever MPI libraries they support on the system. The modules normally just set environment variables, so you might be able to keep your modules loaded, then source the compiler environment script for the compiler you installed, and verify that ‘CC --version’ gives you the correct result. Otherwise, I would ask the admins how to switch the Cray wrapper to use your installed compilers or try to figure it out from the module information.

After which I again started compiling lammps-lammps-icms branch using 14.0.1 compilers. Luckily all goes smooth until the very end where i get following error (at the linking stage i presume?)

These libraries should be in a lib64 directory in the installation directory; they are either not installed as expected or the environment is not setup as expected so that the linker will search that directory (LD_LIBRARY_PATH). I would try above instead and get help from sys admin.

If installing your own compiler, might as well use a more recent compiler (e.g. 2016).

Hope that helps. - Mike