Mpirun launches multiple runs (Version 8Feb2023)

Dear all,

I compiled the latest version of LAMMPS (8Feb2023) today and wanted to run it in the same fashion I did with the version i used before, namely the 23Jun2022, like:

mpirun -np nproc lmp -in input

This worked fine for the former version, but with the new version executing the command apparently starts the input in serial nproc times. The execution of just:

mpirun -np 8 lmp

without an input specified gives for the 23Jun2022 version:

LAMMPS (23 Jun 2022 - Update 3)
WARNING: Using I/O redirection is unreliable with parallel runs. Better use -in switch to read input file. (src/lammps.cpp:530)
using 1 OpenMP thread(s) per MPI task

and for the 8Feb2023 version:

LAMMPS (8 Feb 2023)
using 1 OpenMP thread(s) per MPI task
LAMMPS (8 Feb 2023)
using 1 OpenMP thread(s) per MPI task
Total wall time: 0:00:00
LAMMPS (8 Feb 2023)
using 1 OpenMP thread(s) per MPI task
Total wall time: 0:00:00
LAMMPS (8 Feb 2023)
using 1 OpenMP thread(s) per MPI task
Total wall time: 0:00:00
LAMMPS (8 Feb 2023)
using 1 OpenMP thread(s) per MPI task
Total wall time: 0:00:00
LAMMPS (8 Feb 2023)
using 1 OpenMP thread(s) per MPI task
Total wall time: 0:00:00
LAMMPS (8 Feb 2023)
using 1 OpenMP thread(s) per MPI task
Total wall time: 0:00:00
LAMMPS (8 Feb 2023)
using 1 OpenMP thread(s) per MPI task
Total wall time: 0:00:00

To doublecheck I recompiled both versions just now in the exact same fashion, using cmake and the “most”-preset, but the strange behaviour remains. Is this a problem with the version, or might my PC be the issue?

Cheers
Jakob

This effect is usually due to one of two reasons:

  • when configuring LAMMPS, CMake could not find the MPI library automatically and has used the MPI STUBS library. This means you have a serial LAMMPS executable and starting multiple of them will run multiple individual runs since they will not try to communicate
  • when running LAMMPS you are using an mpirun/mpiexec command that is from a different MPI library (e.g. MPICH vs. OpenMPI). Then mpirun cannot communicate the rank assignment and each process will run as if it was launched without.

Please have a look at the output of lmp -h.
At some point it should print out the MPI library and version that was used to compile LAMMPS and provide the hint as to which of the two cases from above applies.

It can be something like this:

MPI v4.0: MPICH Version:	4.0.2
MPICH Release date:	Thu Apr  7 12:34:45 CDT 2022
MPICH ABI:	14:2:2

Or this:

MPI v1.0: LAMMPS MPI STUBS for LAMMPS version 8 Feb 2023
1 Like

Thanks a lot for the feedback, cmake finding the wrong MPI library was indeed the issue!

1 Like

I compiled my version of lmp using cmake after loading mpich. But after compilation the -h command shows the MPI STUBS library
Is there more to making mpich available than the command ‘module load mpich’ ??

Sorry for reviving tis topic but I am experiencing a very similar problem. After launching a simulation with mpirun -np nproc lmp -in input I get an output that looks like:

LAMMPS (10 Dec 2025)
LAMMPS (10 Dec 2025)
LAMMPS (10 Dec 2025)
LAMMPS (10 Dec 2025)
LAMMPS (10 Dec 2025)
...

LAMMPS version will appear as many times as nproc, so probably instead of separating one simulation in separate tasks, nproc independent simulation are launched. The main difference is that LAMMPS is recognising the MPI library. If I check with lmp -h:

MPI v4.1: MPICH Version:      4.2.0
MPICH Release date: Fri Feb  9 12:29:21 CST 2024
MPICH ABI:          16:0:4

which agrees with the output of mpirun --version:

HYDRA build details:
    Version:                                 4.2.0
    Release Date:                            Fri Feb  9 12:29:21 CST 2024

I even tried reinstalling LAMMPS from scratch to no avail. I should mention that I am working on Windows Subsystem for Linux (WSL2)

I cannot comment on WSL, but I know for a fact that the native pre-compiled version of LAMMPS using the MS-MPI package works for me as it should, and also compiling LAMMPS using MSVC++ and the MS-MPI SDK has not given me problems when I tested it. Since I use Windows in a virtual machine, it seemed to me overkill to also install WSL :grinning_face: .

@chrisp Here is a suggestion for some additional debugging. Take the bench/in.lj file and append the line: info config comm and report that output back.

Running with a single MPI task (mpirun -n 1 lmp -in in.lj) I get at the end of the output:

Info-Info-Info-Info-Info-Info-Info-Info-Info-Info-Info
Printed on Thu Jan 22 19:21:41 2026

LAMMPS version: 10 Dec 2025 / 20251210
Git info: release / patch_10Dec2025 / cd8851c4e75c42744e221ae3c457c6e4645f0ad5

OS information: Linux "Ubuntu 24.04.3 LTS" 6.6.87.2-microsoft-standard-WSL2 x86_64

sizeof(smallint): 32-bit
sizeof(imageint): 32-bit
sizeof(tagint):   32-bit
sizeof(bigint):   64-bit

Compiler: GNU C++ 13.3.0 with OpenMP 4.5
C++ standard: C++17
Embedded fmt library version: 10.2.0
Embedded JSON class version: 3.12.0

Active compile time flags:

-DLAMMPS_GZIP
-DLAMMPS_SMALLBIG

Available compression formats:

Extension: .gz     Command: gzip
Extension: .bz2    Command: bzip2
Extension: .xz     Command: xz
Extension: .lzma   Command: xz


Installed packages:

BROWNIAN COLLOID COLVARS DPD-BASIC EXTRA-COMMAND EXTRA-COMPUTE EXTRA-FIX
EXTRA-PAIR MOLECULE OPENMP PYTHON TALLY UEF

Communication information:
MPI library level: MPI v4.1
MPI version: MPICH Version:      4.2.0
MPICH Release date: Fri Feb  9 12:29:21 CST 2024
MPICH ABI:          16:0:4
Comm style = brick,  Comm layout = uniform
Communicate velocities for ghost atoms = no
Communication mode = single
Communication cutoff = 2.8
Nprocs = 1,   Nthreads = 1
Processor grid = 1 x 1 x 1

Info-Info-Info-Info-Info-Info-Info-Info-Info-Info-Info

Are you sure that your mpirun command is from MPICH and not OpenMPI?
Ubuntu usually installs OpenMPI as default MPI and if you’re not doing LAMMPS development and have to use valgrind a lot, I would recommend using that.

Below I attach the full output from mpirun --version. It is not the clearest thing (at least to me) but I see filepaths containing “mpich4.2.0”

HYDRA build details:
    Version:                                 4.2.0
    Release Date:                            Fri Feb  9 12:29:21 CST 2024
    CC:                              gcc -Wdate-time -D_FORTIFY_SOURCE=3 -g -O2 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -ffile-prefix-map=/build/mpich-C3oteG/mpich-4.2.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -fdebug-prefix-map=/build/mpich-C3oteG/mpich-4.2.0=/usr/src/mpich-4.2.0-5build3 -Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -Wl,-z,relro
    Configure options:                       '--with-hwloc-prefix=/usr' '--with-device=ch4:ucx' 'FFLAGS=-O2 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -ffile-prefix-map=/build/mpich-C3oteG/mpich-4.2.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -fstack-clash-protection -fcf-protection -fdebug-prefix-map=/build/mpich-C3oteG/mpich-4.2.0=/usr/src/mpich-4.2.0-5build3 -fallow-invalid-boz -fallow-argument-mismatch' '--prefix=/usr' 'CFLAGS=-g -O2 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -ffile-prefix-map=/build/mpich-C3oteG/mpich-4.2.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -fdebug-prefix-map=/build/mpich-C3oteG/mpich-4.2.0=/usr/src/mpich-4.2.0-5build3' 'LDFLAGS=-Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -Wl,-z,relro' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=3'
    Process Manager:                         pmi
    Launchers available:                     ssh rsh fork slurm ll lsf sge manual persist
    Topology libraries available:            hwloc
    Resource management kernels available:   user slurm ll lsf sge pbs cobalt
    Demux engines available:                 poll select

I don’t know what is going one here. It does not seem to make sense. There must be an explanation, but without having physical access to the machine there is little else that I can do.

Seems like it’s this bug in MPICH:

2 Likes

Reinstalled with OpenMPI and it seems to work as intended. I am just curious as to why running with in.lj gives once again the output:

Communication information:
MPI library level: MPI v4.1
MPI version: MPICH Version:      4.2.0
MPICH Release date: Fri Feb  9 12:29:21 CST 2024
MPICH ABI:          16:0:4
Comm style = brick,  Comm layout = uniform
Communicate velocities for ghost atoms = no
Communication mode = single
Communication cutoff = 2.8
Nprocs = 8,   Nthreads = 1
Processor grid = 2 x 2 x 2

Does this mean that LAMMPS for some reaso still recognizes MPICH as the available MPI implementation? I have confirmed from the CMake configuration that OpenMPI directories are used.