You can parallelize each LAMMPS instance and over different EVB states and also run the main LAMMPS instance on a separate group of MPI processes.
The important step is to use a new communicator for each LAMMPS instance that you want to use concurrently or alternately. This can be done with mpi4py directly. Please see below for a simple example that will run two LAMMPS instances concurrently on half the processes and maintains two LAMMPS instances per process, so there are 4 LAMMPS simulations in total, but two are running concurrently on different processors each. You can see that by looking at the individual log files.
Second, you can reduce the overhead of running the neighbor list rebuild by adding ‘pre no’ to the run 0 command (you also want to use post no, which you can always do). However, with ‘pre no’ you have to be careful. If there are topology changes, you must not use it and also you need to regularly rebuild your neighbor lists as atoms are moving around. Depending on your system and the timestep, that can be every other run 0 command or every 5 or even every 10. Please keep in mind the law of diminishing returns. When rebuilding the neighbor lists only every 5 steps, the overhead is already reduced by 80% and for every 10 steps, the overhead is reduced by 90%, so there is not much benefit from going beyond that.
#!/usr/bin/env python3
from lammps import lammps
from mpi4py import MPI
me_global = MPI.COMM_WORLD.Get_rank()
np_global = MPI.COMM_WORLD.Get_size()
comm_global = MPI.COMM_WORLD
if me_global == 0:
print("Running with %d processors" % np_global)
# new communicators for half the ranks each
color = me_global % 2
key = me_global
num = int(np_global/2)
comm = []
comm.append(comm_global.Split(color, key))
comm.append(comm_global.Split(color, key))
lmp = []
args = ['-nocite', '-screen', 'none', '-var', 'id', str(me_global), '-log', 'log.lammps-'+str(me_global)]
lmp.append(lammps(cmdargs=args, comm=comm[0]))
lmp[0].file('in.melt')
args = ['-nocite', '-screen', 'none', '-var', 'id', str(me_global+num), '-log', 'log.lammps-'+str(me_global+num)]
lmp.append(lammps(cmdargs=args, comm=comm[1]))
lmp[1].file('in.melt')
lmp[0].command('run 200 pre no post no')
lmp[1].command('run 200 pre no post no')
lmp[0].command('run 200 pre no post no')
lmp[1].command('run 200 pre no post no')
lmp[0].close()
lmp[1].close()
using the following file as in.melt:
units lj
atom_style atomic
lattice fcc 0.8442
region box block 0 10 0 10 0 10
create_box 1 box
create_atoms 1 box
mass * 1.0
velocity all create 3.0 87287 loop geom
pair_style lj/cut 3.0
pair_coeff 1 * 1.0 1.0 1.122
neighbor 0.3 bin
neigh_modify every 10 delay 0 check no
fix 1 all nve
variable id index 0
dump id all atom 50 dump-${id}.lammpstrj
thermo 50
run 200 post no