Running parallel instances of LAMMPS coupled to C++

LAMMPS LAMMPS Mailing List Mirror
1

I'm trying to make a parallel C++ code that runs parallel instances of LAMMPS, but I'm running into some issues. Here's a very simplified example of my code:

//////////////////////////////////////////////////////////////////////////////

#include <iostream>
#include <string>
#include <sstream>
#include "mpi.h"

// LAMMPS files
#include "lammps.h"
#include "input.h"
#include "atom.h"
#include "library.h"

using namespace std;

// LAMMPS namespace and pointers
using namespace LAMMPS_NS;
char *args1[] = {
    (char *) "lmp",
    (char *) "-log",
    (char*) "none",
     0};
LAMMPS *lmp;

int main(int argc, char **argv)
{

  // Start MPI
  MPI_Init(&argc,&argv);

  // Create instance of LAMMPS
  lmp = new LAMMPS(3,args1,MPI_COMM_WORLD);

  // Finalize MPI
  MPI_Finalize();
}

//////////////////////////////////////////////////////////////////////////////

This is simply the standard way of calling LAMMPS within C++.

I successfully compile with:
     mpiCC test.cpp -o test /path/to/liblammps.so

I successfully execute with:
     mpirun -np 1 test

I cannot, however, successfully execute with:
     mpirun -np 2 test

It seems whenever I use more than one process, the program freezes and the other processes never actually start an instance of LAMMPS.

Can I use the same dynamic variable (lmp) for all processes, like I am doing in this example? How can I run multiple instances in parallel?

Thanks for your time.

2

I am not exactly sure how to do stuff the way you are trying to do. I tend to just use the library interface instead (in library.h). That works fine to write driver-like programs, and you can still run LAMMPS in parallel.

If by running multiple instances of LAMMPS in parallel you mean two or more LAMMPSes running at the same time, I am not sure if you should pass all of them MPI_COMM_WORLD or if you should assign all of them a different rank. Then again, I am not an MPI programmer at all, I am just able to copy-paste stuff to make it work.

3

I am also using the library interface (library.h).

Yes, I am trying to run multiple instances of LAMMPS in parallel (one instance per processor) in a single code instead of running a single instance in parallel. I have tried sending separate dynamic variable instances to each process by rank, as well as a single dynamic variable to each process like I showed in my example. Both had the same result - the extra process (rank 1 or higher) seemed to not run an instance of LAMMPS.

Thanks.

4

I’m not sure exactly what you are trying to accomplish. Maybe you can do what you want by using the partition command?

http://lammps.sandia.gov/doc/partition.html

5

I’m writing a generalized potential fitting code that uses LAMMPS as a calculator and for the library of potentials. The reason I need multiple instances is that I want to split the potential evaluations into separate processes for speed.

Each evaluation has it’s own set of parameters (pair coefficients), so each evaluation needs to be under a different instance of LAMMPS.

6

I'm trying to make a parallel C++ code that runs parallel instances of
LAMMPS, but I'm running into some issues. Here's a very simplified example
of my code:

//////////////////////////////////////////////////////////////////////////////

#include <iostream>
#include <string>
#include <sstream>
#include "mpi.h"

// LAMMPS files
#include "lammps.h"
#include "input.h"
#include "atom.h"
#include "library.h"

using namespace std;

// LAMMPS namespace and pointers
using namespace LAMMPS_NS;
char *args1[] = {
    (char *) "lmp",
    (char *) "-log",
    (char*) "none",
     0};
LAMMPS *lmp;

int main(int argc, char **argv)
{

  // Start MPI
  MPI_Init(&argc,&argv);

  // Create instance of LAMMPS
  lmp = new LAMMPS(3,args1,MPI_COMM_WORLD);

  // Finalize MPI
  MPI_Finalize();
}

//////////////////////////////////////////////////////////////////////////////

This is simply the standard way of calling LAMMPS within C++.

I successfully compile with:
     mpiCC test.cpp -o test /path/to/liblammps.so

I successfully execute with:
     mpirun -np 1 test

I cannot, however, successfully execute with:
     mpirun -np 2 test

It seems whenever I use more than one process, the program freezes and the
other processes never actually start an instance of LAMMPS.

Can I use the same dynamic variable (lmp) for all processes, like I am
doing in this example? How can I run multiple instances in parallel?

​which version of LAMMPS is this with?

i just took your code and compiled it against the (static) lammps library
from the latest code ​and there is no obvious problem:

[[email protected]... wrap] mpirun \-np 4 \./wrap\.x LAMMPS \(15 Apr 2016\-ICMS\) &nbsp;&nbsp;using 1 OpenMP thread\(s\) per MPI task \[akohlmey@\.\.\.3785\.\.\. wrap\]

the parallel program launches and finishes. there is one small issue with
your test. you are missing a "delete lmp;" before MPI_Finalize().
that should properly tear down the LAMMPS instance and the output should be
instead:

[[email protected]... wrap] mpirun \-np 4 \./wrap\.x LAMMPS \(15 Apr 2016\-ICMS\) &nbsp;&nbsp;using 1 OpenMP thread\(s\) per MPI task Total wall time: 0:00:00 \[akohlmey@\.\.\.3785\.\.\. wrap\]

as others have pointed out, your description of what you want to achieve is
somewhat ambiguous. do you want to have *independent* LAMMPS instances (one
for each MPI rank, each processing a different calculation), or do you want
to run a parallel LAMMPS session with just one input across the multiple
LAMMPS ranks.

axel.

7

Or I should say - the evaluations are currently done consecutively under 1 instance. I’d like to speed up the optimization by splitting the evaluations to be done in parallel. In order for them to be in parallel, they must be different instances because each evaluation has its own set of pair coefficients.

8

Is there a reason you cannot do this with world style variables http://lammps.sandia.gov/doc/variable.html and the partition command?

9

I can only confirm what Axel says, here it seems to work fine… I compile like
mpicxx main.cpp -o main -lmpi -llammps -I/home/stefan/projects/lammps/src/

and the program runs and finishes no matter how many processors I assign with mpirun. =/

10

I'm writing a generalized potential fitting code that uses LAMMPS as a
calculator and for the library of potentials. The reason I need multiple
instances is that I want to split the potential evaluations into separate
processes for speed.

Each evaluation has it's own set of parameters (pair coefficients), so
each evaluation needs to be under a different instance of LAMMPS.

​then you cannot pass the world communicator as argument. you have to first
split this into separate communicators (each of which can consist of one or
more MPI ranks) with MPI_Comm_split()​ and then pass the split
communicators as argument. when you pass MPI_COMM_WORLD, there will only
be one input processed.

axel.

11

Axel’s point is critical to understand.

If you are running your C-code on P procs under MPI and want to

run P indendent instances of LAMMPS, then each proc has

to call the LAMMPS lib to invoke a 1-proc instance of LAMMPS

with a communicator that is a one-proc communicator. If they

all invoke LAMMPS with MPI_COMM_WORLD, you get

1 instance of LAMMPS.

This is not a LAMMPS thing, it’s the way MPI works.

Steve

12

I will look into this. World-style variables would help with optimization of non-tabulated parameters, but many potentials in LAMMPS use files of tabulated parameters in which the variable command cannot be used. My current code has a user-interface that allows you to specify which parameters in a tabulated file you’d like to optimize, and I’d need different instances for each tabulated parameter file if the potential evaluations are done in parallel.

13

here is a quick-n-dirty modification of your code template, that does a split by generating multiple LAMMPS instances with 2 processors per instance (just to demonstrate the principle. if you care about speed, this would be a way to get even more).

#include
#include
#include
#include “mpi.h”

// LAMMPS files
#include “lammps.h”
#include “input.h”
#include “atom.h”
#include “library.h”

using namespace std;

// LAMMPS namespace and pointers
using namespace LAMMPS_NS;
char *args1[] = {
(char *) “lmp”,
(char ) “-log”,
(char) “none”,
0};
LAMMPS *lmp;

int main(int argc, char **argv)
{
int color,key,global,local;
MPI_Comm comm;

// Start MPI
MPI_Init(&argc,&argv);

MPI_Comm_rank(MPI_COMM_WORLD, &global);
color = global / 2;
key = global;
MPI_Comm_split(MPI_COMM_WORLD, color, key, &comm);
MPI_Comm_rank(comm,&local);

// printf(“global rank: %d, key: %d, local rank: %d\n”,global,key,local);

// Create instance of LAMMPS
lmp = new LAMMPS(3,args1,comm);
lmp->input->one(“region box block 0 1 0 1 0 1”);
lmp->input->one(“create_box 1 box”);

delete lmp;

// Finalize MPI
MPI_Finalize();
}

and here the corresponding output when run across 8 cpus.

[akohlmey@…3785… wrap] mpirun -np 8 ./wrap.x LAMMPS (15 Apr 2016-ICMS) LAMMPS (15 Apr 2016-ICMS) using 1 OpenMP thread(s) per MPI task LAMMPS (15 Apr 2016-ICMS) using 1 OpenMP thread(s) per MPI task LAMMPS (15 Apr 2016-ICMS) using 1 OpenMP thread(s) per MPI task using 1 OpenMP thread(s) per MPI task Created orthogonal box = (0 0 0) to (1 1 1) Created orthogonal box = (0 0 0) to (1 1 1) Created orthogonal box = (0 0 0) to (1 1 1) Created orthogonal box = (0 0 0) to (1 1 1) 1 by 1 by 2 MPI processor grid Total wall time: 0:00:00 1 by 1 by 2 MPI processor grid Total wall time: 0:00:00 1 by 1 by 2 MPI processor grid Total wall time: 0:00:00 1 by 1 by 2 MPI processor grid Total wall time: 0:00:00 [[email protected]... wrap]

14

Hey,

this is MPI programming,
not really LAMMPS related.

You need to split your communicator,
probably you want to create groups,
then you can do this.

Best,
S.

15

Wow thanks a lot Axel, this works for me and is exactly what I was trying to do. I’m almost done with this fitting code (fits forces, structures, etc…) which is general for every potential in LAMMPS. I’ll try to submit it as an add-on once finished and hopefully it would be useful to people needing parameters for different materials and potentials.

Thanks for your help.