Hello,
I am trying to run LAMMPS through python with the intention to calculate forces in python when needed by LAMMPS. So far I ran the “demo.py” script in “python/examples” and was successful, which tells me that (hopefully) my compilation for the python library interface worked.
Now I am using the “demo.py” and “in.demo” as a basis to build my own script. Specifically, I would like to import both lammps and tensorflow libraries in a python script to calculate the forces using neural networks from atom positions in LAMMPS, then return the forces back into lammps. In “in.demo” I removed the “pair_style” and “pair_coeff” commands and replaced them with:
Hi Alejandro, you need to distinguish from the case when you launch Python and it calls functions from the LAMMPS library, vs. when you launch LAMMPS and it calls Python (i.e. a callback mechanism). There are cases where you can use both modes of coupling in the same run, but they are distinct ways of coupling the two programs nonetheless.
Examples of the callback scheme are not in python/examples, but rather in examples/python. Having the two locations may be a bit odd, but it makes sense somewhat, because the former shows general examples about running things from Python, but the latter is part of the main folder of the LAMMPS examples, where Python callbacks are one of many features.
Lastly, between running the examples and writing your own input you really need to read the documentation of the features that you plan to use (e.g. fix python/move or pair_style python).
https://lammps.sandia.gov/doc/fix_python_move.html
https://lammps.sandia.gov/doc/pair_python.html
Giacomo
the python library interface and the PYTHON package in LAMMPS are two different things. they can be used together, but neither is a requirement for the other.
you are mixing those up.
now the python command allows you to connect some python code to be launched from LAMMPS, e.g. as a python style variable.
but a) as LAMMPS already indicated, your python command is incorrect and b) these can only return one global scalar, not a tuple or list.
to programming the application you want to do is far more complex than what you seem to be expecting.
you can trigger executing a python function from LAMMPS regularly during an MD or minimization run,
but you need a more complex mechanism. e.g. you need to use the scatter and gather functions to explicitly communicate positions from LAMMPS to your machine learning code and then return the results. that code will also have to deal with periodic boundaries and domain decomposition parallelization in LAMMPS (and thus consider local and ghost atoms) and - since have no pair style, you would also need to set a communication cutoff to have a sufficient number of ghost atoms available.
so before implementing interactions with your machine learning code, i strongly suggest you implement something simpler, e.g. a lennard-jones interaction, with an interface similar to what you plan to implement for your machine learning potential and then test it against the internal lennard-jones implementation. that way you can make certain, that your flow of data is correct without being impacted by complications from the tensorflow library (which will happen and add another layer of difficulties to your problem).
axel.
Hello Axel (and others),
Thanks for the response. I figured I was trying to oversimply the problem. I know that there are some existing packages like “Quest” and “Flowtential” (https://github.com/jbelof/Flowtential) that communicate back-and-forth between LAMMPS and some other code (the former with DFT, and the latter with a neural network). I tried to get “Flowtential” to work, but when I compile the makefile I usually get an error like “libcouple_mpi missing”. I also tried the March 2017 version of LAMMPS, which this package was made for, and I was able to successfully compile the package, but when I attempt to run the package (i.e. ./lmpdnn -np 1 in.lammps) I get “ERROR: Fix external callback function not set (…/fix_external.cpp:102)”.
Sorry that the description of the problem is not so detailed… I am just not familiar enough with the packages to know what is going on. Since the “Flowtential” is heavily based on “Quest,” I think reading their documentation and contacting them is my best bet to get a neural network working. The Flowtential is especially attractive since their python script does exactly what I already do in my neural network python code, that is, obtain atom positions, consider periodic boundary to calculate neighbors, compute forces from NN using neighbor positions, and finally predict the atomic forces and return them. Perhaps in their package they already consider the complexities you mentioned. Additionally, I think I would just have to edit this python script to implement my version of the neural network, but I will see.
Another issue to think about is that, if LAMMPS references python when requiring the forces, the python script is time-consuming in that the tensorflow packages take some time to import (maybe ~4-6 secs) which would be devastating if this occurs at each timestep. This is why I was thinking about running LAMMPS from python, since I would only need to import tensorflow once at the beginning of the script and then run dynamics when needed from LAMMPS.
Thanks for your help, I think I have a better image of the problem now. Certainly if I come up with something that works I will share it.
Alejandro
Hi Alejandro
Another issue to think about is that, if LAMMPS references python when requiring the forces, the python script is time-consuming in that the tensorflow packages take some time to import (maybe ~4-6 secs) which would be devastating if this occurs at each timestep.
Shouldn’t you be able to access the pre-initialized TensorFlow objects without reinitializing, but just re-using at each step the same objects?
But regardless, if you are looking beyond a proof of principle calculation (i.e. more than a tiny number of atoms and more than one process), the issue becomes far more complex. As Axel pointed out there are many things to look at more closely on the LAMMPS side.
I would add that you may also need to ensure that garbage collection for the Python objects is carried out, which I’ve seen not to be guaranteed when you call only very specific Python functions from a few places in LAMMPS, and not return control to the Python interpreter until much later.
Giacomo
Hello Alejandro,
I have been thinking about this for a few months now. Having python calculate lammps forces is not an easy issue and to make something efficient is even harder.
If you are serious about this, what Axel suggested is 100% the best way to go. If you are set on making this, shoot me an email I would love to chat.
Best,
Charlie
apart from the already mentioned methods, there is also the option to use fix external and define a callback from python.
it is also possible to access the neighbor list(s).
there is an ongoing project to refactor the library interfaces (for c, fortran, python and c++) and improve the documentation, so that may be worth revisiting in a few months. we hope to have this ready (or at least the bulk of it) for the next stable release.
axel.