Hello everyone,
I’m running a molecular dynamics (MD) simulation using LAMMPS and experiencing unexpectedly long wall times. I’d appreciate your help identifying potential bottlenecks or optimization opportunities.
System details:
System composition:
2400 water molecules (SPC/E model)
40 surfactant molecules (custom molecule with ~30–40 atoms each)
Total atoms: ~10,000–12,000
Simulation time:2.5 ns
Time step:0.5 fs
Cutoff: 10.0 Å
Long-range electrostatics: PPPM with kspace_style pppm 1e-5
Force fields:OPLS-AA for surfactants, standard LJ + Coulomb for water
Bond/Angle/Dihedral:Harmonic for bonds/angles, OPLS for dihedrals
Hardware and software details:
CPU: Intel i9 (13th gen)
GPU: NVIDIA RTX 3060 (12 GB VRAM)
RAM: 32 GB
Parallelization: 1 GPU + 6 CPU threads (using package gpu and -sf gpu)
Performance issue:
For a 2.5 ns simulation, the total wall time is ~19 hours (~760 minutes).
If i am increasing the timestep My simulation is getting crashed.
Hello,
It is not clear what you are asking. Are you looking for tips to improve simulation speed?
Best,
Simon
Unexpectedly long wall times compared to what?
To provide suggestions for that, one first has to determine what is the expected performance for the same or sufficiently similar systems on your machine and similar equipment and compare your performance with what to expect. This would have to be done with and without GPU. Given that you have a somewhat older and not a high-end GPU, it may be quite possible that parallelizing over all CPU cores with MPI may be faster than using the one GPU.
Also, before looking closer, one has to determine that there are no issues with your hardware, no other processes competing with your LAMMPS process for resources, and whether your input has no mistakes or intrinsic inefficiencies. The rather vague description you provide of your system and setup is not sufficient to make any expectation of performance. You are not even mentioning what your LAMMPS version is and what platform you are running on and how you installed LAMMPS.
For example, you should be able to run a timestep of 2fs with correct application of constraints, but we don’t know if you are doing this as needed or incorrectly.
As some background information about how to measure and report performance with LAMMPS, you could have a look at: 7.2. Measuring performance — LAMMPS documentation
-------------------- SYSTEM INITIALIZATION --------------------
units real
atom_style full
dimension 3
boundary p p p
newton on
comm_modify cutoff 10.0
-------------------- FORCE FIELD SETUP --------------------
GPU-optimized styles (remove ‘package gpu’ if not using GPU)
bond_style harmonic
angle_style harmonic
dihedral_style opls
pair_style lj/cut/coul/long 10.0 10.0
kspace_style pppm 1e-5
kspace_modify order 4
special_bonds lj 0.0 0.0 0.5 coul 0.0 0.0 1.0 angle yes
-------------------- SIMULATION BOX --------------------
region box block 0 80 0 80 0 80
create_box 13 box bond/types 7 angle/types 14 dihedral/types 15 extra/bond/per/atom 6 extra/angle/per/atom 10 extra/dihedral/per/atom 12 extra/special/per/atom 60
-------------------- FORCE FIELD PARAMETERS --------------------
include parameters.inc
-------------------- MOLECULE CREATION --------------------
molecule nethoxy nethoxy.mol
create_atoms 0 random 40 87910 NULL mol nethoxy 454756 overlap 4.0
molecule h2omol water.mol
create_atoms 0 random 2400 87910 NULL mol h2omol 454756 overlap 3.5
-------------------- ION GENERATION --------------------
create_atoms 12 random 60 34134 box overlap 1.5
create_atoms 13 random 60 12756 box overlap 1.5
set type 12 charge 1.0
set type 13 charge -1.0
-------------------- GROUP DEFINITIONS --------------------
group H2O type 10 11
group nethoxy type 1 2 3 4 5 6 7 8 9
group ions type 12 13
group allatoms union H2O nethoxy ions
-------------------- OVERLAP REMOVAL --------------------
delete_atoms overlap 3.0 H2O nethoxy mol yes
delete_atoms overlap 2.5 ions ions
delete_atoms overlap 2.5 ions H2O mol yes
delete_atoms overlap 2.5 ions nethoxy mol yes
-------------------- NEIGHBOR LIST SETTINGS --------------------
neighbor 2.5 bin
neigh_modify delay 0 every 1 check yes one 20000 page 500000
-------------------- CONSTRAINTS --------------------
fix fixshake H2O shake 0.0001 20 0 b 1 a 1
-------------------- ENERGY MINIMIZATION --------------------
min_style cg
minimize 1.0e-6 1.0e-8 1000 10000
reset_timestep 0
velocity allatoms create 298.0 12345 mom yes rot yes dist gaussian loop geom
-------------------- EQUILIBRATION: NVT --------------------
fix mynvt all langevin 298.0 298.0 100.0 12345 zero yes
thermo 1000
thermo_style custom step temp press vol density ke pe etotal enthalpy lx ly lz
run 50000
unfix mynvt
write_data final.data
The time required to simulate this system is very high performance is 4.5 ns/day
What is the reason it is taking so much time and what are the ways to optimise it ?
Again, you did not address the main question by @akohlmey. What makes you think that the performances should be higher?
Also, you don’t have any `fix nve’ in your NVT equilibration, are your atom moving during this step???
Your post is hard to read since you didn’t use the proper ``` formatting for quoted input data.
Also, you don’t provide any other information I asked for. Please see the forum guidelines post to learn how to properly post questions here to get answers. I you don’t make sufficiently easy to read and assess the information you provide, you won’t get much specific help. We don’t have the time to do this all by ourselves and cannot provide suggestions when missing crucial pieces of information.
Why did you just remove the attachments and thus make it impossible to reproduce your simulation elsewhere?
I cannot help if you don’t provide required information or allow me to gather this myself. You ask for help, but at the same time make it near impossible to help you. You are just wasting your time and ours. At this point, I have stopped to care about providing help. You need to hope that somebody else doesn’t mind. From a cursory look at your input, you should be much more concerned about running a correct simulation rather than running fast. Producing garbage data faster is not really helping.
2 Likes
water.mol (445 Bytes)
parameters.inc (1.5 KB)
nethoxy.mol (5.7 KB)
nethoxy.lj (3.4 KB)
Sir these are the files. And after 72000 steps bond atom missing error is coming
Thanks: Some comments:
- your density is very low 0.18 g/cm^3 which means your box volume is large, which in turn increases the cost of Kspace (which scales with volume) and reduces the cost of Pair (which scales by the average number of neighbors per atom).
- why do you change the neighbor list “one” and “page” parameters? with such a low density, that should not be needed?
- why do you set the communication cutoff. It will be overridden later anyway.
- the system will shrink considerably. You need to budget for significant equilibration time. Why not create a more dense system from the beginning. For example with change_box?
- Also during shrinking the PPPM accuracy will drop as the system is shrinking. You need to run multiple short runs and continue until the density as stabilized. collecting RDF data before that has happened is utterly meaningless.
- Why not apply shake also to the hydrogen atom bonds of your other molecules?
- Your fix shake command is actually shaking the wrong bonds and angles. As could be seen, if you add shake statistics output
- Using
thermo_modify lost ignore is a very, very bad idea. Ignoring problems instead of fixing them never helps.
The possible reasons to that have been discussed many times and there is some explanation about this in the manual section on Errors as well.
The whole input looks like it was mindlessly copied together from different parts without any regards for the actual details. I suggest you spend some more time with LAMMPS tutorials and the LAMMPS manual to make certain that you understand what you are doing.
Given the large volume and low density, the performance of your simulation is dominated by the kspace calculation. You can see that in the MPI timing summary. For the low density, you can get twice as fast by using double the coulomb cutoff. But as the density is increasing during fix npt, you have to lower the cutoff or else the time spent in Pair will dominate and also the neighbor lists will overflow.
The time allotted in your input for equilibration is in no way sufficient. overall, the simulation result are garbage due to the many small mistakes.
I would suggest to forget about this specific input and start all over and do this step by step. It is extremely difficult to follow all details and the indicated atom and bond types are confusing (better to start in the molecule file with types 1 and then use “offset” keywords with the molecule command.
When trying to enable fix shake for all hydrogen atoms, it seems that there is an incorrect bond topology which assigns too many hydrogen atoms to a single heavier atom.
GI-GO
After going the necessary changes as you mentioned above. I am getting wall time of 2.5 ns/day which is quite low for rtx 3060 graphics card. So any changes which I can make in the lammps code to reduce the wall time.I want to run the simulation for 20 ns.
trial.lj (4.9 KB)
What is the justification for this statement?
You have consistently failed to provide any throughout the entire discussion.
You should worry more about correctness. Most of the issues that have been pointed out in this discussion still exist. Running a simulation that produces garbage results faster is still a waste of time.
At this point, you have exhausted my goodwill and my desire to help you has dropped to zero. I don’t like to help people that don’t really pay attention to my suggestions and concerns. You will have to wait for somebody else to help you. Good luck.