You are probably right. I ran a test case for the LJ fluid and even though the communication time is high, adding more processors doesn’t seem to increase communication time as much as I thought it would. I use infiniband and that really helps.
I have tried Newton off in the past for a long-chain alkane system similar to the test case that really helped too. Another issue I have is that my system is highly inhomogeneous (liquid-like in the middle and vapor-like on both ends). I’m curious to see if omp will help.
I attached my test case to this email. The in.lj_hot is the test case. I also included in.lj_cold which was used to generate the initial configuration.