Post-processing very large LAMMPS data files

Currently I have to post-process a LAMMPS data file that contains ~ tens of millions of particles, and render cross-section images/movies. So I am wondering if I can still do it with VMD by avoiding activating GUI at all, and work through purely from command line and script files? I guess it might be common practice if one works on HPC.

Thanks for sharing any user experience!

Best regard,

Hi

I suggest to use OVITO (http://www.ovito.org/), I faced no problem while analyzing 2 million particles.

Regards
Afshin

Currently I have to post-process a LAMMPS data file that contains ~ tens of
millions of particles, and render cross-section images/movies. So I am
wondering if I can still do it with VMD by avoiding activating GUI at all,
and work through purely from command line and script files? I guess it might
be common practice if one works on HPC.

what you can do with VMD is primarily limited by the amount of RAM you have.
if you want to process serious data, you need a machine with a serious
amount of RAM.
much more than a graphics card.

but also, you need to be smart with how you build your visualizations.
it is often helpful to strip off atoms, e.g. solvent, that you don't
even want to show and operate with "fast" representations (line,
point) until you have settled on a scene that you want to show
containing only the visible/relevant atoms.
some examples are here: https://sites.google.com/site/akohlmey/images-and-movies
everything was done with VMD. some images/movies were generated by
writing out raytracer scene files and then manually rendering them at
the desired resolution on a local HPC cluster with using the raytracer
in paralle.

please note that LAMMPS internal visualization support via dump image
can be used (of course not interactively) with some creativity and
would run/render in parallel.

many of the restrictions that apply to VMD also apply to other
visualization tool. few interactive tools like VMD have as little
overhead for storing extremely large systems and can access the data
as quickly, but others may be more specialized for particular needs.

whatever tool you use, you will need to spend some effort in working
out an efficient workflow and have a good look at the respective
documentation.

axel.

Currently I have to post-process a LAMMPS data file that contains ~ tens of
millions of particles, and render cross-section images/movies. So I am
wondering if I can still do it with VMD by avoiding activating GUI at all,
and work through purely from command line and script files?

> I guess it might

be common practice if one works on HPC.

what you can do with VMD is primarily limited by the amount of RAM you have.
if you want to process serious data, you need a machine with a serious
amount of RAM.
much more than a graphics card.

I've looked at systems with 10^6 particles in VMD using a cheap dell
inspiron laptop with something like ~4Gb of RAM an integrated graphics
card. (I don't think I was using all that RAM, however.) I remember
I was even able to do that using the fancy tachyon renderer with
ambient occlusion enabled. (If I remember correctly, rendering this
way uses significantly more memory. I almost ran out of memory on
this occasion.) It was slow, but it worked.

If you are worried about speed, you can use the "Lines" drawing
representation. (Select the "Graphics"->"Representations" menu option
from the "VMD Main" window.) Then after you have finished oriented
the system to the desired viewing angle, you can change the
representation to something more computationally demanding.

As Axel mentioned, you can save the raytracer scene files and render
them on a faster computer if you need to.

I doubt it helps, but you can run VMD non-interactively from the
terminal by starting it with the "-e" command-line option. Position,
orientation, and screen-size (and most other settings) can be
controlled by TCL commands.

vmd -e commands.tcl

(Thanks, Afshin, for mentioning OVITO. It looks like it also can
render with ambient-occlusion as well, and supports LAMMPS.)

Andrew