Hello people,
Here is a small tip for those who want to frequently save gzipped dump files, while running on multi-core nodes. And that is to replace the call to gzip (which usually is a single core executable on linux systems) with the multi-core gzipping program pigz, see
Using pigz, you avoid the wasted time when e.g. 15 out of 16 cores sit idle, while one core takes care of the zipping process all by itself. Pigz scales very well over multiple cores. On a very short run that was mostly just writing out big gzipped dump files every few steps, I got a factor ~6 speedup on an octocore system and factor 15 on an 48-core system. The reason why the 48-core test didn't scale as good as on the octocore was probably that writing to disk was more of a bottleneck there.
I couldn't get this to work by aliasing gzip to pigz in my account of job submit file, but that may just be how our cluster and queue system are set up. But it is very easy to just replace a few instances of the word gzip in dump.cc with pigz, requiring exactly 20 letters of code changes:
[[email protected]...:src]$ diff dump.cpp dump.cpp-orig
463,464c463,464
< char pigz[128];
< sprintf(pigz,"pigz -6 > %s",filecurrent);