Enhancing Computational Efficiency using OVITO's python module

OVITO General questions
1

This data extraction takes roughly about an hour or more. Each pipeline consists roughly 120 dumps.
How can I reduce computational time? My PC has 6 cores and 12 threads
Part of my analysis script is as follow:

import os
os.environ['OVITO_THREAD_COUNT'] = "12"
from ovito.io import *
from ovito.data import *
from ovito.modifiers import *
from ovito.pipeline import *
import glob
import sys
import re

def cna_analysis(pipeline, i):
# CNA analysis
    modifier = CommonNeighborAnalysisModifier(
        mode=CommonNeighborAnalysisModifier.Mode.AdaptiveCutoff
    )

    pipeline.modifiers.append(modifier)
    if not os.path.exists(f"CNA"): 
        os.makedirs(f"CNA")

# Evaluate pipeline
    data = pipeline.compute()

    export_file(pipeline, 
                f"CNA/cna{i}.txt", 
                "txt/attr",
                columns =["Timestep",
                        "CommonNeighborAnalysis.counts.BCC",
                        "CommonNeighborAnalysis.counts.FCC", 
                        "CommonNeighborAnalysis.counts.HCP", 
                        "CommonNeighborAnalysis.counts.ICO",
                        "CommonNeighborAnalysis.counts.OTHER"], 
                multiple_frames = True)

for i in range(1, {n}):
   print("Processing {i}")
   pipeline = import_file(f"DUMPS{i}/pkadump.*.gz", multiple_frames=True)
   cna_analysis(pipeline, i)

Thank you for your response.

2

Hi.

Please replace the multiple_frames = True keyword parameter in the export_file() call with frame = i.

multiple_frames makes the function export all frames of the trajectory. And since you are calling this in a loop over all trajectory frames, it currently exports the entire trajectory 120 times (that’s 120*120 CNA computations).

Another inefficiency in your approach is that import_file("DUMPS1/pkadump.*.gz") in a loop reads frame 0 over and over again. It should be called just once to avoid this. Also, the extra pipeline.compute() call is unnecessary and slows the process down, because, without any parameter, it will compute frame 0, not the i-th frame.

The most efficient way to process the entire trajectory would be without any for-loop in your Python script. Simply import the dump file sequence, set up the modifier(s), then export the entire trajectory:

pipeline = import_file("DUMPS1/pkadump.*.gz", multiple_frames=True)
pipeline.modifiers.append(CommonNeighborAnalysisModifier(
        mode=CommonNeighborAnalysisModifier.Mode.AdaptiveCutoff
))
export_file(pipeline, 
                "CNA/cna.*.txt", 
                "txt/attr",
                columns =["Timestep",
                        "CommonNeighborAnalysis.counts.BCC",
                        "CommonNeighborAnalysis.counts.FCC", 
                        "CommonNeighborAnalysis.counts.HCP", 
                        "CommonNeighborAnalysis.counts.ICO",
                        "CommonNeighborAnalysis.counts.OTHER"], 
                multiple_frames = True)

You can add the following to the top your script to track progress:

import ovito
ovito.enable_logging()
3

Thank you for your response. There’s a slight correction; I am looping over consecutive events. My exact situation is, I am reading the 120 dumps for 50 consecutive events. I need the for loop.

# corrected
# 50 consecutive events, each has 120 dumps
for i in range(1, 51):
   print("Processing {i}")
   pipeline = import_file(f"DUMPS{i}/pkadump.*.gz", multiple_frames=True)
   cna_analysis(pipeline, i)
4

Thanks, this will be very helpful for me.

1 Like
5

Oh yes, that’s something I misunderstood, sorry. Then your original script is mostly fine, I would say - aside from the one useless pipeline.compute() call.

How large is the system that you’re processing (how many atoms)? And how big is one of the pkadump.gz snapshot files?

I’m not entirely sure if the import_file(…, multiple_frames=True) hurts performance. The multiple_frames option is not necessary here. The * placeholder in the filename already tells OVITO to load a series of files as a trajectory. multiple_frames=True additionally tells it to scan each individual file to discover the frames it contains (probably just 1 each in your case).

6

My system comprises 531360 atoms. Total space taken by 50 events is roughly 112 GB of storage. Size of single dump file is roughly 19 MB.
I am in immense need to reduce total computation time. My PC has 6 cores and 12 threads.
I am performing Wigner-Seitz, Vacancy Cluster, Interstitial Cluster, PTM and Cluster, CNA and DXA on 50 consecutive simulations. So, it’s taking a lot of time. So, is there any other ways/ options to enhance computation?
Thank you for your time.

7

Have you tried using the multiprocessing approach outlined in the manual: Advanced topics — OVITO Python Reference 3.15.5 documentation

You could process each directory in parallel using 1 or 2 threads each.