Add finished VASP calculation to database

esimmen · September 4, 2024, 9:10am

Hi, it happened once or twice that my vasp calculation finishes fine but the workflow fails at the reading in state. Since I don’t always want to rerun the entire calculation, I wanted to write a function that just reads the finished Vasp job and adds it to the mongoDB. Does anyone already have a function like that or suggestions how to best do it?

I discussed with my colleague that we could have a maker that only reads in the calculation by taking the directory of the successful calculation or its fw_id as an argument.

Thanks for your help!

fraricci · September 4, 2024, 9:38am

Hi there!

Do you mind sharing some more details, such as which workflow are you using and version of atomate (1 or 2)?

At a general level, I’d check which function is responsible for adding the vasp output into the db in the workflow you’re using. In atomate1 there something like VaspToDbTaskDrone that’s responsible for that. An idea could be to run that function in those folder that where the output was not uploaded. Alternatively, again for atomate1, you could try to rerun at the task level. This could work if the last task is the one responsible for uploading the output.

For atomate2, I’m not sure how you could do that. Maybe other can chime in.

Hope this helps.
FR

esimmen · September 4, 2024, 10:08am

Sure. I’m using atomate2, and workflows mostly StaticMaker() or DoubleRelaxMaker()

fraricci · September 5, 2024, 2:07pm

I gather some information to answer you question.

In atomate2 this should be the function used to parse the output and you could potentially reuse it manually for all the directories where the job stopped.

github.com

materialsproject/atomate2/blob/f14586c7b29a7d5cbf7eabfe399728ab6362e0e7/src/atomate2/vasp/jobs/base.py#L249


      
                      files_list=_FILES_TO_ZIP,
                  )
          
                  return Response(
                      stop_children=stop_children,
                      stored_data={"custodian": task_doc.custodian},
                      output=task_doc,
                  )
          
          
          def get_vasp_task_document(path: Path | str, **kwargs) -> TaskDoc:
              """Get VASP Task Document using atomate2 settings."""
              kwargs.setdefault("store_additional_json", SETTINGS.VASP_STORE_ADDITIONAL_JSON)
          
              kwargs.setdefault(
                  "volume_change_warning_tol", SETTINGS.VASP_VOLUME_CHANGE_WARNING_TOL
              )
          
              if SETTINGS.VASP_RUN_BADER:
                  kwargs.setdefault("run_bader", _BADER_EXE_EXISTS)
                  if not _BADER_EXE_EXISTS:

which internally use this in emmet

github.com

materialsproject/emmet/blob/34e8cf88574e6cad59febe27a235a3126fcc27d0/emmet-core/emmet/core/tasks.py#L509


      
          def set_entry(self) -> datetime:
              if (
                  not self.entry
                  and self.calcs_reversed
                  and getattr(self.calcs_reversed[0].output, "structure", None)
              ):
                  self.entry = self.get_entry(self.calcs_reversed, self.task_id)
              return self
          
          @classmethod
          def from_directory(
              cls: Type[_T],
              dir_name: Union[Path, str],
              volumetric_files: Tuple[str, ...] = _VOLUMETRIC_FILES,
              store_additional_json: bool = True,
              additional_fields: Optional[Dict[str, Any]] = None,
              volume_change_warning_tol: float = 0.2,
              task_names: Optional[list[str]] = None,
              **vasp_calculation_kwargs,
          ) -> _T:
              """

The upload is done directly by jobflow into the db that is specified in your settings.

github.com

materialsproject/atomate2/blob/f14586c7b29a7d5cbf7eabfe399728ab6362e0e7/src/atomate2/vasp/jobs/base.py#L245


      
                  # gzip folder
                  gzip_output_folder(
                      directory=Path.cwd(),
                      setting=SETTINGS.VASP_ZIP_FILES,
                      files_list=_FILES_TO_ZIP,
                  )
          
                  return Response(
                      stop_children=stop_children,
                      stored_data={"custodian": task_doc.custodian},
                      output=task_doc,
                  )
          
          
          def get_vasp_task_document(path: Path | str, **kwargs) -> TaskDoc:
              """Get VASP Task Document using atomate2 settings."""
              kwargs.setdefault("store_additional_json", SETTINGS.VASP_STORE_ADDITIONAL_JSON)
          
              kwargs.setdefault(
                  "volume_change_warning_tol", SETTINGS.VASP_VOLUME_CHANGE_WARNING_TOL
              )

In alternative you could simply have a maker identical to the one you’re using but without the line that write input and run vasp. So that it just does the parsing and uploading. I think you could either implement your own maker and run it in the right directories, or in a more hacky way, you could comment the lines that run vasp and do a rerun in the same folder (which is possible with Fireworks with something like --task-level --prev-dir, check these flags).

Note that if you run another maker or manually parse and upload the output into the db, you flows’ status won’t be updated. With the second option of rerunning the hacked maker you could also have the state of the job/flow.

Please, do not take it for granted I think it could work but I haven’t test it myself.

Hope this helps!
FR