Hash failure for POTCAR

For the same set of Vasp pseudopotentials, I noticed that there is a hash checking failure for the 12.22 version of pymatgen, while the same pseudopotential directory has worked without problem from the first setup up to including the pymagen 11.11 version. Is something broken in the hash function?

On a related note, I observed that there are a set of “54” LDA and PBE PAW pseudopotentials on the Vasp site (e.g. PAW POTCAR files: PBE version 54 (recommend) and PAW POTCAR files: LDA version 54 (recommend)) versus the PAW PS ( PAW POTCAR files: LDA & PBE, 5.2 & 5.4 (original unvie release version)). The names enclosed by parenthesis are those from the Vasp portal. Could it be that the hash failure is caused the has functions matching the newer set of PAW PS? The Vasp site states that the (recommended) PAW PS have (i) the TITLE string is set to the directory in which the POTCAR file reside for: O_GW_new, Ge_GW, G_GW_new, Cd_GW, Br_GW, B_GW. (ii) W_sv potential has been added. (iii) HASH key added to all POTCAR files. FOR STRICT COMPATIBILITY, download the set of potentials in potpaw_LDA_PBE_52_54_orig.tar.gz! Any suggestions are welcome.


Hi @Paul_Fons, welcome to the forum!

Tagging @rkingsbury here since he implemented the hashing functionality.

For total energies to be directly comparable between different VASP calculations, the same pseudopotentials must be used. For this reason, the Materials Project has stuck to an older set of pseudopotentials to be consistent across its entire database. If you do not need your energies to be comparable to those from the Materials Project (e.g. if you’re not constructing phase diagrams/convex hulls etc.) then you’re welcome to use a newer pseudopotential version and disregard this warning – and, in fact, this might be the recommended option.

The intent of the hashing was to add an additional check to prevent the accidental use of incompatible pseudopotentials when using pymatgen when MP-compatible pseudopotentials were desired.

Hi @Paul_Fons, happy New Year!

When we implemented hash checking, we decided to check both the hash of the entire POTCAR file (including title, headers, etc.) and the hash of just the data that defines the pseudopotential. If a POTCAR fails the whole-file hash check you will see a UserWarning, while a BadHashError is raised if the hash of the POTCAR data itself doesn’t match our records.

Are you seeing the error or the warning?

Thank you for your reply. I am specifying the PBE_54 pseudopotentials as recommended by the VASP developers. The error is fatal and no POTCAR file is generated. I have regenerated the pymatgen POTCAR directory after downloading the most recent 54 PAW pseudopotentials from the VASP website and the error persists. Could it be that the hashes are incorrect. Unfortunately, there the pseudopotentials on the VASP website are not dated so it is not possible to know when the pseudopotentials were last changed, although the description indicates that the headers of the 54 pseudopotentials were modified, it would appear that the pseudopotentials themselves are the same. Since the files are text, it would seem that even if the pseudopotentials are unchanged but for the header, a reformatting of the files would result in a hash failure. In any case, I am using the correct, most recent PBE_54 PAW pseudopotentials and I should not be encountering a fatal hash failure I would think.

Hi @Paul_Fons, it sounds like you’re seeing a BadHashError then, which means that pymatgen’s hash of the POTCAR is different than what we have in our validation table. If only the header changed but the POTCAR itself were the same, you would see a UserWarning about the hashes that would not be fatal.

Let me download the latest POTCARs again and investigate. Which Input Set are you using?

@Paul_Fons – also worth noting that PBE_54 are not the default, if you want to use PBE_54 you have to explicitly tell pymatgen with user_potcar_functional=PBE_54 when you create a VASP input set. If you’re not doing this, you might get the BadHashError @rkingsbury mentioned.

The potential files are the latest ones from the VASP website. Note there are two versions of the PBE_54 and LDA_54 potentials on the website. The newest is referred to as “Recommend” while the second, older versions are referred to as " original unvie release version". I am using the latest versions hence it would seem that the hash values in pymatgen as incorrect. I have also specified that the pseudopotential version by the pmgrc (xml) file with the line “PMG_DEFAULT_FUNCTIONAL: PBE_54”. This according to the pymatgen webpage should result in the default use of the 54 PAW pseudopotential as specified on the pymatgen webpage as below:

If you are using newer sets of pseudopotential files from VASP, the directory names may be different, e.g., POT_GGA_PAW_PBE_52. For such cases, please also add a default functional specification as follows:

pmg config --add PMG_DEFAULT_FUNCTIONAL PBE_52

Thanks for pointing out the multiple version of PBE_54 @Paul_Fons, I don’t think I’d noticed that. Our systems should have the most up to date POTCARs but we’re investigating.

I’m having trouble reproducing your error. Can you post a code snippet that generates the fatal error, along with the error message?

Thank you for your comment. Here is an example of the hash failure (for Silicon).

In [1]: from pymatgen import Lattice, Structure, Molecule
…: coords = [[0, 0, 0], [0.75,0.5,0.75]]
…: lattice = Lattice.from_parameters(a=3.84, b=3.84, c=3.84, alpha=120,
…: beta=90, gamma=60)
…: struct = Structure(lattice, [“Si”, “Si”], coords)
…: coords = [[0.000000, 0.000000, 0.000000],
…: [0.000000, 0.000000, 1.089000],
…: [1.026719, 0.000000, -0.363000],
…: [-0.513360, -0.889165, -0.363000],
…: [-0.513360, 0.889165, -0.363000]]
…: methane = Molecule([“C”, “H”, “H”, “H”, “H”], coords)

In [2]: struct
Structure Summary
abc : 3.84 3.8399999999999994 3.84
angles : 119.99999999999999 90.0 60.00000000000001
volume : 40.038580810231124
A : 3.84 0.0 2.351321854362918e-16
B : 1.92 2.7152900397563426 -1.919999999999999
C : 0.0 0.0 3.84
PeriodicSite: Si (0.0000, 0.0000, 0.0000) [0.0000, 0.0000, 0.0000]
PeriodicSite: Si (3.8400, 1.3576, 1.9200) [0.7500, 0.5000, 0.7500]

In [3]: static = MPStaticSet(struct)

In [4]: static.write_input(output_dir=‘test’)

BadHashError Traceback (most recent call last)
----> 1 static.write_input(output_dir=‘test’)

~/pymatgen/lib/python3.6/site-packages/pymatgen/io/vasp/sets.py in write_input(self, output_dir, make_dir_if_not_present, include_cif, potcar_spec, zip_output)
696 include_cif=include_cif,
697 potcar_spec=potcar_spec,
–> 698 zip_output=zip_output
699 )
700 for k, v in self.files_to_transfer.items():

~/pymatgen/lib/python3.6/site-packages/pymatgen/io/vasp/sets.py in write_input(self, output_dir, make_dir_if_not_present, include_cif, potcar_spec, zip_output)
181 same name as the InputSet (e.g., MPStaticSet.zip)
182 “”"
–> 183 vinput = self.get_vasp_input()
185 if potcar_spec:

~/pymatgen/lib/python3.6/site-packages/pymatgen/io/vasp/sets.py in get_vasp_input(self)
158 kpoints=self.kpoints,
159 poscar=self.poscar,
–> 160 potcar=self.potcar,
161 )

~/pymatgen/lib/python3.6/site-packages/pymatgen/io/vasp/sets.py in potcar(self)
129 “”"
130 potcar = Potcar(self.potcar_symbols, functional=self.potcar_functional)
–> 131 validate_potcar_hash(potcar)
132 return potcar

~/pymatgen/lib/python3.6/site-packages/pymatgen/io/vasp/sets.py in validate_potcar_hash(potcar)
2872 validation. Verify the integrity of your
2873 POTCAR files. ".format(
-> 2874 psingle.symbol
2875 )
2876 )

BadHashError: POTCAR data hash for POTCAR Si did not pass validation. Verify the integrity of your POTCAR files.

A slight update. I had thought that having a .pmgrc.yaml entry specifying the potential would be switch effective, however, I now suspect that it didn’t work or worked in an unexpected way. (the yaml file line reads as “PMG_DEFAULT_FUNCTIONAL: PBE_54”) as when I specified the potential change manually using user_potcar_functional, I received a warning, not a hash error.

For the same Silicon structure, here is the output

static = MPStaticSet(struct,user_potcar_functional=‘PBE_54’)
/home/matstud/pymatgen/lib/python3.6/site-packages/pymatgen/io/vasp/sets.py:413: BadInputSetWarning: Overriding the POTCAR functional is generally not recommended as it significantly affect the results of calculations and compatibility with other calculations done with the same input set. Note that some POTCAR symbols specified in the configuration file may not be available in the selected functional.

In [5]: static.write_input(output_dir=‘temp’)
/home/matstud/pymatgen/lib/python3.6/site-packages/pymatgen/io/vasp/sets.py:2885: UserWarning: POTCAR file hash for POTCAR Si did not pass validation. It is possible that the metadata in the POTCAR has changed. We will continue loading the POTCAR because the hash of the data itself has passed validation.

Hi @Paul_Fons, thanks for sharing those updates. Each InputSet can define its own default functional that takes precedence over the global default set in PMG_DEFAULT_FUNCTIONAL.

When we implemented hash checking, we moved definition of the default functional for each InputSet from the pymatgen code into the respective .yaml files that define INCAR settings, POTCAR symbols, etc. for each InputSet. So if you look in MPStaticSet.yaml you will see a line that reads POTCAR_FUNCTIONAL: PBE. This means the InputSet will expect PBE pseudopotentials unless the user_potcar_functional argument is set to something else. This change was intended to minimize accidental use of unintended POTCARs.

The Warning that you are getting now might be related to the fact that, as you pointed out, there are two different versions of the PBE_54 functionals that differ only in their metadata / headers. It’s possible that we hashed against a different version than the one you have. We’ll look into that.

The hashes in vasp_potcar_file_hashes.json don’t appear to be md5 hashes of any POTCAR files. If so, using a POTCAR that does not match the data hash in vasp_potcar_pymatgen_hashes.json will always fail.

see https://github.com/materialsproject/pymatgen/issues/1746

@Paul_Fons I wanted to make sure you saw that we did indeed identify a bug in our hash files and have a fix in progress. Thank you for reporting and please let us know if you experience any other problems after the fix is released.