NoSuchKey error

Materials Project on-staff-radar
1

Hi, I’ve been trying to download a large dataset of materials (with HasProps.charge_density and fields=material_id). I query the list of materials (for e.g. upto 2000-3000 Hexagonal symmetry samples) in order to write vasp files by first instantiating mpr.get_charge_density_from_material_id(material_id) of the respective materials.

The mpr.get_charge_density_from_material_id(material_id) returns error for multiple samples as follows: NoSuchKey: An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.

On checking the potential error, I realised that the material-id/key is not present for those particular materials in the AWS S3 buckets (materialsproject-{raw,parsed,build}).

This, prevents me from generating the vasp files in one shot, and I have to restart every time one of the material-id keys doesn’t exist in the list.

Is there a way around this? Should I look into boto3 (python interfacing with AWS systems) and set up some flags where it breaks down/or any other code changes using which I won’t stumble into this problem ?

Thanks a lot for your help in advance!

2

@Sancran19 Thanks for reaching out and providing detailed information about your issue.

Everything you’re doing is correct. I can confirm that mp-1104027, for instance, wasn’t found but most should be there. I’ll have to look into why the specific material that is supposed to have a charge density is not found. For now, you’d have to catch the exception and continue downloading the rest:

try:
    mpr.get_charge_density_from_material_id(material_id) ....
except Exception as:
    print(ex)
    continue

You could also go through our AWS OpenData repository directly if you prefer:

The API client uses the same repo behind the scenes to retrieve the charge density data.