Downloading data for all materials and keeping it up to date

Materials Project
1

Hi all,

for our planned spin-off ExoMatter we will reguarly use Materials Project data, which has been extremely helpful. Of course we will properly cite and credit the data.
It would be a lot more efficient and decrease bandwidth use to download data on all materials in regular intervals and keep our own version of the database instead of using the API all the time.
Our suggestion would be to pull the database once a month. Is this okay to do and do you have any suggestions on how to best approach this?

Best,

Josua

2

Hi @josua_exomatter,

The easiest way to keep an up-to-date copy of our database would be to access data via the new API (next-gen.materialsproject.org), and request the last_updated field to figure out which full updated documents you want from each endpoint at some regular interval.

I will note that for endpoints serving much larger data, such as band structure, dos, and charge density objects, we request that you contact us directly to organize big data transfers. This is because the data is costly to provide, and we intentionally limit its availability in extremely large batches via the API.

– Jason

3

Hi Jason,

thanks a lot for the reply! As of now, we don’t need endpoints with band structure data and the like, but I will contact you if that changes.

Best,

Josua