Downloading the Full Database via OPTIMADE – Overcoming the 1000-Result Limit

OPTIMADE
1

I am new to using OPTIMADE, and I am trying to retrieve an entire database without applying filters. However, I am encountering a restriction that prevents me from downloading more than 1000 results at a time. The following warning is displayed:

Warning:
“Reached 1000 results for https://optimade.materialsproject.org/, exceeding max_results_per_provider parameter (1000). Stopping download.”

For my research, I need to download the complete dataset to compare it with our in-house database of compounds. When using the Materials Project API directly, I can retrieve the entire database, but this does not seem possible through OPTIMADE.

Is there a way to increase this limit or an alternative approach to obtaining the full dataset? Additionally, as someone new to OPTIMADE, I find the documentation somewhat challenging to follow. If anyone could recommend a good tutorial or share example code for querying and handling large datasets efficiently, I would greatly appreciate it.

Thank you for your guidance!

2

See this post Bulk Download Assistance for Crystal struct + properties Using OPTIMADE API(AFLOW Data as showcase)

these docs for the client Using the OPTIMADE client - OPTIMADE Python tools

and the tutorials at GitHub - Materials-Consortia/optimade-tutorial-exercises: Tutorial exercises for the OPTIMADE API

3

Thanks.