About the arrow category

Starting from database version 2026.04.13, the Materials Project has begun storing all of its core data products on AWS S3 in the form of parquet files with additional metadata management technologies (currently: Delta Lake). Apache Arrow, Parquet, and Open Table Formats (OTFs) may be unfamiliar topics, so this is the place to ask questions and get guidance.

See the Materials Projects Documentation on Data Lakes (as pertains to MP) for additional curated details.