Tencent Cloud server blocked by Materials Project IP/ASN policy

Materials Project Materials Project Data/API
1

Hello Materials Project team,

I am encountering an IP/ASN block when accessing the Materials Project API from a Tencent Cloud server used for a controlled research deployment.

Materials Project account email:

[email protected]

Blocked public IP:
43.138.141.74

Observed behavior:

  • Materials Project heartbeat from my local network: HTTP 200
  • One-record summary query from my local network: HTTP 200
  • Materials Project heartbeat from the Tencent Cloud server: HTTP 403
  • One-record summary query from the Tencent Cloud server: HTTP 403

The same Materials Project credential was used in both environments. The server response indicates that the source IP address or ASN has been temporarily blocked by the Materials Project Cloudflare policy.

Cloud environment:

  • Provider: Tencent Cloud
  • Region: Guangzhou, China
  • Operating system: Ubuntu 22.04
  • Workload: 2D-MatPilot research deployment

Use case:

2D-MatPilot is an evidence-grounded research pre-screening and validation-planning workbench for two-dimensional materials.

The deployment only needs to perform one explicit synchronization of a small curated set of approximately 12 Materials Project records. It is not a continuous crawler, bulk downloader, or public API proxy.

Traffic and data controls:

  • Requests are batched rather than sent in per-material loops.
  • Only required response fields are requested.
  • Results are cached in PostgreSQL after synchronization.
  • Synchronization is explicitly initiated by an operator.
  • Repeated downloads of the same data are avoided.
  • Only one individual Materials Project API key is used.
  • The API key is not shared with users or exposed by the application.
  • Fixture fallback is disabled.
  • No formal Materials Project synchronization has been completed from the blocked server.

The production deployment was stopped immediately after the network block was detected. No public service has been activated.

Could a Materials Project team member please review the Cloudflare block for this controlled research use case and advise whether the public IP can be allowed?

Please let me know if you need the Cloudflare Ray ID, request timestamps, client version, or a redacted HTTP response.

Thank you.

2

Hi @wenxuan_liu ,

Both Alibaba and Tencent have unfortunately been sources of mis-/abuse in the past which is why we decided to disallow traffic.

For your use-case though there is a pretty simple workaround to just use MP’s Open Data S3 repositories: MP AWS Open Data intro docs

You mentioned:

The deployment only needs to perform one explicit synchronization of a small curated set of approximately 12 Materials Project records.

Which records are needed for this? Since your scope is pretty small I can probably direct you to what you need pretty quickly.

If you ever want to expand your project in the future I would recommend using the web-based browsers (parsed data, build data) to find out what is available on S3 that might generally suit your needs.

You can either use the aws cli to download data, or use query engines to interact with the increasing amount of data products that are stored in cloud native formats.

3

Hi,

Thank you again for offering to help locate the records.

I went back through our project and audited the material IDs carefully. Our curated set actually consists of 12 target materials, but after checking them against the current Materials Project data, only 10 currently have canonical MP IDs that we have been able to verify:

  1. Graphene / C — mp-aaaaaabw

  2. MoS2 — mp-aaacgmaf

  3. WS2 — mp-aaaaaaiq

  4. WSe2 — mp-aaaaacsb

  5. h-BN — mp-aaaaablw

  6. Black phosphorus / P — mp-aaaaaagb

  7. MoSe2 — mp-aaaaackw

  8. SnSe2 — mp-aaaaaazp

  9. InSe — mp-aaaabhot

  10. PtSe2 — mp-aaaaabqx

The remaining two target materials are:

  1. CrI3

  2. TiS2

These two are where I would particularly appreciate your guidance.

Our older configuration associated CrI3 with mp-570478, but that ID no longer appears to resolve in the current database. We found mp-aaacrbov as a possible CrI3 candidate, but have not treated it as authoritative yet.

Likewise, our older configuration associated TiS2 with mp-1207, but our identity audit indicates that this now resolves to NdFe2 rather than TiS2. We found several TiS2 candidates, but have not selected one because we want to make sure we use the appropriate structure/polytype.

Could you help us identify the appropriate current Materials Project records for CrI3 and TiS2?

For the 10 confirmed records above, we would also appreciate a pointer to the most appropriate Open Data S3 dataset / object paths for retrieving their structure, composition, electronic and thermodynamic properties.

For the current deployment, we plan to synchronize this small curated set into our local database rather than continuously query the API.

Longer term, however, the project is intended to expand beyond these 12 materials into larger-scale materials screening. If possible, I would also be interested in your recommendation for which Materials Project Open Data product — parsed data, build data, or the cloud-native/data-lakehouse datasets — would be the best foundation for that larger-scale workflow.

Thank you very much for your help.

Best,
wenxuan_liu

4

Hi @wenxuan_liu ,

advice here may be overkill, but it’s good reference for future forum users

Gotcha, since you are listing the new id formats I’m going to go with cloud-native datasets since the following will be more future proof. I’m attaching a small notebook that pulls data for the 10 material_ids you noted from the summary collection.

matsci_tencent_cloud_issue.ipynb (5.0 KB)

If summary is not the collection you need you can see a list of all cloud-native datasets here: MP Docs: Supported Data Products, the approach is the same.

The notebook uses the deltalake library to serve as a query engine since it’s a convenient in-process engine. You can also set up things like trino as a dedicated/external query engine.

For mp-570478 and mp-1207:

Not sure why mp-1207 would ever have been TiS2 on your end. mp-1207 was originally calculated back in 2011, so that should never have drifted.

For mp-570478 I’m not sure where you would have got this id. I can’t find any record of that, even in our original tasks collection (stores all the base calculations) that dates back to 2011, so I can’t give you any meaningful advice on that one. Sorry!

Small note on the material_id formats, the base representation of the ids changed with the last database release (MP Docs: Identifier Systems). The formats are interchangeable when using the website, mp-api client, and REST API (ex: mp-aaaaaabw == mp-48), but when interacting with the datasets on S3 you might have to do a bit of pre-processing.

emmet-core has a class for this (emmet.core.mpid.AlphaID):

from emmet.core.mpid import AlphaID
print(AlphaID("mp-aaaaaabw").formatted) # or str(AlphaID("mp-aaaaaabw"))
# -> mp-48
print(AlphaID("mp-49", padlen=8))
# -> mp-aaaaaabx

You’ll want the alphabetical version (mp-aaaaaabx) for S3.

There is a setting in your user dashboard to swap your display format on the website if you want to see the AlphaID versions all the time → ‘Identifier Format’

And final final note, I just want to make sure you’re aware of MP’s terms of use that applies to all of MP’s data regardless of where you are retrieving your data. For your system this likely just means that you should acknowledge/cite MP where appropriate.

This part may or may not be relevant for you

, but for thoroughness I will point out also that a subset of MP’s data is licensed as BY-NC. The mp-api and website handle filtering data for you based on what you’ve agreed to, but when interacting with S3 the onus is on you to filter the data appropriately. Modifying the query string in that notebook to exclude all the BY-NC is straightforward:

...
# pull the whole 'BY-C' summary dataset
query_str = f"""
    SELECT *
    FROM   summary
    WHERE  version='2026-04-13'
      AND  builder_meta.license!='BY-NC'
"""
...
5

should have pointed this part out too, the summary collection has this info. You can obviously just print one of those entries retrieved in that notebook, or just consult the SummaryDoc API Reference from emmet-core to search for what you need.

6

Hi,

Thank you very much for the detailed explanation and for preparing the notebook.

The cloud-native dataset approach looks like the right long-term direction for our project. We’ll start with the summary collection and test the workflow with the 10 confirmed material IDs.

Thanks also for clarifying mp-1207 and mp-570478. We’ll correct those historical mappings on our side and keep CrI3 and TiS2 unresolved until the appropriate records are verified.

We’ll also make sure to preserve Materials Project attribution and handle the license metadata appropriately, including filtering BY-NC data where required.

Thanks again — this gives us a much clearer path forward.

Best,
wenxuan_liu

7

Thanks for pointing that out as well. The summary collection looks like it covers exactly what we need for the current stage, so we’ll start there and use SummaryDoc to identify and validate the fields needed by our screening workflow.

Really appreciate the clarification and the notebook!