Get_phonon_dos_by_material_id returns None; many mp-api dependency issues with latest upgrade

Hello!

I have a program which I have been using for a long time which calls “MPRester.get_phonon_dos_by_material_id”, and yesterday (without any changes to my own environment), I began to receive ‘None’ types for this call (for systems which I know have phonon DOS in MP because I have been pulling and using them all year.) I see all the past posts whose similar issues with get_dos_by_material_id, get_structure_by_material_id, etc. were resolved by updating the mp-api version (so now I’m at v0.45.6) , but when I did this: I got the following:
ImportError: cannot import name 'PhononBS' from 'emmet.core.phonon' (C:\Users\xxxxxx\miniforge3\envs\ElasticMod\Lib\site-packages\emmet\core\phonon.py). Did you mean: 'Phonon'?

Following my nose through the proceeding cascade of dependency errors, I tried upgrading Pymatgen (now v2025.5.28), emmet-core (now v0.84.7), pydantic (now v2.11.6), and pydantic-core (v2.33.2) until I reached the following error:

    PDOS = mpr.get_phonon_dos_by_material_id(mpid)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\[...]\Lib\site-packages\mp_api\client\mprester.py", line 1251, in get_phonon_dos_by_material_id
    doc = self.materials.phonon.search(material_ids=material_id, fields=["ph_dos"])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\[...]\Lib\site-packages\mp_api\client\routes\materials\phonon.py", line 60, in search
    return super()._search(
           ^^^^^^^^^^^^^^^^
  File "C:\[...]\Lib\site-packages\mp_api\client\core\client.py", line 1191, in _search
    return self._get_all_documents(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\[...]\Lib\site-packages\mp_api\client\core\client.py", line 1264, in _get_all_documents
    results = self._query_resource(
              ^^^^^^^^^^^^^^^^^^^^^
  File "C:\[...]\Lib\site-packages\mp_api\client\core\client.py", line 464, in _query_resource
    raise MPRestError(
mp_api.client.core.client.MPRestError: invalid fields requested: ['ph_dos']. Available fields: ['builder_meta', 'nsites', 'elements', 'nelements', 'composition', 'composition_reduced', 'formula_pretty', 'formula_anonymous', 'chemsys', 'volume', 'density', 'density_atomic', 'symmetry', 'identifier', 'phonon_method', 'phonon_bandstructure', 'phonon_dos', 'epsilon_static', 'epsilon_electronic', 'born', 'force_constants', 'last_updated', 'sum_rules_breaking', 'structure', 'total_dft_energy', 'volume_per_formula_unit', 'formula_units', 'supercell_matrix', 'primitive_matrix', 'code', 'post_process_settings', 'thermal_displacement_data', 'calc_meta', 'material_id', 'task_ids']

Naturally, seeing there was a clear keyword mistake, I went into mp_api/client/MPRester.py and changed the keyword from “ph_dos” to “phonon_dos” to see if this would fix the issue, but instead I was met with:

    make_tensor_heat_map(trueTensor, VASPTensor.voigt, mpid=mpid, vasp=True, nionic_steps=nsteps, showDebye=True)
  File "c:\[...]\elastic_tensor_from_bulk_relax.py", line 1317, in make_tensor_heat_map
    PDOS = mpr.get_phonon_dos_by_material_id(mpid)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\[...]\site-packages\mp_api\client\mprester.py", line 1251, in get_phonon_dos_by_material_id
    doc = self.materials.phonon.search(material_ids=material_id, fields=["phonon_dos"])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\[...]\site-packages\mp_api\client\routes\materials\phonon.py", line 60, in search
    return super()._search(
           ^^^^^^^^^^^^^^^^
  File "C:\[...]\site-packages\mp_api\client\core\client.py", line 1191, in _search
    return self._get_all_documents(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\[...]\site-packages\mp_api\client\core\client.py", line 1264, in _get_all_documents
    results = self._query_resource(
              ^^^^^^^^^^^^^^^^^^^^^
  File "C:\[...]\site-packages\mp_api\client\core\client.py", line 570, in _query_resource
    data = self._submit_requests(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\[...]\site-packages\mp_api\client\core\client.py", line 717, in _submit_requests
    initial_data_tuples = self._multi_thread(
                          ^^^^^^^^^^^^^^^^^^^
  File "C:\[...]\site-packages\mp_api\client\core\client.py", line 939, in _multi_thread
    data, subtotal = future.result()
                     ^^^^^^^^^^^^^^^
  File "C:\[...]\Lib\concurrent\futures\_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "C:\[...]\Lib\concurrent\futures\_base.py", line 401, in __get_result
    raise self._exception
  File "C:\[...]\Lib\concurrent\futures\thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\[...]\site-packages\mp_api\client\core\client.py", line 1017, in _submit_request_and_process
    data["data"] = self._convert_to_model(data["data"])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\[...]\Lib\site-packages\mp_api\client\core\client.py", line 1059, in _convert_to_model
    data_model(
  File "C:\[...]\Lib\site-packages\pydantic\main.py", line 253, in __init__
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\[...]\Lib\site-packages\pydantic\_internal\_mock_val_ser.py", line 100, in __getattr__
    raise PydanticUserError(self._error_message, code=self._code)
pydantic.errors.PydanticUserError: `MPDataDoc` is not fully defined; you should define `PhononComputationalSettings`, then call `MPDataDoc.model_rebuild()`.

For further information visit https://errors.pydantic.dev/2.11/u/class-not-fully-defined

Now, having already updated pydantic and pydantic-core, I have officially hit a wall. Can someone please help me understand what’s going on and address this dependency issue with mp-api?

Thanks for reporting this. There are new convenience functions in mpr.materials.phonon that should work: api/mp_api/client/routes/materials/phonon.py at b80ce7347df282e6d4dd470a6627bb90ab19bf57 · materialsproject/api · GitHub I’ll work on updating the root-level functions and release a new client. I’ll report back when done.

1 Like

I just released mp-api==0.45.7. Upgrade and give it a try. Thanks!

1 Like

Thank you for working to address this so quickly! I updated and now I actually am getting a PhononDOS object returned from MPRester.get_phonon_dos_by_material_id(), but strangly I get the following error:

  File "c:\[...]\Debye_model.py", line 112, in fit_Debye_temperature
    cv.append(PDOS.cv(t) / nformulas_per_cell) # note that I divide by nsites for the unit cell because the returned heat capacity values are in units of J/(K*mol-c),
              ^^^^^^^
  File "c:\[...]\site-packages\pydantic\main.py", line 994, in __getattr__
    raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')
AttributeError: 'PhononDOS' object has no attribute 'cv'

It looks like the underlying variables of PhononDOS have not changed in pymatgen.phonon.dos.PhononDOS (that is, I still see a cv() function call in the PhononDOS object)

The PhononDOS object now used in mp-api is from emmet.core.phonon. It comes with a to_pmg property to convert to pymatgen objects if needed: emmet/emmet-core/emmet/core/phonon.py at 6ac77dd796606e454c8be95df11993ba77c53bcb · materialsproject/emmet · GitHub @Aaron_Kaplan

ah I see. Everything works as desired once I use the to_pmg extention - thank you for your help. Forgive me if I sound critical, but for the sake of friendly discussion - don’t you think its confusing for the api call to return an object with distinct functionalities/variables and the exact same name? What is the purpose of returning a half-baked version of the pymatgen object instead of returning the actual pymatgen object?

I find myself spending a lot of time sorting through the multitude of different document types that mp-api calls return these days and I’m not going to lie, it can get frustrating from a user-interface perspective. On top of this, I struggle to find any sufficient documentation on WHAT the api returns (or even where to find it in the source code) and how its actually different from the pymatgen object, especially because so often APIDocs are really lacking in functionality compared to the pymatgen object.

The data objects that the API client returns are designed for efficient storage / retrieval. The pymatgen objects, while designed around ease of use and analysis features, are generally non-performant for bulk storage/retrieval

As they’re intentionally designed around different purposes, I wouldn’t say these are half-baked

The MP class names are also different from the pymatgen classes (PhononDOS vs CompletePhononDos, and PhononBS vs PhononBandstructureSymmLine)

As for return types: The MP data models we’re designing now allow for interchange between different formats. That’s why the newer Phonon BS and DOS objects have a to_pmg method, and why the newer trajectories support both to_pmg and to_ase.

I think we all agree that updating our documentation would be ideal, but we’re a small team of 3-4 people and we have to prioritize advancing/maintaining our infrastructure per our funding. External users are more than welcome to contribute updates to our docs.

I’d like to second @Aaron_Kaplan’s response. We are now closing in on 650,000 users and have 5,000 daily active users requesting millions of data points and downloading TBs of data. The responsibility to develop and operate the Materials Project platform with a 99.98% uptime currently rests on 3 scientific research engineers. We are taking questions and feedback on our forum seriously. We try to be responsive and fix issues or provide solutions with fast turnarounds. As Aaron mentioned and as you and @Wenhao_Sun probably already know, the best way for community members - who benefit from this free resource - to contribute back, is to use the “Edit on Github” button in our docs to suggest changes or improve them. Thanks!

Of course - please know that I sympathize with the amount of work that lies on your shoulders, and I am extremely appreciative of your attentiveness to program issues when they arise. My intention is not to attack an under-supported team, but to open a discussion about the best choices for an intuitive user experience with the mp-api.

I will say I really appreciate the to_pmg and to_ase methods. These are new to me and I think fill a big gap between mp-api and pymatgen - thank you for that. It also makes sense to me that mp-api objects are optimized for efficient retrieval and return, I can see how this would take priority over the pymatgen objects’ full functionality. Thank you for helping me understand this.

My problem lies more with the barriers to intuit the difference between mp-api and pymatgen objects - like when the objects are called the same thing.

In response to @Aaron_Kaplan, it may be true that the mp-api PhononBS object is distinctly named from the pymatgen PhononBandstructureSymmLine/PhononBandstructure object, but the CompletePhononDOS is based on the pymatgen class PhononDOS, which makes it very confusing to troubleshoot because my debugger tells me I have a PhononDOS object without telling me the origin. I assumed this was the pymatgen.phonon.dos.PhononDOS. How am I to know that the object at hand is instead the emmet.core.phonon.PhononDOS object? I understand the distinction now, and have a broader understanding that MPRester will not return the pymatgen object itself, but I’m trying to point out that having these two objects with the same name makes it inherently more difficult for users to troubleshoot on their own. In the future, I will try to suggest changes directly on the github. I’m deeply sorry if I have offended either of you, I’m just trying to point out some UI challenges with the mp-api especially as it changes so often.

Don’t worry about it, it’s easy for miscommunications to come up over text

The UI concerns are something we’re actively mapping out now and we’ll share any updates once those are concrete. For convenience functions, returning a pymatgen object is likely what we will return in the long run

For now, you can always do type introspection on objects: type(mpr.get_phonon_dos_by_material_id("mp-*")) will give you the right import string

1 Like