New mp-api python package compatibility

Materials Project
1

I’m starting to play with the (new) api for the first time, and I get the following error

Traceback (most recent call last):
  File "/home/cluster2/bernstei/src/work/MACE/foundations/t.py", line 2, in <module>
    from mp_api.client import MPRester
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/mp_api/client/__init__.py", line 8, in <module>
    from .mprester import MPRester
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/mp_api/client/mprester.py", line 10, in <module>
    from emmet.core.electronic_structure import BSPathType
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/emmet/core/electronic_structure.py", line 26, in <module>
    from emmet.core.material_property import PropertyDoc
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/emmet/core/material_property.py", line 10, in <module>
    from emmet.core.material import PropertyOrigin
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/emmet/core/material.py", line 13, in <module>
    from emmet.core.structure import MoleculeMetadata, StructureMetadata
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/emmet/core/structure.py", line 18, in <module>
    class StructureMetadata(EmmetBaseModel):
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_model_construction.py", line 182, in __new__
    complete_model_class(
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_model_construction.py", line 491, in complete_model_class
    schema = cls.__get_pydantic_core_schema__(cls, handler)
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/main.py", line 578, in __get_pydantic_core_schema__
    return __handler(__source)
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_schema_generation_shared.py", line 82, in __call__
    schema = self._handler(__source_type)
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 468, in generate_schema
    schema = self._generate_schema(obj)
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 700, in _generate_schema
    schema = self._post_process_generated_schema(self._generate_schema_inner(obj))
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 722, in _generate_schema_inner
    return self._model_schema(obj)
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 541, in _model_schema
    {k: self._generate_md_field_schema(k, v, decorators) for k, v in fields.items()},
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 541, in <dictcomp>
    {k: self._generate_md_field_schema(k, v, decorators) for k, v in fields.items()},
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 886, in _generate_md_field_schema
    common_field = self._common_field_schema(name, field_info, decorators)
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 951, in _common_field_schema
    schema = self._apply_annotations(
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1654, in _apply_annotations
    schema = get_inner_schema(source_type)
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_schema_generation_shared.py", line 82, in __call__
    schema = self._handler(__source_type)
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1635, in inner_handler
    schema = self._generate_schema(obj)
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 700, in _generate_schema
    schema = self._post_process_generated_schema(self._generate_schema_inner(obj))
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 727, in _generate_schema_inner
    return self.match_type(obj)
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 810, in match_type
    return self._match_generic_type(obj, origin)
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 834, in _match_generic_type
    return self._union_schema(obj)
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1036, in _union_schema
    choices.append(self.generate_schema(arg))
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 470, in generate_schema
    metadata_js_function = _extract_get_pydantic_json_schema(obj, schema)
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1990, in _extract_get_pydantic_json_schema
    raise PydanticUserError(
pydantic.errors.PydanticUserError: The `__modify_schema__` method is not supported in Pydantic v2. Use `__get_pydantic_json_schema__` instead.

Is this just a sign that I need a different pydantic version? If so, shouldn’t the pip dependencies be taking care of that automatically?

I’m installing mp-api from pypi, and my pydantic.__version__ is 2.5.2.

2

In a fresh environment with python 3.9 and mp-api v0.39.0 I cannot recreate this issue. Which version of monty are you running with?

– Jason

3

Unfortuantely, I need to this to coexist with some other packages if at all possible, so I can’t easily to a clean install in production (but can certainly do it for testing purposes, at least). Looks like I have a conflict between two installations of monty, if nothing else. Let me see if I can clear that up before we continue here.

4

OK, I cleaned up various version conflicts between system-wide things and my ~/.local directory, and things are somewhat better. However, I have torch available but not actually functional on our head node, and that seems to break mp-api

tin 3156 : python3 t.py
Traceback (most recent call last):
  File "/home/Software/python/system/extra/lib64/python3.9/site-packages/torch/__init__.py", line 174, in _load_global_deps
    ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
  File "/usr/lib64/python3.9/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libcufft.so.11: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cluster2/bernstei/src/work/MACE/foundations/t.py", line 2, in <module>
    from mp_api.client import MPRester
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/mp_api/client/__init__.py", line 7, in <module>
    from .core import MPRestError
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/mp_api/client/core/__init__.py", line 3, in <module>
    from .client import BaseRester, MPRestError
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/mp_api/client/core/client.py", line 22, in <module>
    from emmet.core.utils import jsanitize
  File "/home/cluster2/bernstei/.local/lib/python3.9/site-packages/emmet/core/utils.py", line 9, in <module>
    from monty.json import MSONable
  File "/home/Software/python/system/extra/lib/python3.9/site-packages/monty/json.py", line 58, in <module>
    import torch
  File "/home/Software/python/system/extra/lib64/python3.9/site-packages/torch/__init__.py", line 234, in <module>
    _load_global_deps()
  File "/home/Software/python/system/extra/lib64/python3.9/site-packages/torch/__init__.py", line 196, in _load_global_deps
    ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
  File "/usr/lib64/python3.9/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libcudart.so.12: cannot open shared object file: No such file or directory
tin 3157 :

Is there any way to avoid having monty (I guess) try to use torch? Nothing I plan to do will use a GPU, as far as I know. I’ll modify my system to avoid having a broken torch setup, but it’d be nice if, as long as it isn’t actually required, I could use the API without it.

[added later - I can confirm that if I switch to the cpu-torch I have installed it appears to run correctly]

5

Anyway, thanks for the pointer to monty being the issue. I missed that in the original error message.

I’m happy to declare that this is either a monty issue (should handle broken torch more gracefully) or that it’s purely an issue with our installation which caused the broken torch to begin with (in particular that it fails with an OSError, which monty doesn’t catch, rather than an ImportError, which it does).

6

Thread closed due to inactivity, please open a new thread to address related issues.