New MP-API fails in Jupyter

I made a new conda environment with the Python 3.10.5 on CentOS 7.9 and installed the new version of the Materials Project API using “pip install mp-api” after installing pip via anaconda. The installation went fine. I then installed jupyter whose installation had some trouble with a dependency, but functionally seems to work.

   Uninstalling pyzmq-22.3.0:
      Successfully uninstalled pyzmq-22.3.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
maggma 0.47.3 requires pyzmq==22.3.0, but you have pyzmq 23.1.0 which is incompatible.

After starting Jupyter, I tried executing the python notebook from the 2021 Materials Project workshop that dealt with the API, namely " [MAPI Lesson (filled).ipynb" While the raw call requests.get to the database worked fine, attempting to read in the mp-api library failed

from mp_api import MPRester

mpr = MPRester(MP_API_KEY)
mpr.materials.available_fields

resulted in:

ModuleNotFoundError                       Traceback (most recent call last)
Input In [10], in <module>
----> 1 from mp_api import MPRester

ModuleNotFoundError: No module named 'mp_api'

The confusing thing is that if I start up ipython

Python 3.10.5 | packaged by conda-forge | (main, Jun 14 2022, 07:06:46) [GCC 10.3.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.4.0 -- An enhanced Interactive Python. Type '?' for help.

in the same environment, the same input works fine. What is wrong with the installation and why does it not work in Jupyter?

Are you able to install other packages and import them via Jupyter successfully?

Usually this error is because Jupyter is being launched from an environment different from the environment that the package was installed into, and usually requires installing a specific kernel (ipykernel) for that environment into Jupyter.

Yes I have installed matminer and pymatgen without problem. Is there a workaround for this?
Jupyter was installed via anaconda into a new virtual environment. I install mp-api using pip as recommended on the installation page. Matminer, pymatgen all seem fully functional. The pip installation was carried out within a new conda enviroment explicitly made for mp-api.
e.g.
conda create env -n mpapi
pip install mp-api
conda install jupyter

As I mentioned above using the mp-api under ipython (as opposed to Jupyter works fine)

Here are the details of the packages installed for jupyter and ipython. Do you have any ideas for a workaround or a fix?

jupyter --version
Selected Jupyter core packages...
IPython          : 8.4.0
ipykernel        : 6.15.0
ipywidgets       : 7.7.0
jupyter_client   : 7.3.4
jupyter_core     : 4.10.0
jupyter_server   : not installed
jupyterlab       : not installed
nbclient         : 0.6.4
nbconvert        : 6.5.0
nbformat         : 5.4.0
notebook         : 6.4.12
qtconsole        : 5.3.1
traitlets        : 5.3.0
(mpapi) [email protected]:/data/Vasp/Friction/H2O_try2>ipython
Python 3.10.5 | packaged by conda-forge | (main, Jun 14 2022, 07:06:46) [GCC 10.3.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.4.0 -- An enhanced Interactive Python. Type '?' for help.

In the mpapi environment, an “conda list” gives:

# packages in environment at /data/miniconda/envs/mpapi:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
aioitertools              0.10.0                   pypi_0    pypi
anyio                     3.6.1                    pypi_0    pypi
argon2-cffi               21.3.0                   pypi_0    pypi
argon2-cffi-bindings      21.2.0                   pypi_0    pypi
asttokens                 2.0.5                    pypi_0    pypi
attrs                     21.4.0                   pypi_0    pypi
automat                   20.2.0                   pypi_0    pypi
backcall                  0.2.0                    pypi_0    pypi
bcrypt                    3.2.2                    pypi_0    pypi
beautifulsoup4            4.11.1                   pypi_0    pypi
bleach                    5.0.0                    pypi_0    pypi
boltons                   21.0.0                   pypi_0    pypi
boto3                     1.24.12                  pypi_0    pypi
botocore                  1.27.12                  pypi_0    pypi
bravado                   11.0.3                   pypi_0    pypi
bravado-core              5.17.0                   pypi_0    pypi
bzip2                     1.0.8                h7f98852_4    conda-forge
ca-certificates           2022.6.15            ha878542_0    conda-forge
certifi                   2022.6.15                pypi_0    pypi
cffi                      1.15.0                   pypi_0    pypi
charset-normalizer        2.0.12                   pypi_0    pypi
click                     8.1.3                    pypi_0    pypi
constantly                15.1.0                   pypi_0    pypi
crochet                   2.0.0                    pypi_0    pypi
cryptography              37.0.2                   pypi_0    pypi
cycler                    0.11.0                   pypi_0    pypi
debugpy                   1.6.0                    pypi_0    pypi
decorator                 5.1.1                    pypi_0    pypi
defusedxml                0.7.1                    pypi_0    pypi
dnspython                 2.2.1                    pypi_0    pypi
emmet-core                0.28.2                   pypi_0    pypi
entrypoints               0.4                      pypi_0    pypi
executing                 0.8.3                    pypi_0    pypi
fastapi                   0.78.0                   pypi_0    pypi
fastjsonschema            2.15.3                   pypi_0    pypi
fido                      4.2.2                    pypi_0    pypi
filetype                  1.0.13                   pypi_0    pypi
flask                     2.1.2                    pypi_0    pypi
flatten-dict              0.4.2                    pypi_0    pypi
fonttools                 4.33.3                   pypi_0    pypi
future                    0.18.2                   pypi_0    pypi
hyperlink                 21.0.0                   pypi_0    pypi
idna                      3.3                      pypi_0    pypi
incremental               21.3.0                   pypi_0    pypi
inflect                   5.6.0                    pypi_0    pypi
ipykernel                 6.15.0                   pypi_0    pypi
ipython                   8.4.0                    pypi_0    pypi
ipython-genutils          0.2.0                    pypi_0    pypi
ipywidgets                7.7.0                    pypi_0    pypi
itsdangerous              2.1.2                    pypi_0    pypi
jedi                      0.18.1                   pypi_0    pypi
jinja2                    3.1.2                    pypi_0    pypi
jmespath                  1.0.1                    pypi_0    pypi
joblib                    1.1.0                    pypi_0    pypi
json2html                 1.3.0                    pypi_0    pypi
jsonpointer               2.3                      pypi_0    pypi
jsonref                   0.2                      pypi_0    pypi
jsonschema                3.2.0                    pypi_0    pypi
jupyter                   1.0.0                    pypi_0    pypi
jupyter-client            7.3.4                    pypi_0    pypi
jupyter-console           6.4.3                    pypi_0    pypi
jupyter-core              4.10.0                   pypi_0    pypi
jupyterlab-pygments       0.2.2                    pypi_0    pypi
jupyterlab-widgets        1.1.0                    pypi_0    pypi
kiwisolver                1.4.3                    pypi_0    pypi
latexcodec                2.0.1                    pypi_0    pypi
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 12.1.0              h8d9b700_16    conda-forge
libgomp                   12.1.0              h8d9b700_16    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libzlib                   1.2.12               h166bdaf_1    conda-forge
maggma                    0.47.3                   pypi_0    pypi
markupsafe                2.1.1                    pypi_0    pypi
matminer                  0.7.4                    pypi_0    pypi
matplotlib                3.5.2                    pypi_0    pypi
matplotlib-inline         0.1.3                    pypi_0    pypi
mistune                   0.8.4                    pypi_0    pypi
mongogrant                0.3.3                    pypi_0    pypi
mongomock                 4.0.0                    pypi_0    pypi
monotonic                 1.6                      pypi_0    pypi
monty                     2022.4.26                pypi_0    pypi
mp-api                    0.24.2                   pypi_0    pypi
mpcontribs-client         4.2.10                   pypi_0    pypi
mpmath                    1.2.1                    pypi_0    pypi
msgpack                   1.0.4                    pypi_0    pypi
nbclient                  0.6.4                    pypi_0    pypi
nbconvert                 6.5.0                    pypi_0    pypi
nbformat                  5.4.0                    pypi_0    pypi
ncurses                   6.3                  h27087fc_1    conda-forge
nest-asyncio              1.5.5                    pypi_0    pypi
networkx                  2.8.4                    pypi_0    pypi
notebook                  6.4.12                   pypi_0    pypi
numpy                     1.22.4                   pypi_0    pypi
openssl                   3.0.3                h166bdaf_0    conda-forge
orjson                    3.7.2                    pypi_0    pypi
packaging                 21.3                     pypi_0    pypi
palettable                3.3.0                    pypi_0    pypi
pandas                    1.4.2                    pypi_0    pypi
pandocfilters             1.5.0                    pypi_0    pypi
paramiko                  2.11.0                   pypi_0    pypi
parso                     0.8.3                    pypi_0    pypi
pexpect                   4.8.0                    pypi_0    pypi
pickleshare               0.7.5                    pypi_0    pypi
pillow                    9.1.1                    pypi_0    pypi
pint                      0.19.2                   pypi_0    pypi
pip                       22.1.2             pyhd8ed1ab_0    conda-forge
plotly                    5.8.2                    pypi_0    pypi
prometheus-client         0.14.1                   pypi_0    pypi
prompt-toolkit            3.0.29                   pypi_0    pypi
psutil                    5.9.1                    pypi_0    pypi
ptyprocess                0.7.0                    pypi_0    pypi
pubchempy                 1.0.4                    pypi_0    pypi
pure-eval                 0.2.2                    pypi_0    pypi
pyasn1                    0.4.8                    pypi_0    pypi
pyasn1-modules            0.2.8                    pypi_0    pypi
pybtex                    0.24.0                   pypi_0    pypi
pycparser                 2.21                     pypi_0    pypi
pydantic                  1.8.2                    pypi_0    pypi
pydash                    5.1.0                    pypi_0    pypi
pygments                  2.12.0                   pypi_0    pypi
pyisemail                 1.4.0                    pypi_0    pypi
pymatgen                  2022.5.26                pypi_0    pypi
pymongo                   4.1.1                    pypi_0    pypi
pynacl                    1.5.0                    pypi_0    pypi
pyopenssl                 22.0.0                   pypi_0    pypi
pyparsing                 3.0.9                    pypi_0    pypi
pyrsistent                0.18.1                   pypi_0    pypi
python                    3.10.5          ha86cf86_0_cpython    conda-forge
python-dateutil           2.8.2                    pypi_0    pypi
python_abi                3.10                    2_cp310    conda-forge
pytz                      2022.1                   pypi_0    pypi
pyyaml                    6.0                      pypi_0    pypi
pyzmq                     23.1.0                   pypi_0    pypi
qtconsole                 5.3.1                    pypi_0    pypi
qtpy                      2.1.0                    pypi_0    pypi
readline                  8.1.2                h0f457ee_0    conda-forge
requests                  2.28.0                   pypi_0    pypi
requests-futures          1.0.0                    pypi_0    pypi
rfc3987                   1.3.8                    pypi_0    pypi
robocrys                  0.2.7                    pypi_0    pypi
ruamel-yaml               0.17.21                  pypi_0    pypi
ruamel-yaml-clib          0.2.6                    pypi_0    pypi
s3transfer                0.6.0                    pypi_0    pypi
scikit-learn              1.1.1                    pypi_0    pypi
scipy                     1.8.1                    pypi_0    pypi
send2trash                1.8.0                    pypi_0    pypi
sentinels                 1.0.0                    pypi_0    pypi
service-identity          21.1.0                   pypi_0    pypi
setuptools                62.5.0          py310hff52083_0    conda-forge
simplejson                3.17.6                   pypi_0    pypi
six                       1.16.0                   pypi_0    pypi
sniffio                   1.2.0                    pypi_0    pypi
soupsieve                 2.3.2.post1              pypi_0    pypi
spglib                    1.16.5                   pypi_0    pypi
sqlite                    3.38.5               h4ff8645_0    conda-forge
sshtunnel                 0.4.0                    pypi_0    pypi
stack-data                0.3.0                    pypi_0    pypi
starlette                 0.19.1                   pypi_0    pypi
strict-rfc3339            0.7                      pypi_0    pypi
swagger-spec-validator    2.7.3                    pypi_0    pypi
sympy                     1.10.1                   pypi_0    pypi
tabulate                  0.8.9                    pypi_0    pypi
tenacity                  8.0.1                    pypi_0    pypi
terminado                 0.15.0                   pypi_0    pypi
threadpoolctl             3.1.0                    pypi_0    pypi
tinycss2                  1.1.1                    pypi_0    pypi
tk                        8.6.12               h27826a3_0    conda-forge
tornado                   6.1                      pypi_0    pypi
tqdm                      4.64.0                   pypi_0    pypi
traitlets                 5.3.0                    pypi_0    pypi
twisted                   22.4.0                   pypi_0    pypi
typing-extensions         4.2.0                    pypi_0    pypi
tzdata                    2022a                h191b570_0    conda-forge
ujson                     5.3.0                    pypi_0    pypi
uncertainties             3.1.7                    pypi_0    pypi
urllib3                   1.26.9                   pypi_0    pypi
wcwidth                   0.2.5                    pypi_0    pypi
webcolors                 1.12                     pypi_0    pypi
webencodings              0.5.1                    pypi_0    pypi
werkzeug                  2.1.2                    pypi_0    pypi
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
widgetsnbextension        3.6.0                    pypi_0    pypi
wrapt                     1.14.1                   pypi_0    pypi
xz                        5.2.5                h516909a_1    conda-forge
yelp-bytes                0.3.0                    pypi_0    pypi
yelp-encodings            1.0.0                    pypi_0    pypi
zlib                      1.2.12               h166bdaf_1    conda-forge
zope-interface            5.4.0                    pypi_0    pypi

Yes, you need to either install mp-api inside the same environment that you launch Jupyter from (note that you can pip install inside a conda environment) or, alternatively, you need to install a kernel so that Jupyter can access your second environment, e.g.:

conda activate mpapi
pip install ipykernel
python -m ipykernel install --user --name mpapi

This will make your mpapi conda environment accessible from Jupyter.

Hope this helps! More info on this can be found online; this is not a situation unique to the mp-api.

Thank you for your reply. I think I understand the concept, but I don’t understand the details. What is the exact procedure (from creating the environment onwards? I just gave it a try using

conda create -n mpapi
conda install pip
pip install mp-api
pip install ipykernel
pip install notebook
python -m ipykernel install --user --name mpapi

… and had the same problem with mp_api not being defined. Perhaps having the exact sequence will help me understand things better.

To be honest, I don’t really understand what the “python -m ipykernel install --user --name mpapi” command does as well. The “man python” result seems to indicate that this causes python to search sys.path and run the module as a script. What is being installed when I do this? What does the --name mpapi cause python to do – does python know about the mpapi environment of conda? (although obviously I am still doing something wrong as I still have the same problem). Is there a web page somewhere that documents this behavior that I could study up upon (as I have been occasionally bitten by this problem(bug?) before. I am grateful for your help!

I was re-reading what you wrote, namely "that you must install mp-api in the same environment I will run Jupyter from and I realized some of my preconceptions must be incorrect.

I thought when I made a (new) conda environment and switched to it and then installed pip using conda that all of the packages subsequently installed by pip (or conda) would be “in the same environment”. Due to the problems I am having accessing mp_api in Jupyter, it would seem that this is not the case. I am confused though why my last example did not work. Inside (a new) mpapi environment (created within conda), I installed pip (via conda). I then via pip, I installed mp-api, ipykernel, and notebook. I would have thought that all of these would be installed in the same environment. As I mentioned in my last note, I don’t really understand what “python -m ipykernel install --user --name mpapi” installs, so I am not sure how to troubleshoot my problem, but what is being installed in a different environment in the above?