atomate 0.7.8+ throws json error with structure optimization

Summary

When trying to run a structure optimization, newer atomate versions throw monty error when doing loadfn(zpath("custodian.json")) during the FWAction update.

Test code used to generate the wf:

from pymatgen import MPRester
from atomate.vasp.workflows.presets.core import wf_structure_optimization
structure = mpr.get_structure_by_material_id(‘mp-1’)
wf = wf_structure_optimization(structure)
lpad.add_wf(wf)

``

Stacktrace from running

/projects/matqm/synthesis/cenv/lib/python3.4/site-packages/pymatgen/init.py:89: UserWarning:
Pymatgen will drop Py2k support from v2019.1.1. Pls consult the documentation
at https://www.pymatgen.org for more details.
at https://www.pymatgen.org for more details.""")
/projects/matqm/synthesis/cenv/lib/python3.4/site-packages/pymatgen/io/cif.py:44: UserWarning: Please install optional dependency pybtex if youwant to extract references from CIF files.
warnings.warn(“Please install optional dependency pybtex if you”
ERROR:custodian.custodian:
{ ‘actions’: [ { ‘action’: { ‘_set’: { ‘ISYM’: 0}},
‘dict’: ‘INCAR’}],
‘errors’: [ ‘rot_matrix’],
‘handler’: <custodian.vasp.handlers.VaspErrorHandler object at 0x2b80de941ac8>}
/projects/matqm/bin/vasp.std: no process killed
Traceback (most recent call last):
File “/projects/matqm/synthesis/cenv/lib/python3.4/site-packages/fireworks/core/rocket.py”, line 262, in run
m_action = t.run_task(my_spec)
File “/projects/matqm/synthesis/cenv/lib/python3.4/site-packages/atomate/vasp/firetasks/run_calc.py”, line 207, in run_task
return FWAction(stored_data=loadfn(zpath(“custodian.json”)))
File “/projects/matqm/synthesis/cenv/lib/python3.4/site-packages/monty/serialization.py”, line 82, in loadfn
return json.load(fp, *args, **kwargs)
File “/opt/python/lib/python3.4/json/init.py”, line 268, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File “/opt/python/lib/python3.4/json/init.py”, line 312, in loads
s.class.name))
TypeError: the JSON object must be str, not ‘bytes’
INFO:rocket.launcher:Rocket finished

``

The same problem persists when using any structure.

The end of this stacktrace makes it appear the problem is with the JSON library. After looking into it some more, it is a problem with

return FWAction(stored_data=loadfn(zpath(“custodian.json”)))

``

particularly that loadfn doesn’t seem to work on a zipped json file?

Using atomate 0.7.7 with an identical workflow and configuration works fine. It appears the problem might be related to Github issue #231.

System

  • Distribution: CentOS release 6.6 (Final)
  • Linux kernel 2.6.32-696.18.7.el6.x86_64
  • Python 3.4.2

Python package versions from pip list

Package|Version

:-----:|:-----:

atomate|0.8.4

certifi|2018.11.29

chardet|3.0.4

Click|7.0

custodian|2018.8.10

cycler|0.10.0

decorator|4.3.0

FireWorks|1.8.2

Flask|1.0.2

flask-paginate|0.5.1

gunicorn|19.9.0

idna|2.7

itsdangerous|1.1.0

Jinja2|2.10

kiwisolver|1.0.1

MarkupSafe|1.1.0

matplotlib|2.2.3

monty|1.0.4

mpmath|1.0.0

networkx|2.2

numpy|1.15.4

palettable|3.1.1

pandas|0.22.0

pip|18.1

pydash|4.7.4

PyDispatcher|2.0.5

pymatgen|2018.11.30

If anyone else has had similar issues, I’ll make a bug report on the repo, and we should look into why the tests are not catching this problem.

This problem might be because of a monty (or some other package) versioning issue. Not saying it is 100% to do with the atomate code, just would be good to get another set of eyes on this.

After trying this on Cori (and also on Alex G’s setup), this issue is a monty issue specific to Python 3.4, not Python 3.6+

So for most people it will be a non issue.

**specifically the json module of Python 3.4

···

On Monday, December 10, 2018 at 5:37:17 PM UTC-8, [email protected] wrote:

After trying this on Cori (and also on Alex G’s setup), this issue is a monty issue specific to Python 3.4, not Python 3.6+

So for most people it will be a non issue.

Hi Alex

I guess this is an upstream problem with monty. You could try asking monty to support Py3.4 if desired.

In the meantime I will update the atomate docs so it’s clear that only Py3.6 and higher are supported

···

On Monday, December 10, 2018 at 5:37:44 PM UTC-8, [email protected] wrote:

**specifically the json module of Python 3.4

On Monday, December 10, 2018 at 5:37:17 PM UTC-8, [email protected] wrote:

After trying this on Cori (and also on Alex G’s setup), this issue is a monty issue specific to Python 3.4, not Python 3.6+

So for most people it will be a non issue.