Problem to add python plug-in into NOMAD Oasis with docker

NOMAD and FAIRmat
1

Hello,

We are having problem to add new python plugin into NOMAD.
In https://nomad-lab.eu/prod/v1/staging/docs/plugins/plugins.html, it is described how to specify a plugin in the nomad.yaml file. In the Dockerfile, we added the lines

COPY new_parsers/nomadapbs ./new_parsers/nomadapbs
RUN pip install ./new_parsers/nomadapbs
COPY nomad.yaml ./

and the nomad.yaml file (removing the **/nomad.yaml entry in the .dockerignore file)

normalize:
normalizers:
    include:
      - MetainfoNormalizer
plugins:
    include: 'parsers/apbs'
    options:
      parsers/apbs:
       python_package: nomadapbs

but somehow the parser is not recognised. We upload data into NOMAD-OASIS and data are not processed because parser is not recognized.
We are using the 1.2.0 version.
Also, we don’t understand how should multiple plugins be listed?

Any help would me appreaciated.

this is the relevant output from the docker log file:

#39 5.304 cannot set config setting plugins_options={‘parsers/apbs’: {‘python_package’: ‘nomadapbs’}}: 3 validation errors for ParsingModel[Dict[str, Union[nomad.config.plugins.Schema, nomad.config.plugins.Parser]]]
#39 5.304 root → parsers/apbs → name
#39 5.304 field required (type=value_error.missing)
#39 5.304 root → parsers/apbs → name
#39 5.304 field required (type=value_error.missing)
#39 5.304 root → parsers/apbs → parser_class_name
#39 5.304 field required (type=value_error.missing)

2

Hi srdjan125,
There seems to be an error in the indentation of your nomad.yaml file. The plugins block should not be indented under normalize.

normalize:
  normalizers:
    include:
      - MetainfoNormalizer
plugins:
  include: 'parsers/apbs'
  options:
    parsers/apbs:
      python_package: nomadapbs

Let me know if that helps. Otherwise you could maybe post the content of your nomad_plugin.yaml file.

To include multiple plugins you just need to list all of them:

plugins:
  include:
    - 'schemas/first_plugin'
    - 'schemas/second_plugin'
  options:
    schemas/first_plugin:
      python_package: first_plugin_package
    schemas/second_plugin:
      python_package: second_plugin_package
3

Hello,

Thank you for answer @hampusnasstrom . Indentation we have is correct, it was just error during copying here. Second answer (how to include several parsers) is helpful but it cannot resolve first issue - nomad cannot recognize new parsers. Is the problem in some options in docker file?

This is the content of nomad_plugin.yaml file:

plugin_type: parser
name: parsers/apbs
description: |
        This is a simple hello world parser. This is meant as a template. Fork the github
        project to create your own parser.
mainfile_name_re: ^.*\.in$
parser_class_name: nomadapbs.APBSParser
code_name: APBS
4

Hey, I am not from Nomad, but I ran plugins successfully already. My parser plugin is e.g. here: GitHub - RoteKekse/nomad-chemical-energy-parser

i didnt do it with pip. i just put the folder.

the important point is that you need to add the path to the container, which i do like this: How to develop, publish, and install plugins - Documentation

the really important point is and i this can create the issue you have, is that you change the ownership of the plugin folder to 1000:1000, i think it is missing in the docs.

sudo chown -R 1000:1000 <path-to-plugin>

another issue could be that you didnt share the right level. you can go inside the nomad_oasis_app container to check in the plugins folder if everything looks correct.

i hope this helps

best Micha

5

@MichaelGoette thanks for your advice! it helped and it is finally working

1 Like
6

Thanks @MichaelGoette for helping out! We will look into the ownership problem: this needs to be documented or otherwise solved.

Mounting the plugin as a docker volume is simpler at least during development than doing COPY.