Troubles to see the plugin content in the 'NOMAD Oasis with plugins' tutorial

Hello !

I am working in a science project as a data manager and we will use a NOMAD Oasis as a common local repository to later publish entries on the central database but we need to develop new parsers to deal with unsupported file formats.
To this extent, I am investigating to install plugins on an NOMAD Oasis following both the written documentation (hot to write a parser, hot to install plugins) and the tutorial video on Youtube.

I did already successfully create an Oasis on my computer using the nomad-oasis.zip file to make some upload and YAML schema tests but now, I am using the nomad-oasis-with-plugins.zip file to test an Oasis with plugins.
However, after following the instructions in the tutorial, the Oasis launches correctly (though really slowly compared to an Oasis without plugins) but I can’t find the designed schema (ExampleSection) or parse successfully the test file as shown in the video.
To explain a bit more: I work with Ubuntu 20.04, I did create a virtual environment with Python 3.9.19 (installed via Pyenv) and pip installed nomad-lab on it. Then, I followed the installation steps found in the documentation (changing the ownership of the files and exporting the COMPOSE_FILE and the PYTHONPATH ) but it seems that the content of the plugins is not visible on the UI.

As a test, I tried the command nomad parse tests/data/test.archive.yaml --show-archive as demonstrated in the video and it prints the same result, however with an additional error message displayed before the archive is printed. The error message is below:

cannot set config setting plugins_options={'schemas/example': {'python_package': 'nomadschemaexample'}}: 3 validation errors for ParsingModel[Dict[str, Union[nomad.config.plugins.Schema, nomad.config.plugins.Parser]]]
__root__ -> schemas/example -> name
  field required (type=value_error.missing)
__root__ -> schemas/example -> name
  field required (type=value_error.missing)
__root__ -> schemas/example -> parser_class_name
  field required (type=value_error.missing)

I don’t really understand why the content of the plugins is not available on the UI as I followed the different installation steps.
However, I wanted to have your insights on a specific point. I am working on my professional computer that lives in a bigger installation and it matters when I have to change the owner of the files in the zip files. The command line says sudo chown -R 1000 .volumes but in my facility, the user 1000 is already taken and has different restrictions than me as a user. For instance, I can’t save a modification in a Python file owned by this owner without sudo.
I don’t see any problems with changing the owner to 1000 for .volumes, though.

So I wonder if changing the ownership of the plugin folders to 1000 (and then adding new restrictions to them in my case) could have an impact on the imports of the plugins’ content ?

I know that the problem is not that well-defined here but I don’t really know where the things are not working correctly…
Anyway, thanks in advance for your consideration !
Guillaume Gaisné
European Synchrotron Radiation Facility

A small update on this topic.

I decided to try to make it work on a Windows machine, to test without the user 1000 potential issue.
In the end, the result is the same as on my Linux machine.
So now, it is on a Windows 11 Pro, version 23H2, 64-bit with Docker Desktop v4.29.0 installed.

I first test an Oasis without plugins, to see if everything was working and it did, the docker compose pull and up -d worked correctly and the GUI showed up nicely.
I then deleted every image and volume and installed the nomad-oasis-with-plugins following the command.
I adapted the export COMPOSE_FILE=docker-compose.yaml:docker-compose.plugins.yaml command to fit the Windows command line, so set COMPOSE_FILE=docker-compose.yaml;docker-compose.plugins.yaml.
The pull and up -d command worked correctly afterwards, so as the GUI but I have the exact same behaviour as with Linux:
- the Oasis is really slowed down (it takes around 10s to access the upload page once I click ‘Create new upload’)
- following the tutorial video, I can’t find the ‘ExampleSection’ schema when I try to create an entry with a schema using ‘Create from schema’.

Hope it can help to narrow the problem a bit more
Guillaume Gaisné
ESRF
Thanks for your consideration