The GUI has it’s own stripped down version of the metainfo that it uses internally (nomad-FAIR/gui/src/metainfo.json). I think you will need to update it for the GUI to recognize your newly added metainfo.
If you are running the GUI through an up-to-date docker container, this file should be automatically updated. But if you are running/building the GUI with yarn, you will need to update this file manually. Updating the file should be as easy as running the following command in the nomad-FAIR repository root folder (with a python environment that has the nomad-cli tool and you latest parser version):
nomad dev metainfo > gui/src/metainfo.json
This command basically finds and outputs the latest version of the metainfo in a format that can be easily used by the GUI. Notice that you don’t have to push gui/src/metainfo.json, or gui/src/parserMetadata.json into git: they will be autogenerated during the docker build. You only need to update it manually if you are running the GUI locally e.g. for development. Otherwise your git diff looks reasonable to me.
Yeah, this is actually with the docker build so I think I don’t have my parser metainfo scheme in proper shape so it is not picked up during the docker build recipe.
The problem is that I started from the nomad-parser-example, which now I see as a bad idea. I should have probably started from some real parser as the example parser is at this point outdated (one of the issues is that the metainfo file scheme is different for the example parser and some standard parsers like vasp). The example parser has the metainfo.py file in the exampleparser folder, while all the orther parser have separate folder, like (vaspparser/metainfo) and extra init.py file, etc. I’ve tried to make the lobster parser consistent with the other real parsers but it is possible I messed up something.
If this is the case, we need to update the nomad-parser-example repository. The idea is that you should be able to use it as a basis for creating a new parser. If this is currently not the case, then we need to update it.
But yes, if you have some slight difference compared to e.g. the VASP parser, then it is possible that the lobster metainfo is not picked up correctly. Incorrect or missing __init__.py in the metainfo folder might be a probable culprit.
The thing is at this point I’ve added the _init_.py file, I’ve made the directory structure consistent with the rest of the parsers and I’ve also added
which is another thing seen in the real parsers metainfo but not in the example parser. But none of this seems to help. Could you please take a look at the current lobster code to check if you can spot anything obviously wrong with respect to the metainfo?
I had a quick look and it seems you are missing this line at the end of lobsterparser/metainfo.py:
m_package.__init_metainfo__()
With this added, I was able to get the lobster metainfo printed out with the CLI command nomad dev metainfo, and thus it should also get correctly into your docker image.
Hopefully this was the last missing piece and you can get Lobster correctly running
Hm, I spoke too fast, the generated gui/src/metainfo.json has the x_lobster_* stuff but it still doesn’t show up in the gui.
Anyway, at this point the parser is mostly finished. I will still add some smaller features, but the main missing thing is at this point the workflow stuff. Any progress on that front? Also about the upstreaming, should we do it now or later?
I assume you want to discuss how to have parsing after the related calculations have parsed? Or do you refer to how represent workflows in the metainfo?
For the first thing. We plan to update our processing to support multiple parsing stages. A parser decide if it want to run first or in a later stage. This will provide guarantees that certain parsers run strictly after others.
However, this will take a while and will be entangled in other changes to the processing. It will be part of the next major release this fall. We also plan to provide parsers with an interface that allows easy access to already parsed entries in the same upload. This should help to identify and reference other entries.
You have to decide yourself, if you want to add your stuff to the existing hack or if you want to wait.
The metainfo problem was the browser cache after all. Refreshing the window was not enough but I can see the lobster specific metainfo if I use a new private window. So thanks for all the help.
Regarding the rest, yeah I can wait. But as I indicated earlier I’m finishing my current postdoc stay in August and from Autumn I’m in another department, so yeah who knows how I can follow up long-term. I’m still interested in having working parsers for codes I use regularly (like Wien2k) but LOBSTER specifically is something which is heavily used at my current department but not by myself. My goal right now is to get the parser to the state that it can be integrated to the nomad-FAIR somehow so that when there is some rewrite/restructuring happening it doesn’t get left behind.
Maybe at this point it could be helpful if the parser could be deployed in the central NOMAD so I can potentially get better testing and fix issues which don’t show up in our oasis?
We can include you parser in our builds. Our release cycles will be slower than what you can do with your Oasis of course. I opened this issue (Include lobster parser (#546) · Issues · nomad-lab / nomad-FAIR · GitLab). For better long term support, please consider transferring your project to the nomad-coe organisation.
