Hi,
I am working on a demo for our institute to show how nomad could help to document experimental procedures and improve FAIR-compliance.
The specific use-case is a PLD lab. The idea is to showcase the user-experience: a researcher creates a new sample, does a measurement, registers it in nomad, adds a reviewer and does some additional analysis using a jupyter notebook.
I have tried several approaches, but I haven’t been able to connect everything and make it all work.
The best I currently have:
An XRD-measurement file (.brml), which is processed to a nice ELNXRayDiffraction entry.
A .csv and a schema that should (but isn’t) populated by the table.
Hi Tim, thanks for reaching out and sorry for the delay. It took me sometime to figure out the underlying issues. Generally speaking, using tabular annotations to parse data from csv/excel files is a bit tricky to get right. I will give the fixes for your current use-case followed by some recommendations.
In MandatoryGeometry, the base_sections used, “nomad.datamodel.metainfo.basesections.Section”, does not exist. Use “nomad.datamodel.data.ArchiveSection” instead.
In SputterDeposition, as data_file is a quantity, it should be defined under definitions.sections.SputterDeposition.quantities, rather than `definitions.sections.SputterDeposition`. Shifting the indentation by 2 spaces in the right should fix it.
Defaults provided for float type quantities should be int or float, not string. These should be fixed for MandatoryGeometry.diameter, SputterSampleParameters.distance_to_source, SputterStep.duration
Finally, here’s some very subtle issues related to TableData.
In SputterDeposition, the following order of base_sectionsresult in expected behaviour:
After making these fixes, I was able to create an entry with the custom YAML schema, upload the csv, and successfully parse the data into the entry upon processing.
For a prototype as in your case, using the tabular annotations might be a good idea. But they are hard to get right by a normal user who simply wants to upload and parse data. In a production setting, I would suggest setting up plugin entry points for schema and using the schema normalizemethod to parse custom csv files. This would look something like:
class SputterDeposition(PhysicalVaporDeposition, EntryData):
…
def normalize(self, archive: 'EntryArchive', logger: 'BoundLogger') -> None:
"""
Redefining the `normalize` method to read the data csv file and
populate the quantities. This method is called when the entry
is processed/saved.
"""
super().normalize(archive, logger)
data_dict = {}
if self.data_file is not None:
data_dict = my_read_csv(self.data_file, archive, logger)
if data_dict:
self.write_data(data_dict, logger)
# populate quantities and subsections with the extracted data