Nomad API Persitent Write Token

Hey Nomad,

Imaging there is a share where all instrument data is stored and either the experimentalist is using nomad and uploads the data by hand or every evening new entries are automatically uploaded to the right user or to a default user collecting all data in nomad which is not uploaded by an experimentalist. You can think about a service lab e.g. microscopes, where users have the option to use nomad but they dont have to.

First. How long are API tokens valid?
Further, is it possible to have API tokens which persist indefinitely and can only be used to upload data, but not to delete or edit?

Best Micha

There are multiple authentication schemes at play. There is an access token and upload token.

The regular access token (a JWT token issued by the user-management system) gives access to all functions, but has limited validity (24h) and needs to be refreshed.

The upload token is a fixed per user token. It is only available for some functions (e.g. the PUT uploads and PUT uploads/<upload_id>/raw/<path>). You can get the upload token with the access token.

Of course, we can think about a third type of token. Similar as the user-based upload token, we could have an upload-based token that users could transfer to 3rd-party apps to provide write access to individual uploads or something like that.

We have to distinguish a bit. What is managed by the user-management system (like the access token) and what is managed by NOMAD itself (the upload token). We could introduce additional API tokens at the user-management system level. These would allow to authenticate apps, but those apps would still be required to be authorised by the user, e.g. through an oauth flow to do any non public operation.

A general API token that provides access without specific user authorisation would be quite sensitive and would only make sense in an oasis context. But, technically possible of course.

Let us know, what you think and if the existing options would work for you.

Hey Markus,

do I understand correctly, the upload token already exists? Does it has limited validity too? Can it use the POST /upload endpoint too? Or what do you mean by PUT /uploads, I didnt find it in the API documentation.

If all answers to this are yes, I think this would be what is needed. So you would once create such a upload token and then all of the work a user does is decorated with that token, even if the user is not interested in using Nomad at all.

How would one get such an upload token at the moment?

Best Micha

There is are (at least) two relevant endpoints:

  • POST /uploads link – allows you to create a new upload (and provide a .zip with files at the same time)
  • PUT /uploads/<upload_id>/raw/<path> link – allows to overwrite, add new files to an existing upload

Both support the token parameter. This one is permanent, based on a secret that you can define in your oasis config nomad.yaml (services.api_secret).

I just realise that we do not have any API that would give you the token directly. Maybe we should change that. The idea would be that you can get the upload token once, e.g. using an access token. Currently you can only get it indirectly via the uploads/command-examples endpoint.

This was originally added to have a simply way to upload data via copy&paste a shell command from the GUI.

Ah ok so this upload token is not user specific?

No, it is user specific. It is generated from the user id and a non user-specific secret.

Ok, I am still unsure what the exact steps are to get this token through the API assuming I have valid user credentials.

We do not have any API that would give you the token directly at the moment. Currently you can only get it indirectly via the uploads/command-examples endpoint using the available access-token.

If you do not have an access token yet. GET auth/token can give you one for username/password-style credentials.

In the future we will extend the auth/token endpoint to also give you the upload token.

For completeness on this post. how to use the access token see: Using the APIs - NOMAD Documentation that helped me a lot. with the uploads/command-examples one still needs to extract the token since one gets only the url as:

http://nomad-oasis-url/nomad-oasis/api/v1/uploads?token=4nlllGqFSgunrlhiUFwk1w.M3h05J2qkODbDU9oThkwdsfdsgfdsgsdg