I’m trying to update our base installation on nomad.warwick.ac.uk, where Docker is not allowed by ITS.
As far as I see, the base linux instructions have not been touched since the beta version. I have tried to use the changes in the docker sections about nomad.yaml and nginx.conf but I’m not sure if all changes are correct for our case.
Steps
I have tried to start from scratch as far as possible (e.g. deleted indices of earlier attempts from elasticsearch), so we can use this to update the instruction as well:
$ mkdir v1.1
$ cd v1.1
$ virtualenv -p `which python3` nomadpyenv
$ source nomadpyenv/bin/activate
$ curl 'https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/archive/v1.1.0/nomad-FAIR-v1.1.0.tar.gz' -o nomad-lab.tar.gz
$ pip install install --upgrade pip
$ pip install nomad-lab.tar.gz[all]
$ vi nomad.yaml
$ nomad admin ops nginx-conf > nginx.conf
$ vi nginx.conf
$ cp nginx.conf /etc/nginx/default.d/nomad.conf
$ sudo service nginx restart
$ nomad admin ops gui-config
Traceback (most recent call last):
File "/home/admin/nomad-oasis/v1.1/nomadpyenv/bin/nomad", line 8, in <module>
sys.exit(run_cli())
File "/home/admin/nomad-oasis/v1.1/nomadpyenv/lib/python3.7/site-packages/nomad/cli/cli.py", line 71, in run_cli
return cli(obj=POPO()) # pylint: disable=E1120,E1123
File "/home/admin/nomad-oasis/v1.1/nomadpyenv/lib/python3.7/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/home/admin/nomad-oasis/v1.1/nomadpyenv/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/admin/nomad-oasis/v1.1/nomadpyenv/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/admin/nomad-oasis/v1.1/nomadpyenv/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/admin/nomad-oasis/v1.1/nomadpyenv/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/admin/nomad-oasis/v1.1/nomadpyenv/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/admin/nomad-oasis/v1.1/nomadpyenv/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/admin/nomad-oasis/v1.1/nomadpyenv/lib/python3.7/site-packages/nomad/cli/admin/admin.py", line 241, in gui_config
shutil.copytree(gui_folder, run_gui_folder)
File "/home/admin/.pyenv/versions/3.7.9/lib/python3.7/shutil.py", line 318, in copytree
names = os.listdir(src)
FileNotFoundError: [Errno 2] No such file or directory: '/home/admin/nomad-oasis/v1.1/nomadpyenv/lib/python3.7/site-packages/nomad/app/flask/static/gui'
Not sure why it’s missing.
The current version of the nomad.yaml:
services:
api_host: 'nomad.warwick.ac.uk'
api_base_path: '/nomad-oasis'
https: True
https_upload: True
admin_user_id: 'admin'
oasis:
is_oasis: true
uses_central_user_management: true
north:
jupyterhub_crypt_key: '79f88d1ad2c7d97bbe2df4bbaf7d3977ee3a71a1e6c7759294e30bd44022276f'
hub_ip_connect: '127.0.0.1' # 172.17.0.1 is docker-specific?
meta:
deployment: 'NOMAD Oasis UoW'
deployment_id: 'nomad.warwick.ac.uk'
maintainer_email: '[email protected]'
mongo:
db_name: nomad_v1
elastic:
entries_index: nomad_v1_entries
materials_index: nomad_v1_materials
# client.url entry missing in v1.1 docs
# keycloak entries missing in v1.1 docs
# rabbitmq entries missing in v1.1 docs
The current version of /etc/nginx/default.d/nomad.conf (server entry removed, auto-generated hostnames replaced with localhost):
location / {
proxy_pass http://localhost:8000;
}
location ~ /nomad-oasis\/?(gui)?$ {
rewrite ^ /nomad-oasis/gui/ permanent;
}
location /nomad-oasis/gui/ {
proxy_intercept_errors on;
error_page 404 = @redirect_to_index;
proxy_pass http://localhost:8000;
}
location @redirect_to_index {
rewrite ^ /nomad-oasis/gui/index.html break;
proxy_pass http://localhost:8000;
}
location ~ \/gui\/(service-worker\.js|meta\.json)$ {
add_header Last-Modified $date_gmt;
add_header Cache-Control 'no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0';
if_modified_since off;
expires off;
etag off;
proxy_pass http://localhost:8000;
}
location ~ /api/v1/uploads(/?$|.*/raw|.*/bundle?$) {
client_max_body_size 35g;
proxy_request_buffering off;
proxy_pass http://localhost:8000;
}
location ~ /api/v1/.*/download {
proxy_buffering off;
proxy_pass http://localhost:8000;
}
In the old v0.8 we used a run script:
source /home/admin/$VIRTUALENV/bin/activate
gunicorn "${params[@]}" -b 0.0.0.0:8000 nomad.app:app >> nomad.app.log 2>&1 &
celery worker -l info -A nomad.processing -Q celery,calcs,uploads >> nomad.processing.log 2>&1 &
In the NOMAD tutorial last week, @mscheidgen noted that we should use the UvicornWorker class instead:
gunicorn "${params[@]}" --worker-class=uvicorn.workers.UvicornWorker -b 0.0.0.0:8000 nomad.app.main:app >> nomad.app.log 2>&1 &
But the v1.1 documentation suggests using uvicorn (this was updated even in the bare Linux section 
):
python -m nomad.cli admin ops gui-config
python -m uvicorn --host 0.0.0.0 nomad.app.main:app
Since this seems to depend on the failing gui-config command, I didn’t go further.
Questions
- Are the
client,keycloak, andrabbitmqentries still necessary in our case? - Why is
…/guinot found? - Should we use
uvicornorgunicorn?
Thanks for your help!