Issue with running app container (NOMAD oasis)

NOMAD and FAIRmat
1

Dear NOMAD-Team,

I’m writing because our research group is trying to set up a NOMAD Oasis for testing, and I’m having some trouble with the initial installation. I followed the instructions in the “How to install an Oasis” guide on your website (https://nomad-lab.eu/prod/v1/staging/docs/oasis/install.html), but I’ve run into some issues.

The problem I’m facing is that the “nomad_oasis_app” container doesn’t change it health state from “started” to “healthy.” This is causing the “nomad_oasis_proxy” container to not start as expected, because of the condition in the docker-compose file. It appears that the application within the app container is not launching correctly. The last log entry of the app container is “Waiting for application startup”, but then nothing further happens and I don’t know what I’m doing wrong. I hope that perhaps this issue has occurred before, and you can assist me.

I’ve used the the nomad.yaml, docker-compose.yaml, and nginx.conf files from your basic zip folder (https://nomad-lab.eu/prod/v1/staging/docs/assets/nomad-oasis.zip). I only made a few changes in the docker-compose file: I took the “stable” Nomad image instead of the “latest” and adjusted the docker group to match my computer. Everything else in the configuration is as per your instructions.

Below you find the logs from both the “app” and “worker” containers. If you need more information or have any ideas to help me troubleshoot this issue, please let me know. Thanks a lot in advance for your help.

  • nomad_oasis_app logs

2023-10-10 15:09:05 config key does not exist: logstash_enable
2023-10-10 15:09:24 INFO: Started server process [7]
2023-10-10 15:09:24 INFO: Waiting for application startup.

  • nomad_oasis_worker logs

2023-10-10 15:00:25 config key does not exist: logstash_enable
2023-10-10 15:01:23
2023-10-10 15:01:23 -------------- celery@2ef458c23d7d v5.2.7 (dawn-chorus)
2023-10-10 15:01:23 — ***** -----
2023-10-10 15:01:23 – ******* ---- Linux-6.3.13-linuxkit-x86_64-with-glibc2.36 2023-10-10 13:01:23
2023-10-10 15:01:23 - *** — * —
2023-10-10 15:01:23 - ** ---------- [config]
2023-10-10 15:01:23 - ** ---------- .> app: nomad.processing:0x7ff993d7c220
2023-10-10 15:01:23 - ** ---------- .> transport: amqp://rabbitmq:**@rabbitmq:5672//
2023-10-10 15:01:23 - ** ---------- .> results: disabled://
2023-10-10 15:01:23 - *** — * — .> concurrency: 4 (prefork)
2023-10-10 15:01:23 – ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
2023-10-10 15:01:23 — ***** -----
2023-10-10 15:01:23 -------------- [queues]
2023-10-10 15:01:23 .> celery exchange=celery(direct) key=celery
2023-10-10 15:01:23
2023-10-10 15:01:23
2023-10-10 15:01:23 [tasks]
2023-10-10 15:01:23 . nomad.app.resources.routers.resources.retrieve_resources
2023-10-10 15:01:23 . nomad.processing.base.proc_task

2

Thanks for your post and feedback.

Normally the INFO: Waiting for application startup. would be followed by an “startup complete”. There is a lot of initial stuff happening when the app gets booted and it might be too long for the healthcheck to pass.

Can you check the healthcheck in the app service and increase the values for start_period to 60s or something like this.

You can also try to disable a few plugins in the nomad.yaml. Especially the nexus parse is quite heavy on start-up. You could add this to configs/nomad.yaml

plugins:
  exclude:
    - parsers/nexus
3

Thank you for the quick response. The suggestion to disable a few plugins was great. I followed your advice and excluded the Nexus Parsers and suddenly the startup worked without any problems.

Thanks again for the quick and very helpful response.