Hello,
I am able to use atomate2 to run jobs locally on a computing cluster, but I want to do all the setup and processing on my desktop and only use the cluster for calculations. My reason for doing this is because not all of the nodes can access an external database, and I also don’t want to leave any scripts running on the login node.
This is what my ideal setup looks like:
- create atomate2 workflow on my desktop
- run jobs on cluster
- store and process data on desktop
- submit next step in workflow to cluster
etc.
This is what I have so far:
- create atomate2 workflow on my desktop
- convert to fireworks
- upload job to mongodb atlas
- download job to cluster login node from mongodb
- send to queue in offline mode
After entering the queue, the job fails. It looks like it still tries to query mongodb even though it’s in offline mode.
Is this workflow possible with atomate2? I looked in the documentation for atomate2 and jobflow, but the part about running remotely is missing.
Thank you.