Workflow with local and queue jobs both

FireWorks
1

Hello,
I’m just starting to use Fireworks and I have a workflow containing tasks that need to be run locally and some need to be submitted to slurm.
I’m able to launch workflows entirely locally using rlaunch or submitting each firework on slurm using qlaunch but not a mix of both using a queue.

An example workflow is defined below:

fws:
- fw_id: 1
  spec:
    _tasks:
    - _fw_name: ScriptTask
      script: echo 'Task 1, NOT SLURM' > TASK1.txt
- fw_id: 2
  spec:
    _tasks:
    - _fw_name: ScriptTask
      script: echo 'TASK 2 , SLURM' > TASK2.txt
- fw_id: 3
  spec:
    _tasks:
    - _fw_name: ScriptTask
      script: echo 'Task 3, NOT SLURM' > TASK3.txt
links:
  1:
  - 2
  2:
  - 3
metadata: {}

I tried specifying the _fworker key on the tasks that aren’t supposed to be submitted to the queue but that didn’t work and qlauncher submitted my first job to slurm.
Also, I’m not sure how to define two workers on the same machine, i.e. one to run locally other to submit jobs to slurm.

Thanks

2

Hi

Note that you can only specify the worker at the “Firework” level and not at the task level. Since you have just 1 task per Firework it doesn’t matter in this case.

You want to do the following:

  • In the “spec” part of each firework, add another entry called “_fworker”. For example:
fws:
- fw_id: 1
  spec:
    _fworker: notslurm
    _tasks:
    - _fw_name: ScriptTask
      script: echo 'Task 1, NOT SLURM' > TASK1.txt
- fw_id: 2
  spec:
    _fworker: slurm
    _tasks:
    - _fw_name: ScriptTask
      script: echo 'TASK 2 , SLURM' > TASK2.txt
- fw_id: 3
  spec:
    _fworker: notslurm
    _tasks:
    - _fw_name: ScriptTask
      script: echo 'Task 3, NOT SLURM' > TASK3.txt
links:
  1:
  - 2
  2:
  - 3
metadata: {}

Next, set up two my_fworker.yaml files - e.g., my_fworker_slurm.yaml and my_fworker_notslurm.yaml. They should look like:

my_fworker_slurm.yaml

name: slurm
category: ''
query: '{}'

my_fworker_notslurm.yaml

name: notslurm
category: ''
query: '{}'

Finally, you want to run the qlaunch command with the first of these, and the rlaunch command with the second of these:

qlaunch -w path/to/my_fworker_slurm.yaml <OTHER_STUFF>
rlaunch -w path/to/my_fworker_notslurm.yaml <OTHER_STUFF>

If all goes as expected, the qlaunch command will only pull fw 2 and the rlaunch command will pull 1 and 3. Note that you might need to run rlaunch in the infinite loop mode so it doesn’t just quit after finishing fw 1.

3

also, the relevant docs for any others interested (as there are more options for achieving this kind of behavior) are at:

https://materialsproject.github.io/fireworks/controlworker.html

under the heading: Controlling the Worker that executes a Firework

4

Thank You. I just had to run rlaunch first and then qlaunch because of the dependencies and then it worked.
Also, what I did wrong was to assume that qlaunch will launch the fireworks according to the _fworker defined but the queue launcher can only launch queue jobs.

5

Ok great

Note that you can have qlaunch and rlaunch running simultaneously, e.g. by using the “&” character at the end of the command as in standard Linux usage

e.g.

qlaunch -w path/to/my_fworker_slurm.yaml rapidfire --nlaunches infinite &
rlaunch -w path/to/my_fworker_slurm.yaml rapidfire --nlaunches infinite &