Keeping workers alive in multiple diamond workflow

FireWorks
1

Hi,

I’m trying to create a workflow that features multiple fork/joins, so occasionally the workflow only has a single Firework to do with all other Fireworks being dependent on the single Firework. When running this with rlaunch multi, workers will search for a FW, see none currently ready and terminate. This is problematic as it forces the execution of the Workflow to become serial after the first join in the workflow.

Is there an option which would allow the Workers to recognise that they are waiting for a currently active dependency and remain alive? Ie sleep for x seconds and recheck for a FW to run?

Thanks,

Richard

2

Did you already try

rlaunch multi --nlaunches infinite

See “rlaunch multi -h” for more info

···

On Wednesday, August 16, 2017 at 12:30:18 PM UTC-7, Richard Gowers wrote:

Hi,

I’m trying to create a workflow that features multiple fork/joins, so occasionally the workflow only has a single Firework to do with all other Fireworks being dependent on the single Firework. When running this with rlaunch multi, workers will search for a FW, see none currently ready and terminate. This is problematic as it forces the execution of the Workflow to become serial after the first join in the workflow.

Is there an option which would allow the Workers to recognise that they are waiting for a currently active dependency and remain alive? Ie sleep for x seconds and recheck for a FW to run?

Thanks,

Richard

3

Hi

Yeah this wasn’t a problem in the end. When a worker doesn’t see an available firework it sleeps, and my toy example was so small that it looked like everything was proceeding in serial.

Thanks anyway

Richard

···

On Wednesday, 16 August 2017 23:22:43 UTC+1, Anubhav Jain wrote:

Did you already try

rlaunch multi --nlaunches infinite

See “rlaunch multi -h” for more info

On Wednesday, August 16, 2017 at 12:30:18 PM UTC-7, Richard Gowers wrote:

Hi,

I’m trying to create a workflow that features multiple fork/joins, so occasionally the workflow only has a single Firework to do with all other Fireworks being dependent on the single Firework. When running this with rlaunch multi, workers will search for a FW, see none currently ready and terminate. This is problematic as it forces the execution of the Workflow to become serial after the first join in the workflow.

Is there an option which would allow the Workers to recognise that they are waiting for a currently active dependency and remain alive? Ie sleep for x seconds and recheck for a FW to run?

Thanks,

Richard