Multiple fireworks and spec

FireWorks
1

Hi

My workflows consist of multiple fireworks, each firework containing only a single task. I’m still learning and figuring things out - so I’m not 100% totally sure I’m doing this the right way.

Currently I need to be able to pass a specific parameter/spec to a firework which only happens to be about the 5th firework in my workflow. At the moment I’d simply do something like this:

firework5 = Firework(FireworkFiveTask(), spec={‘new_param’: ‘123’}, parents=[firework4], name=‘Firework 5’, fw_id=5)

The problem with this is that the new spec defined for this firework will now forget any previous spec information that was passed from firework3 or firework2 and so on.

Is there a way that I can define a spec upfront which carries it’s values throughout each firework in my workflow?

I’ve noticed this in the docs:

update_spec: (dict) A data dict that will update the spec for any remaining FireTasks and the following Firework. Thus, this parameter can be used to pass data between FireTasks or between FireWorks. [1]

Which leads me to believe that the spec data is only passed to the following firework, once. It doesn’t remain constant throughout the many fireworks in my workflow.

Is this correct?

If so, is there anyway that I could give my workflow a spec upfront which gets passed down to each and every firework while updating the spec at the same time?

eg.

firework = Firework(Firework(), spec={‘initial_spec_data’: initial_spec_data}, name=‘Firework’, fw_id=1)

firework1 = Firework(Firework1(), parents=[firework], name=‘Firework1’, fw_id=2)

firework2 = Firework(Firework2(), parents=[firework1], name=‘Firework2’, fw_id=3) # and still have access to ‘initial_spec_data’ in the fw spec here

[1] https://pythonhosted.org/FireWorks/guide_to_writing_firetasks.html

Thanks!

2

I don’t know if I completely understand - why not just do this up front?

firework = Firework(Firework(), spec={‘initial_spec_data’: initial_spec_data}, name=‘Firework’, fw_id=1)

firework1 = Firework(Firework1(), spec={‘initial_spec_data’: initial_spec_data}, parents=[firework], name=‘Firework1’, fw_id=2)

firework2 = Firework(Firework2(), spec={‘initial_spec_data’: initial_spec_data}, parents=[firework1], name=‘Firework2’, fw_id=3)

Note that you are correct that passing the information dynamically (via update_spec) only updates the first level of children. There is no option to pass the information to all downstream children without passing at each step. Note that if you could actually hack this by using the “_add_launchpad_and_fw_id” reserved keyword set to True in the spec. Then you will have access to the LaunchPad and fw_id inside your FireTask. You could use this information to query for all the downstream fireworks and update their spec via the LaunchPad function. But, it is more of a hack.

Best,

Anubhav

···

On Thu, Mar 10, 2016 at 9:14 PM, Zachary Ball [email protected] wrote:

Hi

My workflows consist of multiple fireworks, each firework containing only a single task. I’m still learning and figuring things out - so I’m not 100% totally sure I’m doing this the right way.

Currently I need to be able to pass a specific parameter/spec to a firework which only happens to be about the 5th firework in my workflow. At the moment I’d simply do something like this:

firework5 = Firework(FireworkFiveTask(), spec={‘new_param’: ‘123’}, parents=[firework4], name=‘Firework 5’, fw_id=5)

The problem with this is that the new spec defined for this firework will now forget any previous spec information that was passed from firework3 or firework2 and so on.

Is there a way that I can define a spec upfront which carries it’s values throughout each firework in my workflow?

I’ve noticed this in the docs:

update_spec: (dict) A data dict that will update the spec for any remaining FireTasks and the following Firework. Thus, this parameter can be used to pass data between FireTasks or between FireWorks. [1]

Which leads me to believe that the spec data is only passed to the following firework, once. It doesn’t remain constant throughout the many fireworks in my workflow.

Is this correct?

If so, is there anyway that I could give my workflow a spec upfront which gets passed down to each and every firework while updating the spec at the same time?

eg.

firework = Firework(Firework(), spec={‘initial_spec_data’: initial_spec_data}, name=‘Firework’, fw_id=1)

firework1 = Firework(Firework1(), parents=[firework], name=‘Firework1’, fw_id=2)

firework2 = Firework(Firework2(), parents=[firework1], name=‘Firework2’, fw_id=3) # and still have access to ‘initial_spec_data’ in the fw spec here

[1] https://pythonhosted.org/FireWorks/guide_to_writing_firetasks.html

Thanks!

–

You received this message because you are subscribed to the Google Groups “fireworkflows” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

Visit this group at https://groups.google.com/group/fireworkflows.

To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/a966f7cb-e967-448c-ad47-680df863f5bd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.