Results from previous Firework's mod_spec lost after detour

FireWorks
1

In this example, the final result of the meep list is ['t5', 't6', 't3', 't4'], but I expect it to be ['t1', 't2', 't5', 't6', 't3', 't4']. I think what Change execution order of FWActions · Issue #430 · materialsproject/fireworks · GitHub and Propagation of FWAction's only happens for last Firetask in Firework · Issue #431 · materialsproject/fireworks · GitHub are saying is that if I pass propagate=True in the action, this should work, but it still doesn’t; I’m not sure if they’re related.

from fireworks import Firework, LaunchPad, Workflow
from fireworks.core.firework import FiretaskBase, FWAction
from fireworks.core.rocket_launcher import rapidfire
from fireworks.utilities.fw_utilities import explicit_serialize


@explicit_serialize
class PrintTask(FiretaskBase):
    optional_params = ["optparam"]

    def run_task(self, fw_spec) -> FWAction:
        print(f"PrintTask::self: {self}")
        print(f"PrintTask::fw_spec: {fw_spec}")
        return FWAction()


@explicit_serialize
class PushTask(FiretaskBase):
    required_params = ["lname", "thing"]

    def run_task(self, fw_spec) -> FWAction:
        print(f"PushTask::self: {self}")
        print(f"PushTask::fw_spec: {fw_spec}")
        return FWAction(
            mod_spec=[{"_push": {f'things->{self["lname"]}': self["thing"]}}],
        )


if __name__ == "__main__":
    t1 = PushTask(lname="meep", thing="t1")
    t2 = PushTask(lname="meep", thing="t2")
    t3 = PrintTask()
    fw1 = Firework([t1, t2, t3])

    t4 = PushTask(lname="meep", thing="t3")
    t5 = PushTask(lname="meep", thing="t4")
    t6 = PrintTask()
    fw2 = Firework([t4, t5, t6])

    t7 = PushTask(lname="meep", thing="t5")
    t8 = PushTask(lname="meep", thing="t6")
    t9 = PrintTask()
    fw3 = Firework([t7, t8, t9])

    wf1 = Workflow([fw1, fw3], {fw1: [fw3]})
    wf2 = Workflow([fw2])
    wf1.append_wf(new_wf=wf2, fw_ids=[fw3.fw_id], detour=True)

    launchpad = LaunchPad.auto_load()
    launchpad.reset("", require_password=False)
    launchpad.add_wf(wf1)
    rapidfire(launchpad, pdb_on_exception=True)
2

A slightly better example, using the same tasks as above:

    t1 = PushTask(lname="meep", thing="t1")
    t2 = PushTask(lname="meep", thing="t2")
    fw1 = Firework([t1, t2])

    t3 = PushTask(lname="meep", thing="t3")
    t4 = PushTask(lname="meep", thing="t4")
    fw2 = Firework([t3, t4])

    t5 = PushTask(lname="meep", thing="t5")
    t6 = PushTask(lname="meep", thing="t6")
    fw3 = Firework([t5, t6])

    t7 = PrintTask()
    fw4 = Firework([t7])

    wf1 = Workflow([fw1, fw3, fw4], {fw1: [fw3], fw3: [fw4]})
    wf2 = Workflow([fw2])
    wf1.append_wf(new_wf=wf2, fw_ids=[fw1.fw_id], detour=True)

where I expect fw2 to be inserted between fw1 and fw3.

Setting propagate=False for PushTask gives

False on PushTask

PushTask::self: <{{mwe2_2.PushTask}}>:{'lname': 'meep', 'thing': 't1', '_fw_name': '{{mwe2_2.PushTask}}'}
PushTask::fw_spec: {'_tasks': [{'lname': 'meep', 'thing': 't1', '_fw_name': '{{mwe2_2.PushTask}}'}, {'lname': 'meep', 'thing': 't2', '_fw_name': '{{mwe2_2.PushTask}}'}], '_fw_env': {}}
PushTask::self: <{{mwe2_2.PushTask}}>:{'lname': 'meep', 'thing': 't2', '_fw_name': '{{mwe2_2.PushTask}}'}
PushTask::fw_spec: {'_tasks': [{'lname': 'meep', 'thing': 't1', '_fw_name': '{{mwe2_2.PushTask}}'}, {'lname': 'meep', 'thing': 't2', '_fw_name': '{{mwe2_2.PushTask}}'}], '_fw_env': {}, 'things': {'meep': ['t1']}}
PushTask::self: <{{mwe2_2.PushTask}}>:{'lname': 'meep', 'thing': 't5', '_fw_name': '{{mwe2_2.PushTask}}'}
PushTask::fw_spec: {'_tasks': [{'lname': 'meep', 'thing': 't5', '_fw_name': '{{mwe2_2.PushTask}}'}, {'lname': 'meep', 'thing': 't6', '_fw_name': '{{mwe2_2.PushTask}}'}], 'things': {'meep': ['t1', 't2']}, '_fw_env': {}}
PushTask::self: <{{mwe2_2.PushTask}}>:{'lname': 'meep', 'thing': 't6', '_fw_name': '{{mwe2_2.PushTask}}'}
PushTask::fw_spec: {'_tasks': [{'lname': 'meep', 'thing': 't5', '_fw_name': '{{mwe2_2.PushTask}}'}, {'lname': 'meep', 'thing': 't6', '_fw_name': '{{mwe2_2.PushTask}}'}], 'things': {'meep': ['t1', 't2', 't5']}, '_fw_env': {}}
PrintTask::self: <{{mwe2_2.PrintTask}}>:{'_fw_name': '{{mwe2_2.PrintTask}}'}
PrintTask::fw_spec: {'_tasks': [{'_fw_name': '{{mwe2_2.PrintTask}}'}], 'things': {'meep': ['t5', 't6']}, '_fw_env': {}}
PushTask::self: <{{mwe2_2.PushTask}}>:{'lname': 'meep', 'thing': 't3', '_fw_name': '{{mwe2_2.PushTask}}'}
PushTask::fw_spec: {'_tasks': [{'lname': 'meep', 'thing': 't3', '_fw_name': '{{mwe2_2.PushTask}}'}, {'lname': 'meep', 'thing': 't4', '_fw_name': '{{mwe2_2.PushTask}}'}], 'things': {'meep': ['t1', 't2']}, '_fw_env': {}}
PushTask::self: <{{mwe2_2.PushTask}}>:{'lname': 'meep', 'thing': 't4', '_fw_name': '{{mwe2_2.PushTask}}'}
PushTask::fw_spec: {'_tasks': [{'lname': 'meep', 'thing': 't3', '_fw_name': '{{mwe2_2.PushTask}}'}, {'lname': 'meep', 'thing': 't4', '_fw_name': '{{mwe2_2.PushTask}}'}], 'things': {'meep': ['t1', 't2', 't3']}, '_fw_env': {}}

and changing it to True gives

PushTask::self: <{{mwe2_2.PushTask}}>:{'lname': 'meep', 'thing': 't1', '_fw_name': '{{mwe2_2.PushTask}}'}
PushTask::fw_spec: {'_tasks': [{'lname': 'meep', 'thing': 't1', '_fw_name': '{{mwe2_2.PushTask}}'}, {'lname': 'meep', 'thing': 't2', '_fw_name': '{{mwe2_2.PushTask}}'}], '_fw_env': {}}
PushTask::self: <{{mwe2_2.PushTask}}>:{'lname': 'meep', 'thing': 't2', '_fw_name': '{{mwe2_2.PushTask}}'}
PushTask::fw_spec: {'_tasks': [{'lname': 'meep', 'thing': 't1', '_fw_name': '{{mwe2_2.PushTask}}'}, {'lname': 'meep', 'thing': 't2', '_fw_name': '{{mwe2_2.PushTask}}'}], '_fw_env': {}, 'things': {'meep': ['t1']}}
PushTask::self: <{{mwe2_2.PushTask}}>:{'lname': 'meep', 'thing': 't5', '_fw_name': '{{mwe2_2.PushTask}}'}
PushTask::fw_spec: {'_tasks': [{'lname': 'meep', 'thing': 't5', '_fw_name': '{{mwe2_2.PushTask}}'}, {'lname': 'meep', 'thing': 't6', '_fw_name': '{{mwe2_2.PushTask}}'}], 'things': {'meep': ['t1', 't2']}, '_fw_env': {}}
PushTask::self: <{{mwe2_2.PushTask}}>:{'lname': 'meep', 'thing': 't6', '_fw_name': '{{mwe2_2.PushTask}}'}
PushTask::fw_spec: {'_tasks': [{'lname': 'meep', 'thing': 't5', '_fw_name': '{{mwe2_2.PushTask}}'}, {'lname': 'meep', 'thing': 't6', '_fw_name': '{{mwe2_2.PushTask}}'}], 'things': {'meep': ['t1', 't2', 't5']}, '_fw_env': {}}
PrintTask::self: <{{mwe2_2.PrintTask}}>:{'_fw_name': '{{mwe2_2.PrintTask}}'}
PrintTask::fw_spec: {'_tasks': [{'_fw_name': '{{mwe2_2.PrintTask}}'}], 'things': {'meep': ['t1', 't2', 't5', 't6']}, '_fw_env': {}}
PushTask::self: <{{mwe2_2.PushTask}}>:{'lname': 'meep', 'thing': 't3', '_fw_name': '{{mwe2_2.PushTask}}'}
PushTask::fw_spec: {'_tasks': [{'lname': 'meep', 'thing': 't3', '_fw_name': '{{mwe2_2.PushTask}}'}, {'lname': 'meep', 'thing': 't4', '_fw_name': '{{mwe2_2.PushTask}}'}], 'things': {'meep': ['t1', 't2']}, '_fw_env': {}}
PushTask::self: <{{mwe2_2.PushTask}}>:{'lname': 'meep', 'thing': 't4', '_fw_name': '{{mwe2_2.PushTask}}'}
PushTask::fw_spec: {'_tasks': [{'lname': 'meep', 'thing': 't3', '_fw_name': '{{mwe2_2.PushTask}}'}, {'lname': 'meep', 'thing': 't4', '_fw_name': '{{mwe2_2.PushTask}}'}], 'things': {'meep': ['t1', 't2', 't3']}, '_fw_env': {}}

It make sense that fw2 doesn’t see the updated specs without turning on propagation, but it looks like the detour is not working: instead of ['t1', 't2', 't5', 't6'] for the print task, I would have expected ['t1', 't2', 't3', 't4', 't5', 't6'].