Erratum : This is not a PyTask (as said in my first statement), it is a custom firetask which executes this code from one of its imported module.
···
2017-07-28 11:25 GMT+02:00 David Michéa [email protected]:
Hi Anubhav,
For exemple, here is a stacktrace of a FIZZLED PyTask after correction of the bug:
- “_stacktrace”: “Traceback (most recent call last):\n File “/b/home/ipgs/dmichea/venv_a2s/lib/python3.6/site-packages/fireworks/core/rocket.py”, line 224, in run\n m_action = t.run_task(my_spec)\n File “/b/home/ipgs/dmichea/venv_a2s/lib/python3.6/site-packages/fireworks/user_objects/firetasks/a2s_tasks.py”, line 44, in run_task\n coregis_wf = coregis_S2.get_coregis_wf(src_id)\n File “/b/home/ipgs/dmichea/A2S/code/WFlows/coregis_S2.py”, line 220, in get_coregis_wf\n coregis_fws = create_coregis_fws(src_id)\n File “/b/home/ipgs/dmichea/A2S/code/WFlows/coregis_S2.py”, line 205, in create_coregis_fws\n fw_coregis = Firework([check_scratch_task, do_coregis_task], name=“coregistration”, spec={”_category": “multiseq”})\nNameError: name ‘fw_check_scratch’ is not defined\n"
fw_coregis = Firework([check_scratch_task, do_coregis_task], name=“coregistration”, parents = [fw_check_scratch], spec={"_category": “multiseq”})
fw_coregis = Firework([check_scratch_task, do_coregis_task], name=“coregistration”, spec={"_category": “multiseq”})
#!/usr/bin/env python3
import logging
from fireworks.core.firework import FWAction, Firework, FireTaskBase, Workflow
import mpic_S2
import attributes_S2
import coregis_S2
import correl_S2
import A2S_config
import import_S2
…
class AppendCoregisTask(FireTaskBase):
_fw_name = ‘AppendCoregisTask’
required_params = [“src_id”]
def run_task(self, fw_spec):
(A2S_logger, __) = A2S_config.init_loggers()
src_id = self[“src_id”]
coregis_wf = coregis_S2.get_coregis_wf(src_id)
if coregis_wf:
A2S_logger.info(“create & append co-registration workflow”)
return FWAction(additions=coregis_wf)
else:
A2S_logger.fatal("coregis_S2.get_coregis_wf failed : no co-registration workflow created for src id s" str(src_id))
return FWAction()
…
So could it be linked to my problem ?
PS: The JSON of the FW (seen from the web GUI, I don’t now how to have this info from CLI …)
Firework 2359 : append Coregistration WF
created on 07/28/2017
{
- “_id”: null,
- “archived_launches”: [ ],
- “created_on”: “2017-07-28T08:57:45.892309”,
- “fw_id”: 2359,
-
“launches”: [
{
- "_id": null,
- -
"action": {
- "additions": [ ],
- "defuse_children": false,
- "defuse_workflow": false,
- "detours": [ ],
- "exit": true,
- "mod_spec": [ ],
- -
"stored_data": {
- -
"_exception": {
- "_details": null,
- "_failed_task_n": 0,
- "_stacktrace": "Traceback (most recent call last):\n File \"/b/home/ipgs/dmichea/venv_a2s/lib/python3.6/site-packages/fireworks/core/rocket.py\", line 224, in run\n m_action = t.run_task(my_spec)\n File \"/b/home/ipgs/dmichea/venv_a2s/lib/python3.6/site-packages/fireworks/user_objects/firetasks/a2s_tasks.py\", line 44, in run_task\n coregis_wf = coregis_S2.get_coregis_wf(src_id)\n File \"/b/home/ipgs/dmichea/A2S/code/WFlows/coregis_S2.py\", line 220, in get_coregis_wf\n coregis_fws = create_coregis_fws(src_id)\n File \"/b/home/ipgs/dmichea/A2S/code/WFlows/coregis_S2.py\", line 205, in create_coregis_fws\n fw_coregis = Firework([check_scratch_task, do_coregis_task], name=\"coregistration\", spec={\"_category\": \"multiseq\"})\nNameError: name 'fw_check_scratch' is not defined\n"
},
- "_message": "runtime error during task",
- -
"_recovery": {
- "_all_mod_spec": [ ],
- "_all_stored_data": { },
- "_all_update_spec": { }
},
- -
"_task": {
- "_fw_name": "AppendCoregisTask",
- "src_id": 16314
}
},
- "update_spec": { }
},
- "fw_id": 2359,
- -
"fworker": {
- "category": "multiseq",
- "env": { },
- "name": null,
- "query": "{}"
},
- "host": "hpc-n567",
- "ip": "192.168.1.218",
- "launch_dir": "/b/home/ipgs/dmichea/A2S/launchdir/multiseq/block_2017-07-26-12-37-31-425159/launcher_2017-07-28-08-42-25-415961/launcher_2017-07-28-09-05-21-905507",
- "launch_id": 2317,
- "runtime_secs": 0.37215,
- "state": "FIZZLED",
- -
"state_history": [
- -
{
- "created_on": "2017-07-28T09:05:21.919802",
- "state": "RUNNING",
- "updated_on": "2017-07-28T09:05:22.286604"
},
- -
{
- "created_on": "2017-07-28T09:05:22.291952",
- "state": "FIZZLED"
}
],
- "time_end": "2017-07-28T09:05:22.291952",
- "time_start": "2017-07-28T09:05:21.919802",
- "trackers": [ ]
}
],
- “name”: “append Coregistration WF”,
-
“spec”: {
"_tasks": [
- -
{
- "_fw_name": "AppendCoregisTask",
- "src_id": 16314
}
]
},
- “state”: “FIZZLED”,
- “updated_on”: “2017-07-28T09:05:22.307789”
}
Here is the line 205 of /b/home/ipgs/dmichea/A2S/code/WFlows/coregis_S2.py before correction :
Here is the line after correction (the fw defined as parent does not exists anymore -> changed to a task included in this FW):
As you can see, the stacktrace print the line 205, and there is no more ‘parents’ parameter in it (fw_coregis = Firework([check_scratch_task, do_coregis_task], name=“coregistration”, spec={"_category": “multiseq”})). I can check it importing the module manually : the code is correct now
And it stops on the same bug …
But I was wrong in my first post : this is not a PyTask, it is a custom Task : class AppendCoregisTask in module a2s_tasks:
/b/home/ipgs/dmichea/venv_a2s/lib/python3.6/site-packages/fireworks/user_objects/firetasks/a2s_tasks.py
->
–
2017-07-27 17:44 GMT+02:00 Anubhav Jain [email protected]:
Hi David
PyTask shouldn’t be storing the function contents, only the function name (e.g. “my_module.my_function”) and the desired arguments and keyword arguments into that function (e.g., param1=“bob”). If this is incorrect, can you send me the contents that you are seeing in MongoDB regarding storing the code of the function itself?
If you update the code on your Worker, it should also update when you rerun a PyTask. You should confirm that the updated version of your code is indeed being loaded when you load the same function name manually, i.e., not through FWS. If it still seems like a FWS problem, then maybe send me a dump of the JSON of the FW that is causing problem from your MongoDB.
Best
Anubhav
–
You received this message because you are subscribed to the Google Groups “fireworkflows” group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/fireworkflows.
To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/CAKubV2JqpGa9b3retb%3DPX1_GVC8jPPKz3pkbiemKxbbR0o8mtw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
–
David Michéa
HPC Software Engineer
Tél : +33 (0)6 52 67 58 36
[email protected]
On Thu, Jul 27, 2017 at 6:48 AM, David Michéa [email protected] wrote:
Hi,
I use a lot of PyTask to implement my fireworks.
Sometimes, tasks are FIZZLED because of a bug in the function passed to the PyTask. But when I fix the bug and re_run the FW, It ends up with the same error message : the bug seems not fixed, althought the python module as been updated.
I suppose this is due to the fact that the code of the function seems to have been registered in MongoDB … Am I wrong ?
Is it possible to fix the code in MongoDB so that I can rerun these FWs without error ? Or is there a way to force the FW to reload the function from its module ?
Best regards,
David Michéa
HPC Software Engineer
Tél : +33 (0)6 52 67 58 36
[email protected]
–
–
You received this message because you are subscribed to the Google Groups “fireworkflows” group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/fireworkflows.
To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/CANr6Pra5yFa1VRcYZLVzJpx1mZLgWrOv0_hVgcYS%3D7gENogEaQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Best,
Anubhav
–
David Michéa
HPC Software Engineer
Tél : +33 (0)6 52 67 58 36
[email protected]
