mlaunch in reservation mode

How to use ‘mlaunch’ in reservation mode? Once I change the my_qadapter.yaml to use ‘mlaunch’ instead of ‘rlaunch’ with appropriate commands, how should I submit to the queue? qlaunch -r singleshot or qlaunch -r rapidfire doesn’t seem to work. (I am using v1.04 of FWS)

Hi,

You cannot use mlaunch in reservation mode. The point of reservation mode is to tailor the queue script to a single job. With mlaunch, there will be multiple jobs that get pulled and the queue script cannot be specific to a single job.

You can, however, make sure that mlaunch only pulls a particular “type” of job. e.g., let’s say you have mixed job type in the database and want mlaunch to only pull a certain type. You can use the FireWorker options to make sure that only certain jobs are being run within mlaunch. See the last part of “http://pythonhosted.org/FireWorks/multi_job.html” for more details.

If that’s not helpful, maybe you can describe a bit of what you’re trying to accomplish.

Best,

Anubhav

···

On Thu, Jun 4, 2015 at 9:32 AM, Panchapakesan Ganesh [email protected] wrote:

How to use ‘mlaunch’ in reservation mode? Once I change the my_qadapter.yaml to use ‘mlaunch’ instead of ‘rlaunch’ with appropriate commands, how should I submit to the queue? qlaunch -r singleshot or qlaunch -r rapidfire doesn’t seem to work. (I am using v1.04 of FWS)

You received this message because you are subscribed to the Google Groups “fireworkflows” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/bf8957bc-1651-49cb-98e3-64ab6cf170a2%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Thanks Anubhav. What I am trying to do is to bundle multiple fireworks, with similar computing requirements, using ‘mlaunch’ into one PBS job that can be submitted to a PBS queue, with the mongoDB hosted on a different server (such as the mongo-cloud service) than the fworker.

Ganesh

···

On Friday, June 5, 2015 at 12:49:39 PM UTC-4, Anubhav Jain wrote:

Hi,

You cannot use mlaunch in reservation mode. The point of reservation mode is to tailor the queue script to a single job. With mlaunch, there will be multiple jobs that get pulled and the queue script cannot be specific to a single job.

You can, however, make sure that mlaunch only pulls a particular “type” of job. e.g., let’s say you have mixed job type in the database and want mlaunch to only pull a certain type. You can use the FireWorker options to make sure that only certain jobs are being run within mlaunch. See the last part of “http://pythonhosted.org/FireWorks/multi_job.html” for more details.

If that’s not helpful, maybe you can describe a bit of what you’re trying to accomplish.

Best,

Anubhav

On Thu, Jun 4, 2015 at 9:32 AM, Panchapakesan Ganesh [email protected] wrote:

How to use ‘mlaunch’ in reservation mode? Once I change the my_qadapter.yaml to use ‘mlaunch’ instead of ‘rlaunch’ with appropriate commands, how should I submit to the queue? qlaunch -r singleshot or qlaunch -r rapidfire doesn’t seem to work. (I am using v1.04 of FWS)

You received this message because you are subscribed to the Google Groups “fireworkflows” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/bf8957bc-1651-49cb-98e3-64ab6cf170a2%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

I need the offline/reservation mode because my compute nodes cannot connect to the launchpad. In cases (clusters) where they do, the mlaunch works fine as you prescribed. --Ganesh

···

On Friday, June 5, 2015 at 1:17:55 PM UTC-4, Panchapakesan Ganesh wrote:

Thanks Anubhav. What I am trying to do is to bundle multiple fireworks, with similar computing requirements, using ‘mlaunch’ into one PBS job that can be submitted to a PBS queue, with the mongoDB hosted on a different server (such as the mongo-cloud service) than the fworker.

Ganesh

On Friday, June 5, 2015 at 12:49:39 PM UTC-4, Anubhav Jain wrote:

Hi,

You cannot use mlaunch in reservation mode. The point of reservation mode is to tailor the queue script to a single job. With mlaunch, there will be multiple jobs that get pulled and the queue script cannot be specific to a single job.

You can, however, make sure that mlaunch only pulls a particular “type” of job. e.g., let’s say you have mixed job type in the database and want mlaunch to only pull a certain type. You can use the FireWorker options to make sure that only certain jobs are being run within mlaunch. See the last part of “http://pythonhosted.org/FireWorks/multi_job.html” for more details.

If that’s not helpful, maybe you can describe a bit of what you’re trying to accomplish.

Best,

Anubhav

On Thu, Jun 4, 2015 at 9:32 AM, Panchapakesan Ganesh gp..[email protected] wrote:

How to use ‘mlaunch’ in reservation mode? Once I change the my_qadapter.yaml to use ‘mlaunch’ instead of ‘rlaunch’ with appropriate commands, how should I submit to the queue? qlaunch -r singleshot or qlaunch -r rapidfire doesn’t seem to work. (I am using v1.04 of FWS)

You received this message because you are subscribed to the Google Groups “fireworkflows” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/bf8957bc-1651-49cb-98e3-64ab6cf170a2%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Ok, I understand; unfortunately, there is so far no way to run mlaunch in offline mode. To do this, one would need to pre-generate many FW.json files and then mlaunch would need to select from that pool of pregenerated files. It is certainly possible, but I would put this in the category of something that an outside user would need to lead the development of.

Best,

Anubhav

···

On Fri, Jun 5, 2015 at 11:33 AM, Panchapakesan Ganesh [email protected] wrote:

I need the offline/reservation mode because my compute nodes cannot connect to the launchpad. In cases (clusters) where they do, the mlaunch works fine as you prescribed. --Ganesh

On Friday, June 5, 2015 at 1:17:55 PM UTC-4, Panchapakesan Ganesh wrote:

Thanks Anubhav. What I am trying to do is to bundle multiple fireworks, with similar computing requirements, using ‘mlaunch’ into one PBS job that can be submitted to a PBS queue, with the mongoDB hosted on a different server (such as the mongo-cloud service) than the fworker.

Ganesh

On Friday, June 5, 2015 at 12:49:39 PM UTC-4, Anubhav Jain wrote:

Hi,

You cannot use mlaunch in reservation mode. The point of reservation mode is to tailor the queue script to a single job. With mlaunch, there will be multiple jobs that get pulled and the queue script cannot be specific to a single job.

You can, however, make sure that mlaunch only pulls a particular “type” of job. e.g., let’s say you have mixed job type in the database and want mlaunch to only pull a certain type. You can use the FireWorker options to make sure that only certain jobs are being run within mlaunch. See the last part of “http://pythonhosted.org/FireWorks/multi_job.html” for more details.

If that’s not helpful, maybe you can describe a bit of what you’re trying to accomplish.

Best,

Anubhav

On Thu, Jun 4, 2015 at 9:32 AM, Panchapakesan Ganesh [email protected] wrote:

How to use ‘mlaunch’ in reservation mode? Once I change the my_qadapter.yaml to use ‘mlaunch’ instead of ‘rlaunch’ with appropriate commands, how should I submit to the queue? qlaunch -r singleshot or qlaunch -r rapidfire doesn’t seem to work. (I am using v1.04 of FWS)

You received this message because you are subscribed to the Google Groups “fireworkflows” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/bf8957bc-1651-49cb-98e3-64ab6cf170a2%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

You received this message because you are subscribed to the Google Groups “fireworkflows” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/257c93c8-0855-44d5-bb22-caa4ad0e77df%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.