Queus Adapter for MOAB (msub)

Arun_Gopalan · January 20, 2018, 8:13am

Hi,

I’m taking my baby steps into the world of Fireworks. I wanted to use fireworks through MOAB, do I have to write a queue-adapter from scratch or can I get away with just using a template file…?

Thanks,

Arun

Anubhav_Jain · January 21, 2018, 1:06am

Hi Arun,

There is no MOAB queue adapter in FireWorks currently. I could help you add one relatively quickly. I would just need to know:

The command to submit a batch script (probably just “msub”)
A template for how the batch script looks. e.g., if you can send me an example of a relatively complete (i.e., includes most options) batch script that you’ve used in the past.
The commands to:

a) List all the jobs in the queue (e.g., “qstat” in PBS)

b) List only the jobs from a certain user (e.g., “qstat -u username” in PBS)

c) List only the jobs from a certain queue (e.g., “qstat -q queuename” in PBS)

An example full output that gets printed back when running 3a. I just need to write code that parses that output and counts the number of jobs. If there are header lines in the output, for example, they should be ignored.

After getting those things I or someone on the team can add a MOAB adapter pretty quickly.

Best,

Anubhav

···

On Sat, Jan 20, 2018 at 12:13 AM, Arun Gopalan [email protected] wrote:

Hi,

I’m taking my baby steps into the world of Fireworks. I wanted to use fireworks through MOAB, do I have to write a queue-adapter from scratch or can I get away with just using a template file…?

Thanks,

Arun

–

You received this message because you are subscribed to the Google Groups “fireworkflows” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

Visit this group at https://groups.google.com/group/fireworkflows.

To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/792173c8-1288-409d-9c63-337eea3ff98c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

–
Best,
Anubhav

Anubhav_Jain · January 22, 2018, 5:58pm

Hi Arun,

Based on our offline conversation, I have pushed a sketch for a MOAB queue adapter in FWS v.1.6.5. Please update and give it a shot. There was no way for me to test the code myself, so I’m guessing it might take 2-3 iterations to get this working correctly.

To get it to work, you’ll need to:

Update to FWS v1.6.5
Use a my_qadapter.yaml like this one: https://github.com/materialsproject/fireworks/blob/master/fw_tutorials/queue/qadapter_MOAB.yaml
Modify that file for your needs. e.g., according to your sample file, your ppn was 1 and you had various strings for account, queue, etc.
Note that you had various other commands like “echo” and "ulimit statements in your file. It is possible to add those to the “pre_rocket” (commands before the job runs) or “post_rocket” (commands after the job run) parameters if you want to retain them. But, you might want to first give it a try without the extra stuff. See this link: https://stackoverflow.com/questions/3790454/in-yaml-how-do-i-break-a-string-over-multiple-lines if you’re confused about how to add multiple lines to the value of a key like “pre_rocket”.

Some things that I’m not sure works:

the code that counts the number of jobs in the queue might not work
the code that parses the job id of the submitted job (only used in reservation mode) might not work.

If either of the above is an issue, just send a new help ticket and we’ll respond.

···

On Saturday, January 20, 2018 at 5:07:13 PM UTC-8, ajain wrote:

Hi Arun,

There is no MOAB queue adapter in FireWorks currently. I could help you add one relatively quickly. I would just need to know:

The command to submit a batch script (probably just “msub”)

A template for how the batch script looks. e.g., if you can send me an example of a relatively complete (i.e., includes most options) batch script that you’ve used in the past.

The commands to:

a) List all the jobs in the queue (e.g., “qstat” in PBS)

b) List only the jobs from a certain user (e.g., “qstat -u username” in PBS)

c) List only the jobs from a certain queue (e.g., “qstat -q queuename” in PBS)

An example full output that gets printed back when running 3a. I just need to write code that parses that output and counts the number of jobs. If there are header lines in the output, for example, they should be ignored.

After getting those things I or someone on the team can add a MOAB adapter pretty quickly.

Best,

Anubhav

On Sat, Jan 20, 2018 at 12:13 AM, Arun Gopalan [email protected] wrote:

Hi,

I’m taking my baby steps into the world of Fireworks. I wanted to use fireworks through MOAB, do I have to write a queue-adapter from scratch or can I get away with just using a template file…?

Thanks,

Arun

–

You received this message because you are subscribed to the Google Groups “fireworkflows” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

Visit this group at https://groups.google.com/group/fireworkflows.

To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/792173c8-1288-409d-9c63-337eea3ff98c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

–
Best,
Anubhav