How to add tags to uploaded vasp results in database

Atomate
#1

Hello,

I want to add some tags (as dict) to the vasp results I am generating using atomate and uploading the mLab (MongoDB). I found that metadata can be added to workflow using wf_metadata parameter with get_wf(). But these tags do not get uploaded to the database (or I don’t know how to query them - I download the json from mlab database, did not find the tags in there either).

I think one of the main reason for keeping database would be easy query. So, I am assuming this is something that is already builtin but I haven’t figured it out yet. Can anyone help me with this? I want to know how to put tags (as dicts) to my vasp results and how I can use those tags to query results from the database.

Regards,
ASM Jonayat
MNE, PSU

···
#2

Hi

I am not sure if you are familiar with the “powerups” module but this allows you to do various things like what you request.

The specific powerup that will do what you want is:

atomate.vasp.powerups.add_additional_fields_to_taskdocs

e.g.,

new_wf = add_additional_fields_to_taskdocs(original_wf, {“my_key”: “my_value”})

Then add new_wf to the LaunchPad. “my_key” will show up as field in the task document after running

Best,

Anubhav

···

On Fri, Jan 26, 2018 at 1:06 AM, ASM Jonayat [email protected] wrote:

Hello,

I want to add some tags (as dict) to the vasp results I am generating using atomate and uploading the mLab (MongoDB). I found that metadata can be added to workflow using wf_metadata parameter with get_wf(). But these tags do not get uploaded to the database (or I don’t know how to query them - I download the json from mlab database, did not find the tags in there either).

I think one of the main reason for keeping database would be easy query. So, I am assuming this is something that is already builtin but I haven’t figured it out yet. Can anyone help me with this? I want to know how to put tags (as dicts) to my vasp results and how I can use those tags to query results from the database.

Regards,
ASM Jonayat
MNE, PSU

–

You received this message because you are subscribed to the Google Groups “atomate” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/4cd909b6-8343-440c-b979-b784816bd2fd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

–
Best,
Anubhav

#3

Thank you, thats exactly the funtion I was looking for. I checked the json file after uploading to the database and it in fact has all the tags I put in using that function.

If its no too much to ask could you please also let me know how to query the database using those tags?

For example, I have done 21 different slab calculations, and all of them as tags - {“system”: “surface”, “bulk”: “__different_based_on_the_bulk_system_of_the_slab”, “term”: “__different_based_on_what_term_i_used”}

Now, if I want to query final energy of only those calculations that has the criteria {“system”: “surface”, “bulk”: “NiO”, “term”: “110”} how can I achieve this using mgdb command. I tried following the example in the website but putting this as dict under --crit did not

(atomate_env) login2.stampede2(1366) mgdb query -c /work/03554/tg828535/stampede2/atomate/config/db.json --crit {"system"} --props task_id formula_pretty output.energy Criteria {system} is not a valid JSON string! (atomate_env) login2.stampede2(1367) mgdb query -c /work/03554/tg828535/stampede2/atomate/config/db.json --crit {“system”: “surface”} --props task_id formula_pretty output.energy
usage: mgdb [-h] {init,optimize,insert,query,stats} …
mgdb: error: unrecognized arguments: surface}

``

Or, do I need to use the query_engine to achieve this?

Regards,
Jonayat

···

On Friday, January 26, 2018 at 11:52:11 AM UTC-5, Anubhav Jain wrote:

Hi

I am not sure if you are familiar with the “powerups” module but this allows you to do various things like what you request.

The specific powerup that will do what you want is:

atomate.vasp.powerups.add_additional_fields_to_taskdocs

e.g.,

new_wf = add_additional_fields_to_taskdocs(original_wf, {“my_key”: “my_value”})

Then add new_wf to the LaunchPad. “my_key” will show up as field in the task document after running

Best,

Anubhav

On Fri, Jan 26, 2018 at 1:06 AM, ASM Jonayat [email protected] wrote:

Hello,

I want to add some tags (as dict) to the vasp results I am generating using atomate and uploading the mLab (MongoDB). I found that metadata can be added to workflow using wf_metadata parameter with get_wf(). But these tags do not get uploaded to the database (or I don’t know how to query them - I download the json from mlab database, did not find the tags in there either).

I think one of the main reason for keeping database would be easy query. So, I am assuming this is something that is already builtin but I haven’t figured it out yet. Can anyone help me with this? I want to know how to put tags (as dicts) to my vasp results and how I can use those tags to query results from the database.

Regards,
ASM Jonayat
MNE, PSU

–

You received this message because you are subscribed to the Google Groups “atomate” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/4cd909b6-8343-440c-b979-b784816bd2fd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

–
Best,
Anubhav

#4

Hey Jonayat,

I think the problem with your queries from the command line is that the curly braces are special syntax on the command line. You should be passing the whole dictionary as a string: e.g.

‘{“system”: “surface”}’

That being said, it’s much easier to use the VaspCalcDb in atomate to interface with the calculations in my experience. See an example here:

http://hackingmaterials.github.io/atomate/running_workflows.html#analyzing-a-bandstructure-workflow

where you might do:

atomate_db.collection.find({“system”: “surface”})

Thank you,

Brandon Bocklund

[email protected]

···

On Jan 26, 2018, at 3:48 PM, ASM Jonayat [email protected] wrote:

Thank you, thats exactly the funtion I was looking for. I checked the json file after uploading to the database and it in fact has all the tags I put in using that function.

If its no too much to ask could you please also let me know how to query the database using those tags?

For example, I have done 21 different slab calculations, and all of them as tags - {“system”: “surface”, “bulk”: “__different_based_on_the_bulk_system_of_the_slab”, “term”: “__different_based_on_what_term_i_used”}

Now, if I want to query final energy of only those calculations that has the criteria {“system”: “surface”, “bulk”: “NiO”, “term”: “110”} how can I achieve this using mgdb command. I tried following the example in the website but putting this as dict under --crit did not

(atomate_env) login2.stampede2(1366) mgdb query -c /work/03554/tg828535/stampede2/atomate/config/db.json --crit {"system"} --props task_id formula_pretty output.energy Criteria {system} is not a valid JSON string! (atomate_env) login2.stampede2(1367) mgdb query -c /work/03554/tg828535/stampede2/atomate/config/db.json --crit {“system”: “surface”} --props task_id formula_pretty output.energy
usage: mgdb [-h] {init,optimize,insert,query,stats} …
mgdb: error: unrecognized arguments: surface}

``

Or, do I need to use the query_engine to achieve this?

Regards,
Jonayat

On Friday, January 26, 2018 at 11:52:11 AM UTC-5, Anubhav Jain wrote:

Hi

I am not sure if you are familiar with the “powerups” module but this allows you to do various things like what you request.

The specific powerup that will do what you want is:

atomate.vasp.powerups.add_additional_fields_to_taskdocs

e.g.,

new_wf = add_additional_fields_to_taskdocs(original_wf, {“my_key”: “my_value”})

Then add new_wf to the LaunchPad. “my_key” will show up as field in the task document after running

Best,

Anubhav

On Fri, Jan 26, 2018 at 1:06 AM, ASM Jonayat <jonay…@gmail.com> wrote:

Hello,

I want to add some tags (as dict) to the vasp results I am generating using atomate and uploading the mLab (MongoDB). I found that metadata can be added to workflow using wf_metadata parameter with get_wf(). But these tags do not get uploaded to the database (or I don’t know how to query them - I download the json from mlab database, did not find the tags in there either).

I think one of the main reason for keeping database would be easy query. So, I am assuming this is something that is already builtin but I haven’t figured it out yet. Can anyone help me with this? I want to know how to put tags (as dicts) to my vasp results and how I can use those tags to query results from the database.

Regards,
ASM Jonayat
MNE, PSU

–
You received this message because you are subscribed to the Google Groups “atomate” group.
To unsubscribe from this group and stop receiving emails from it, send an email to atomate+u…@googlegroups.com.
To post to this group, send email to ato…@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/4cd909b6-8343-440c-b979-b784816bd2fd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

–
Best,
Anubhav

–
You received this message because you are subscribed to the Google Groups “atomate” group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/e9690371-3c02-4a68-b11f-855a07b5f977%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

#5

Thanks, Brandon. That solved my issue with the mgdb. I will look into details of VaspCalcDb.

···

On Saturday, January 27, 2018 at 3:46:15 PM UTC-5, Brandon B wrote:

Hey Jonayat,

I think the problem with your queries from the command line is that the curly braces are special syntax on the command line. You should be passing the whole dictionary as a string: e.g.

‘{“system”: “surface”}’

That being said, it’s much easier to use the VaspCalcDb in atomate to interface with the calculations in my experience. See an example here:

http://hackingmaterials.github.io/atomate/running_workflows.html#analyzing-a-bandstructure-workflow

where you might do:

atomate_db.collection.find({“system”: “surface”})

Thank you,

Brandon Bocklund

[email protected]

On Jan 26, 2018, at 3:48 PM, ASM Jonayat [email protected] wrote:

(atomate_env) login2.stampede2(1366) mgdb query -c /work/03554/tg828535/stampede2/atomate/config/db.json --crit {"system"} --props task_id formula_pretty output.energy Criteria {system} is not a valid JSON string! (atomate_env) login2.stampede2(1367) mgdb query -c /work/03554/tg828535/stampede2/atomate/config/db.json --crit {“system”: “surface”} --props task_id formula_pretty output.energy
usage: mgdb [-h] {init,optimize,insert,query,stats} …
mgdb: error: unrecognized arguments: surface}

``

Hi

I am not sure if you are familiar with the “powerups” module but this allows you to do various things like what you request.

The specific powerup that will do what you want is:

atomate.vasp.powerups.add_additional_fields_to_taskdocs

e.g.,

new_wf = add_additional_fields_to_taskdocs(original_wf, {“my_key”: “my_value”})

Then add new_wf to the LaunchPad. “my_key” will show up as field in the task document after running

Best,

Anubhav

Hello,

I want to add some tags (as dict) to the vasp results I am generating using atomate and uploading the mLab (MongoDB). I found that metadata can be added to workflow using wf_metadata parameter with get_wf(). But these tags do not get uploaded to the database (or I don’t know how to query them - I download the json from mlab database, did not find the tags in there either).

I think one of the main reason for keeping database would be easy query. So, I am assuming this is something that is already builtin but I haven’t figured it out yet. Can anyone help me with this? I want to know how to put tags (as dicts) to my vasp results and how I can use those tags to query results from the database.

Regards,
ASM Jonayat
MNE, PSU

–
You received this message because you are subscribed to the Google Groups “atomate” group.
To unsubscribe from this group and stop receiving emails from it, send an email to atomate+u…@googlegroups.com.
To post to this group, send email to ato…@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/4cd909b6-8343-440c-b979-b784816bd2fd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
On Fri, Jan 26, 2018 at 1:06 AM, ASM Jonayat <jonay…@gmail.com> wrote:

–
Best,
Anubhav

Thank you, thats exactly the funtion I was looking for. I checked the json file after uploading to the database and it in fact has all the tags I put in using that function.

If its no too much to ask could you please also let me know how to query the database using those tags?

For example, I have done 21 different slab calculations, and all of them as tags - {“system”: “surface”, “bulk”: “__different_based_on_the_bulk_system_of_the_slab”, “term”: “__different_based_on_what_term_i_used”}

Now, if I want to query final energy of only those calculations that has the criteria {“system”: “surface”, “bulk”: “NiO”, “term”: “110”} how can I achieve this using mgdb command. I tried following the example in the website but putting this as dict under --crit did not

Or, do I need to use the query_engine to achieve this?

Regards,
Jonayat

On Friday, January 26, 2018 at 11:52:11 AM UTC-5, Anubhav Jain wrote:

–
You received this message because you are subscribed to the Google Groups “atomate” group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/e9690371-3c02-4a68-b11f-855a07b5f977%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

#6

Hi Anubhav,

is there a way to use a powerup and specifically atomate.vasp.powerups.add_additional_fields_to_taskdocs

using the yaml file ?

Thanks

FR

···

Il giorno venerdì 26 gennaio 2018 17:52:11 UTC+1, Anubhav Jain ha scritto:

Hi

I am not sure if you are familiar with the “powerups” module but this allows you to do various things like what you request.

The specific powerup that will do what you want is:

atomate.vasp.powerups.add_additional_fields_to_taskdocs

e.g.,

new_wf = add_additional_fields_to_taskdocs(original_wf, {“my_key”: “my_value”})

Then add new_wf to the LaunchPad. “my_key” will show up as field in the task document after running

Best,

Anubhav

On Fri, Jan 26, 2018 at 1:06 AM, ASM Jonayat [email protected] wrote:

Hello,

I want to add some tags (as dict) to the vasp results I am generating using atomate and uploading the mLab (MongoDB). I found that metadata can be added to workflow using wf_metadata parameter with get_wf(). But these tags do not get uploaded to the database (or I don’t know how to query them - I download the json from mlab database, did not find the tags in there either).

I think one of the main reason for keeping database would be easy query. So, I am assuming this is something that is already builtin but I haven’t figured it out yet. Can anyone help me with this? I want to know how to put tags (as dicts) to my vasp results and how I can use those tags to query results from the database.

Regards,
ASM Jonayat
MNE, PSU

–

You received this message because you are subscribed to the Google Groups “atomate” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/4cd909b6-8343-440c-b979-b784816bd2fd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

–
Best,
Anubhav

#7

Unfortunately not at the moment. Someone would need to code a section in the YAML loader that reads in desired powerups and applies them. Not sure if it’s better if the powerups are part of the workflow YAML itself or a separate YAML file (to separate the workflow core from the more user-specific powerups into separate files)

Maybe open up an issue on Github for people to discuss?

···

Best,
Anubhav

#8

Ok I see, it’s not trivial.

Practically, can I still create a wf from a yaml and then use the code you suggested?

In other words, to do a sort of hybrid? something like:

original_wf = wf_from_yaml(‘my_wf.yaml’)

new_wf = add_additional_fields_to_taskdocs(original_wf, {“my_key”: “my_value”})

More generally speaking, is this the correct way to store some info like the mp_id

of the original structure to keep track of the starting point?

Thanks

#9

Yes, that hybrid way is currently the “standard” way to do it.

As for the correct way to store the mp_id of the original structure, there is no documented “standard” way to do it. However, using the powerup to add any additional fields to track (mp_ids, keywords, etc) is one way we’ve done it in the past. Another way is to use the “add_tags” power up instead, which adds the tags to both the workflow object as well as the database object (that’s probably the most “standard” way). That one just takes in a list of tags, though, rather than a dictionary.

···

On Wed, Nov 7, 2018 at 9:54 AM [email protected] wrote:

Ok I see, it’s not trivial.

Practically, can I still create a wf from a yaml and then use the code you suggested?

In other words, to do a sort of hybrid? something like:

original_wf = wf_from_yaml(‘my_wf.yaml’)

new_wf = add_additional_fields_to_taskdocs(original_wf, {“my_key”: “my_value”})

More generally speaking, is this the correct way to store some info like the mp_id

of the original structure to keep track of the starting point?

Thanks

–

You received this message because you are subscribed to the Google Groups “atomate” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

For more options, visit https://groups.google.com/d/optout.

–
Best,
Anubhav