Can't use AI toolkit

NOMAD and FAIRmat
1

So I have published data with embargo, but later lifted the embargo from the edit page, however, I cannot retrieve my dataset from within the tutorial notebooks.

def get_query():
    query = ArchiveQuery(
        query={
            '$and':
            [{'upload_id': 'Uzb4M0iTSDCDBXId8_RDXw'}]
        },

        required={
            'section_run':{
                'section_single_configuration_calculation':{
                    "energy_total": '*',
                },
                'section_system':{
                    "chemical_composition_reduced": '*',
                    'atom_labels':'*',
                    'atom_positions':'*',
                    'lattice_vectors':'*',                    
                    },
            }               
        },            
        per_page=40,
        parallel=5,
        max=200)
    return query

Can I only retrieve non-embargo datasets? How to tell if the embargo is lifted?

2

Thanks for your post. There are two issues.

First, you stumbled over a bug in our new version. Lifting the embargo, removed the embargo information entirely instead of disabling the embargo. I have to fix that ASAP. Do you want me to reestablish the embargo or to actually lift it?

Second, yes you can analyse embargo data, but you have to authenticate yourself to the API. The ArchiveQuery takes username and password keyword args. These should allow you to access your embargo and staging data.

Let us know, if you have further problems

3

I assume this is the correct way to enter username and password, but somehow I get credential errors, eventhough the login is correct.

def get_query():
    query = ArchiveQuery(
        query={
            '$and':
            [{'datasets': 'wy-sac-hf'}]
        },

        required={
            'section_run':{
                'section_single_configuration_calculation':{
                    "energy_total": '*',
                },
                'section_system':{
                    "chemical_composition_reduced": '*',
                    'atom_labels':'*',
                    'atom_positions':'*',
                    'lattice_vectors':'*',                    
                    },
            }               
        },
        username='MY_EMAIL',
        password='MY_PASSWORD',

        per_page=40,
        parallel=5,
        max=200)
    return query

Alternatively, if its easier to lift the embargo to access the data perhaps you could do that for me. @mscheidgen

4

I lifted the embargo for now. The 112 entries of upload Uzb4M0iTSDCDBXId8_RDXw should be accessible now without any authentication. I can re-establish the embargo later, when I figured out what was going wrong.

What you did was perfectly right. The nomad-lab package that provides the ArchiveQuery has a wrong configuration. If you execute the following somewhere before you create the ArchiveQuery, the authentication should work.

from nomad import config
config.keycloak.realm_name='fairdi_nomad_prod'
5

Thank you this works !

6

The AI Toolkit uses the new version now and authentication has been fixed. If you provide username, password, you should be able to access embargo data now. Do you want the embargo on this upload?

7

Oh that would be better yes. So in addition to username and pass tag I add this as well:

from nomad import config
config.keycloak.realm_name='fairdi_nomad_prod'

correct ?

8

This should not be necessary anymore, because we fixed the default configuration. I’ll reestablish the embargo later today.