Unable to connect to MongoDB via db.json

Atomate
1

Hello all,
i am new to atomate and i am trying to setup atomate on my cluster
im using atomate 0.9.4 and python 3.7

i have signed up for MongoDB atlas
and i have entered everything correct into db.json according to what mentioned on the atomate website

{
“host”: “mongodb+srv://cluster0-ipsds.mongodb.net/test?retryWrites=true&w=majority”,
“port”: 27017,
“database”: “mongotest”,
“collection”: “tasks”,
“admin_user”: “myadmin_name”,
“admin_password”: “xxxxxxxx”,
“readonly_user”: “myread_username”,
“readonly_password”: “xxxxxx”,
“authsource”: “mongotest”,
“aliases”: {}
}

and ran the following python script

from atomate.vasp.database import VaspCalcDb
x = VaspCalcDb.from_db_file(“db.json”)
x.reset()
print(“SUCCESS”)

and got following error

Traceback (most recent call last):
File “test.py”, line 3, in
x = VaspCalcDb.from_db_file(“db.json”)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/codes/atomate/atomate/utils/database.py”, line 135, in from_db_file
user, password, **kwargs)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/codes/atomate/atomate/vasp/database.py”, line 37, in init
password, **kwargs)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/codes/atomate/atomate/utils/database.py”, line 51, in init
if self.db.counter.find({"_id": “taskid”}).count() == 0:
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/cursor.py”, line 787, in count
cmd, self.__collation, session=self.__session)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/collection.py”, line 1600, in _count
_cmd, self._read_preference_for(session), session)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1458, in _retryable_read
exhaust=exhaust) as (sock_info,
File “/cm/shared/apps/pc2/EB-SW/software/lang/Python/3.7.0-foss-2018b/lib/python3.7/contextlib.py”, line 112, in enter
return next(self.gen)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1282, in _slaveok_for_server
with self._get_socket(server, session, exhaust=exhaust) as sock_info:
File “/cm/shared/apps/pc2/EB-SW/software/lang/Python/3.7.0-foss-2018b/lib/python3.7/contextlib.py”, line 112, in enter
return next(self.gen)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1222, in _get_socket
self.__all_credentials, checkout=exhaust) as sock_info:
File “/cm/shared/apps/pc2/EB-SW/software/lang/Python/3.7.0-foss-2018b/lib/python3.7/contextlib.py”, line 112, in enter
return next(self.gen)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/pool.py”, line 1138, in get_socket
sock_info.check_auth(all_credentials)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/pool.py”, line 712, in check_auth
auth.authenticate(credentials, self)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/auth.py”, line 564, in authenticate
auth_func(credentials, sock_info)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/auth.py”, line 539, in _authenticate_default
return _authenticate_scram(credentials, sock_info, ‘SCRAM-SHA-1’)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/auth.py”, line 263, in _authenticate_scram
res = sock_info.command(source, cmd)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/pool.py”, line 613, in command
user_fields=user_fields)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/network.py”, line 167, in command
parse_write_concern_error=parse_write_concern_error)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/helpers.py”, line 159, in _check_command_response
raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: Authentication failed.

i have also tried with and without authsource
and also tried with editing host uris

for example

“host”: “mongodb+srv://admin:@cluster0-ipsds.mongodb.net/test?retryWrites=true&w=majority”,

“host”: “mongodb+srv://cluster0-ipsds.mongodb.net/test?retryWrites=true&w=majority”,

“host”: “mongodb+srv://@cluster0-ipsds.mongodb.net/”,

but still ,i am getting the same error

NOTE: i could successfully connect to my database and insert data …i have used following script
import pymongo

import dns

client = pymongo.MongoClient(“mongodb+srv://myadmin_name:[email protected]/test?retryWrites=true&w=majority”)
db = client.mongo
db.inventory.insert_one(
{“item”: “canvas”,
“qty”: 100,
“tags”: [“cotton”],
“size”: {“h”: 28, “w”: 35.5, “uom”: “cm”}})

2

Hi Ram,

First, it looks like there is some disagreement in the database between the host key (the database is listed as “test”) and the remaining keys (database and authsource are listed as “mongotest”). Please ensure that this is your intention and fix if not. You can also just try settnig the host to “mongodb+srv://cluster0-ipsds.mongodb.net”

If that doesn’t work, it means that VaspCalcDb (and its base class, CalcDb) do not allow the use of +srv URL strings for the host. If that is the case, we should fix the code to allow for this.

Let us know how this goes

3

Thanks for your reply anubhav…
To be sure i re-created my database and users and i have entered everything correct still getting the error
{
“host”: “mongodb+srv://cluster0-ipsds.mongodb.net”,
“port”: 27017,
“database”: “dbtest”,
“collection”: “tasks”,
“admin_user”: “user1”,
“admin_password”: “givenmypassword”,
“readonly_user”: “user2”,
“readonly_password”: “givenmypassword”,
“aliases”: {}
}

what i have done
i have also used with “authsource”: “dbtest” and ran,…still getting the same error…
i have also included import dns in utils/database.py and /vasp/database.py and ran…still getting same error

Error
Traceback (most recent call last):
File “test.py”, line 3, in
x = VaspCalcDb.from_db_file("/scratch/hpc-prf-wcat/kuchana/atomate_dev/config/db.json")
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/codes/atomate/atomate/utils/database.py”, line 136, in from_db_file
user, password, **kwargs)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/codes/atomate/atomate/vasp/database.py”, line 39, in init
password, **kwargs)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/codes/atomate/atomate/utils/database.py”, line 52, in init
if self.db.counter.find({"_id": “taskid”}).count() == 0:
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/cursor.py”, line 787, in count
cmd, self.__collation, session=self.__session)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/collection.py”, line 1600, in _count
_cmd, self._read_preference_for(session), session)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1458, in _retryable_read
exhaust=exhaust) as (sock_info,
File “/cm/shared/apps/pc2/EB-SW/software/lang/Python/3.7.0-foss-2018b/lib/python3.7/contextlib.py”, line 112, in enter
return next(self.gen)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1282, in _slaveok_for_server
with self._get_socket(server, session, exhaust=exhaust) as sock_info:
File “/cm/shared/apps/pc2/EB-SW/software/lang/Python/3.7.0-foss-2018b/lib/python3.7/contextlib.py”, line 112, in enter
return next(self.gen)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1222, in _get_socket
self.__all_credentials, checkout=exhaust) as sock_info:
File “/cm/shared/apps/pc2/EB-SW/software/lang/Python/3.7.0-foss-2018b/lib/python3.7/contextlib.py”, line 112, in enter
return next(self.gen)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/pool.py”, line 1138, in get_socket
sock_info.check_auth(all_credentials)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/pool.py”, line 712, in check_auth
auth.authenticate(credentials, self)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/auth.py”, line 564, in authenticate
auth_func(credentials, sock_info)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/auth.py”, line 539, in _authenticate_default
return _authenticate_scram(credentials, sock_info, ‘SCRAM-SHA-1’)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/auth.py”, line 263, in _authenticate_scram
res = sock_info.command(source, cmd)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/pool.py”, line 613, in command
user_fields=user_fields)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/network.py”, line 167, in command
parse_write_concern_error=parse_write_concern_error)
File “/scratch/hpc-prf-wcat/kuchana/atomate_dev/atomate_env_dev/lib/python3.7/site-packages/pymongo/helpers.py”, line 159, in _check_command_response
raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: Authentication failed.

By the way after testing db.json i ran lpad -l my_launchpad.yaml reset
to test launchpad
Even my my_launchpad.yaml is not working…

host: mongodb+srv://cluster0-ipsds.mongodb.net
port: 27017
name: launchtestdb
username: user1
password: givenmyusername
ssl_ca_file: null
logdir: null
strm_lvl: INFO
user_indices: []
wf_user_indices: []

error
Traceback (most recent call last):
File “/scratch/hpc-prf-wcat/kuchana/dev_atomate_orig/atomate_orig_dev/bin/lpad”, line 11, in
load_entry_point(‘FireWorks==1.9.5’, ‘console_scripts’, ‘lpad’)()
File “/scratch/hpc-prf-wcat/kuchana/dev_atomate_orig/atomate_orig_dev/lib/python3.7/site-packages/fireworks/scripts/lpad_run.py”, line 1209, in lpad
args.func(args)
File “/scratch/hpc-prf-wcat/kuchana/dev_atomate_orig/atomate_orig_dev/lib/python3.7/site-packages/fireworks/scripts/lpad_run.py”, line 173, in reset
lp.workflows.count()))[0].upper() == ‘Y’:
File “/scratch/hpc-prf-wcat/kuchana/dev_atomate_orig/atomate_orig_dev/lib/python3.7/site-packages/pymongo/collection.py”, line 1800, in count
return self._count(cmd, collation, session)
File “/scratch/hpc-prf-wcat/kuchana/dev_atomate_orig/atomate_orig_dev/lib/python3.7/site-packages/pymongo/collection.py”, line 1600, in _count
_cmd, self._read_preference_for(session), session)
File “/scratch/hpc-prf-wcat/kuchana/dev_atomate_orig/atomate_orig_dev/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1454, in _retryable_read
read_pref, session, address=address)
File “/scratch/hpc-prf-wcat/kuchana/dev_atomate_orig/atomate_orig_dev/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1253, in _select_server
server = topology.select_server(server_selector)
File “/scratch/hpc-prf-wcat/kuchana/dev_atomate_orig/atomate_orig_dev/lib/python3.7/site-packages/pymongo/topology.py”, line 235, in select_server
address))
File “/scratch/hpc-prf-wcat/kuchana/dev_atomate_orig/atomate_orig_dev/lib/python3.7/site-packages/pymongo/topology.py”, line 193, in select_servers
selector, server_timeout, address)
File “/scratch/hpc-prf-wcat/kuchana/dev_atomate_orig/atomate_orig_dev/lib/python3.7/site-packages/pymongo/topology.py”, line 209, in _select_servers_loop
self._error_message(selector))
pymongo.errors.ServerSelectionTimeoutError: connection closed,connection closed,connection closed

for your refrence i am also adding pictures to let youu know that i have entered correct

Screenshot (46)
Screenshot (46)1344×384 30.4 KB

Screenshot (47)
Screenshot (47)1446×269 20.8 KB

4

Hi Ram,

After playing around with Mongo Atlas, it seems like you need to set authsource to “admin”. I.e., the db.json file should be:

{
   "host": "mongodb+srv://cluster0-ipsds.mongodb.net",
   "port": 27017,
   "database": "dbtest",
   "collection": "tasks",
   "admin_user": "user1",
   "admin_password": "givenmypassword",
   "readonly_user": "user2",
   "readonly_password": "givenmypassword",
   "aliases": {},
   "authsource": "admin"
}

Let me know if that works.

Best,
Alex

5

Yes it worked…Thanks a lot!!!
it would be helpful for others if you update it on atomate website