MRopo_etal_amino_acids_conformer_dataset_SciData2015
10.17172/NOMAD/20150526220502
nomad-raw-download.zip Failed - Unknown server error. Please try again, or contact the server administrator.
The URL points to an old installation of NOMAD that is no longer maintain. The same dataset is also available from the new version of NOMAD. To download all files, please select all entries and hit the download button on the top/right.
https://nomad-lab.eu/prod/v1/gui/dataset/id/y-QJse1eRK2qpXl9mUbW6w
unfortunately the dataset is too big (1.5TB) for your server(s) to handle due to:
- an unreliable network connection
- slow transmission speed (only 5MB/s ?)
- wget continue not supported
the issue is that each of the ~20000 entries has a ~80mb cube.bz2 file which i dont really need anyway.
i copied the download url form my browser and pasted it in a terminal window on the mist cluster at utoronto (which has multiple redundant gigabit fiber lines at the scinet facility 1.8km from TorIX in downtown toronto where all the major fiber networks interconnect so im almost certain the network problem is on your side):
(1) what would i add to this query to download the dataset without *.cube.bz2 files ???
(2) what is the gui query to filter by mainfile ? it would be very convenient if you had minimal documentation of api fields on your website thanks.
“We are currently working to update this content. While we are still working on this, please use our video tutorial as a starting point:”
[Explore data - Documentation]
mmurp027@mist-login01:~/scratch$ wget "https://nomad-lab.eu/prod/v1/api/v1/entries/raw?owner=visible&json_query={%22and%22:[{},{%22datasets.dataset_id:any%22:[%22ZFqQC76ARYWFPCTJGyj7Gw%22]}]}" -O hu.zip
--2024-12-02 21:13:17-- https://nomad-lab.eu/prod/v1/api/v1/entries/raw?owner=visible&json_query=%7B%22and%22:[%7B%7D,%7B%22datasets.dataset_id:any%22:[%22ZFqQC76ARYWFPCTJGyj7Gw%22]%7D]%7D
Resolving nomad-lab.eu (nomad-lab.eu)... 130.183.217.23
Connecting to nomad-lab.eu (nomad-lab.eu)|130.183.217.23|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/zip]
Saving to: ‘hu.zip’
hu.zip [ <=> ] 139.57G 4.52MB/s in 8h 41m
2024-12-03 05:54:45 (4.57 MB/s) - Read error at byte 149861370230 (Success.).Retrying.
--2024-12-03 05:54:46-- (try: 2) https://nomad-lab.eu/prod/v1/api/v1/entries/raw?owner=visible&json_query=%7B%22and%22:[%7B%7D,%7B%22datasets.dataset_id:any%22:[%22ZFqQC76ARYWFPCTJGyj7Gw%22]%7D]%7D
Connecting to nomad-lab.eu (nomad-lab.eu)|130.183.217.23|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/zip]
Saving to: ‘hu.zip’
hu.zip [ <=> ] 18.96G 4.80MB/s in 74m 53s
2024-12-03 07:09:43 (4.32 MB/s) - Read error at byte 149861370230 (Success.).Retrying.
--2024-12-03 07:09:45-- (try: 3) https://nomad-lab.eu/prod/v1/api/v1/entries/raw?owner=visible&json_query=%7B%22and%22:[%7B%7D,%7B%22datasets.dataset_id:any%22:[%22ZFqQC76ARYWFPCTJGyj7Gw%22]%7D]%7D
Connecting to nomad-lab.eu (nomad-lab.eu)|130.183.217.23|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/zip]
Saving to: ‘hu.zip’
hu.zip [ <=> ] 1.56G 4.81MB/s in 5m 47s
2024-12-03 07:15:33 (4.62 MB/s) - Read error at byte 149861370230 (Success.).Retrying.
--2024-12-03 07:15:36-- (try: 4) https://nomad-lab.eu/prod/v1/api/v1/entries/raw?owner=visible&json_query=%7B%22and%22:[%7B%7D,%7B%22datasets.dataset_id:any%22:[%22ZFqQC76ARYWFPCTJGyj7Gw%22]%7D]%7D
Connecting to nomad-lab.eu (nomad-lab.eu)|130.183.217.23|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/zip]
Saving to: ‘hu.zip’
hu.zip [ <=> ] 1.12G 4.52MB/s in 4m 7s
2024-12-03 07:19:44 (4.64 MB/s) - Read error at byte 149861370230 (Success.).Retrying.
--2024-12-03 07:19:48-- (try: 5) https://nomad-lab.eu/prod/v1/api/v1/entries/raw?owner=visible&json_query=%7B%22and%22:[%7B%7D,%7B%22datasets.dataset_id:any%22:[%22ZFqQC76ARYWFPCTJGyj7Gw%22]%7D]%7D
Connecting to nomad-lab.eu (nomad-lab.eu)|130.183.217.23|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/zip]
Saving to: ‘hu.zip’
hu.zip [ <=> ] 156.35G 3.53MB/s