REST query returned with error status code 500

Hi,

Some queries that used to work fine with the new API, no longer appear to be working.

For example, if I wanted to query for materials with 3-5 elements, I would perform the following query:

from mp_api.client import MPRester

with MPRester(NEW_API_KEY) as mpr:
    docs = mpr.summary.search(num_elements = (3,5), fields = ['material_id'])

When I ran this query about a week ago, it ran perfectly fine. But as of yesterday, this no longer works and gives the following error message.

Error message MPRestError Traceback (most recent call last) Cell In [8], line 3 1 from mp_api.client import MPRester 2 with MPRester(API_KEY) as mpr: ----> 3 docs = mpr.summary.search(num_elements=(3,5),fields=['material_id'])

File ~/.local/lib/python3.9/site-packages/mp_api/client/routes/summary.py:287, in SummaryRester.search(self, band_gap, chemsys, crystal_system, density, deprecated, e_electronic, e_ionic, e_total, efermi, elastic_anisotropy, elements, energy_above_hull, equilibrium_reaction_energy, exclude_elements, formation_energy, formula, g_reuss, g_voigt, g_vrh, has_props, has_reconstructed, is_gap_direct, is_metal, is_stable, k_reuss, k_voigt, k_vrh, magnetic_ordering, material_ids, n, num_elements, num_sites, num_magnetic_sites, num_unique_magnetic_sites, piezoelectric_modulus, poisson_ratio, possible_species, shape_factor, spacegroup_number, spacegroup_symbol, surface_energy_anisotropy, theoretical, total_energy, total_magnetization, total_magnetization_normalized_formula_units, total_magnetization_normalized_vol, uncorrected_energy, volume, weighted_surface_energy, weighted_work_function, sort_fields, num_chunks, chunk_size, all_fields, fields)
277 query_params.update(
278 {β€œ_sort_fields”: β€œ,”.join([s.strip() for s in sort_fields])}
279 )
281 query_params = {
282 entry: query_params[entry]
283 for entry in query_params
284 if query_params[entry] is not None
285 }
β†’ 287 return super()._search(
288 num_chunks=num_chunks,
289 chunk_size=chunk_size,
290 all_fields=all_fields,
291 fields=fields,
292 **query_params
293 )

File ~/.local/lib/python3.9/site-packages/mp_api/client/core/client.py:920, in BaseRester._search(self, num_chunks, chunk_size, all_fields, fields, **kwargs)
897 β€œβ€"
898 A generic search method to retrieve documents matching specific parameters.
899
(…)
915 A list of documents.
916 β€œβ€"
917 # This method should be customized for each end point to give more user friendly,
918 # documented kwargs.
β†’ 920 return self._get_all_documents(
921 kwargs,
922 all_fields=all_fields,
923 fields=fields,
924 chunk_size=chunk_size,
925 num_chunks=num_chunks,
926 )

File ~/.local/lib/python3.9/site-packages/mp_api/client/core/client.py:969, in BaseRester._get_all_documents(self, query_params, all_fields, fields, chunk_size, num_chunks)
955 list_entries = sorted(
956 (
957 (key, len(entry.split(β€œ,”)))
(…)
964 reverse=True,
965 )
967 chosen_param = list_entries[0][0] if len(list_entries) > 0 else None
β†’ 969 results = self._query_resource(
970 query_params,
971 fields=fields,
972 parallel_param=chosen_param,
973 chunk_size=chunk_size,
974 num_chunks=num_chunks,
975 )
977 return results[β€œdata”]

File ~/.local/lib/python3.9/site-packages/mp_api/client/core/client.py:294, in BaseRester._query_resource(self, criteria, fields, suburl, use_document_model, parallel_param, num_chunks, chunk_size, timeout)
291 if not url.endswith(β€œ/”):
292 url += β€œ/”
β†’ 294 data = self._submit_requests(
295 url=url,
296 criteria=criteria,
297 use_document_model=use_document_model,
298 parallel_param=parallel_param,
299 num_chunks=num_chunks,
300 chunk_size=chunk_size,
301 timeout=timeout,
302 )
304 return data
306 except RequestException as ex:

File ~/.local/lib/python3.9/site-packages/mp_api/client/core/client.py:556, in BaseRester._submit_requests(self, url, criteria, use_document_model, parallel_param, num_chunks, chunk_size, timeout)
553 remaining -= crit[β€œ_limit”]
555 # Submit requests and process data
β†’ 556 data_tuples = self._multi_thread(use_document_model, params_list, pbar, timeout)
558 for data, _, _ in data_tuples:
559 total_data[β€œdata”].extend(data[β€œdata”])

File ~/.local/lib/python3.9/site-packages/mp_api/client/core/client.py:625, in BaseRester._multi_thread(self, use_document_model, params_list, progress_bar, timeout)
621 finished, futures = wait(futures, return_when=FIRST_COMPLETED)
623 for future in finished:
β†’ 625 data, subtotal = future.result()
627 if progress_bar is not None:
628 progress_bar.update(len(data[β€œdata”]))

File ~/.conda/envs/ml_materials/lib/python3.9/concurrent/futures/_base.py:439, in Future.result(self, timeout)
437 raise CancelledError()
438 elif self._state == FINISHED:
β†’ 439 return self.__get_result()
441 self._condition.wait(timeout)
443 if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]:

File ~/.conda/envs/ml_materials/lib/python3.9/concurrent/futures/_base.py:391, in Future.__get_result(self)
…
729 f"on URL {response.url} with message:\n{message}"
730 )

MPRestError: REST query returned with error status code 500 on URL https://api.materialsproject.org/summary/?nelements_min=3&nelements_max=5&_limit=1000&_fields=material_id&_skip=9000 with message:
Internal Server Error

Other queries appear to be working perfectly fine such as the one below:

with MPRester(NEW_API_KEY) as mpr:
    docs = mpr.summary.search(material_ids=['mp-1069538','mp-540839'])

Would anyone be able to try the first query and see if it works fine on their end?

Hi @AntObi, thank you for bringing this up. I am able to recreate the error and am looking into what the problem is.

– Jason

1 Like

@AntObi, this has been fixed and will go out in our deployment over the next couple days.

– Jason

1 Like

Hi @munrojm. Thank you checking this. What is the timeline for when the fix will go out into deployment?

  • Anthony

Hi @AntObi, we have been a bit delayed but are hoping to get it deployed by the end of this week.

– Jason

@AntObi, the fix is live. I have confirmed the query now works as expected.

– Jason

1 Like

Hi @munrojm, it seems like this error is back as I am experiencing it now although it has been working fine for the past few weeks, would you be able to look into this?

MPRestError: REST query returned with error status code 500 on URL https://api.materialsproject.org/materials/summary/?nelements_min=2&nelements_max=4&exclude_elements=H%2CHe%2CLi%2CBe%2CB%2CC%2CN%2CNe%2CMg%2CAl%2CSi%2CP%2CS%2CCl%2CAr%2CK%2CCa%2CSc%2CTi%2CV%2CCr%2CMn%2CFe%2CCo%2CNi%2CCu%2CZn%2CGa%2CGe%2CAs%2CSe%2CBr%2CKr%2CRb%2CSr%2CY%2CZr%2CNb%2CMo%2CTc%2CRu%2CRh%2CPd%2CAg%2CCd%2CIn%2CSn%2CSb%2CTe%2CI%2CXe%2CCs%2CBa%2CLa%2CCe%2CPr%2CNd%2CPm%2CSm%2CEu%2CGd%2CTb%2CDy%2CHo%2CEr%2CTm%2CYb%2CLu%2CHf%2CTa%2CW%2CRe%2COs%2CIr%2CPt%2CAu%2CHg%2CTl%2CPb%2CBi%2CPo%2CAt%2CRn%2CFr%2CRa%2CAc%2CTh%2CPa%2CU%2CNp%2CPu%2CAm%2CCm%2CBk%2CCf%2CEs%2CFm%2CMd%2CNo%2CLr%2CRf%2CDb%2CSg%2CBh%2CHs%2CMt%2CDs%2CRg%2CCn%2CNh%2CFl%2CMc%2CLv%2CTs%2COg&_limit=1000&_fields=material_id%2Ccomposition with message: Server timed out trying to obtain data. Try again with a smaller request, or remove sorting fields and sort data locally.