Slow `lpad get_fws` command

FireWorks
#1

My lpad get_fws -d less command seems to be quite slow and scales poorly with the number of fws to be returned.

For example, I have the following timing:

  • -m 1: 2.4s
  • -m 10: 4.5s
  • -m 100: 27.6s

I understand that the code needs to query the launches collection, but with -d less option no information about launches are printed.

Am I having the expected behaviour?
I am with fireworks=1.9.5 on WSL Ubuntu 18.04

#2

After some profiling I found this is due to the lpad script trying to deserialize the retrived Fireworks one by one. Most of the time seems to spent on searching and importing the modules.

#3

Hi,

These times are much longer than expected.

Do you have your own custom Firetasks that are being deserialized? If so, perhaps switching to the explicit serialization would help as this avoids the need to search through modules.

If your FW spec are very large (Eg 10MB per Firework) or your database installation is slow, then querying can also be slow. Also if you use the -q flag in your get_fws query and don’t set up proper indexes.

Otherwise I would expect things to finish within seconds. We’ve never had a problem with deserialization itself being slow.

1 Like