hi, is there a way (or guideline) to repeat all benchmark results in my own computer? a docker with all dependency or something?
thanks
Docker would be a nice way to ensure reproducibility, but the barrier to entry for most people is too high, especially for running docker on cluster hardware that is actually capable of training some of the models (e.g., MEGNet, ALIGNN). I’ll add it as an issue to the repo as something to consider.
As of right now, you’ll need to go through installing the requirements for each benchmark you intend to reproduce. Some of the larger ones can be quite compute-intensive (not really possible on a laptop etc.). You can find the requirements and some sample code for most of the algorithms here: matbench github