Hello everyone. I am trying to calculate the FORCE CONSTANTS for shengBTE the 2nd and 3rd IFCS were getting easily but when it comes to the 4th order its shows the following error

init_gesdd failed init
Traceback (most recent call last):
File “fit.py”, line 194, in
sc_nstruct = sc_(nstruct, cs_1, structures)
File “fit.py”, line 91, in sc_
sc_tmp.add_structure(structure)
File “/gpfs/share/home/2201112409/anaconda3/envs/p3/lib/python3.8/site-packages/hiphive/structure_container.py”, line 181, in add_structure
M = self._compute_fit_matrix(atoms)
File “/gpfs/share/home/2201112409/anaconda3/envs/p3/lib/python3.8/site-packages/hiphive/structure_container.py”, line 278, in _compute_fit_matrix
return self._previous_fcm.get_fit_matrix(atoms.get_array(‘displacements’))
File “/gpfs/share/home/2201112409/anaconda3/envs/p3/lib/python3.8/site-packages/hiphive/force_constant_model.py”, line 292, in get_fit_matrix
M = M.dot(self.cs._cvs.toarray())
File “/gpfs/share/home/2201112409/anaconda3/envs/p3/lib/python3.8/site-packages/scipy/sparse/coo.py”, line 321, in toarray
B = self._process_toarray_args(order, out)
File “/gpfs/share/home/2201112409/anaconda3/envs/p3/lib/python3.8/site-packages/scipy/sparse/base.py”, line 1185, in _process_toarray_args
return np.zeros(self.shape, dtype=self.dtype, order=order)
numpy.core.exceptions.MemoryError: Unable to allocate 3.21 GiB for an array with shape (26142, 16469) and data type float64
Traceback (most recent call last):
File “fit.py”, line 197, in
fcs_raw, fcs_rot = fit_paras(name, sc_nstruct, cs_1, sys_name, prim, supercell, cutoffs, sd[1])
File “fit.py”, line 97, in fit_paras
opt = Optimizer(sc.get_fit_data(), train_size=0.99, seed=seed
)
File “/gpfs/share/home/2201112409/anaconda3/envs/p3/lib/python3.8/site-packages/hiphive/structure_container.py”, line 214, in get_fit_data
return np.vstack(M_list), np.hstack(f_list)
File “<array_function internals>”, line 180, in vstack
File “/gpfs/share/home/2201112409/anaconda3/envs/p3/lib/python3.8/site-packages/numpy/core/shape_base.py”, line 282, in vstack
return _nx.concatenate(arrs, 0)
File “<array_function internals>”, line 180, in concatenate
numpy.core._exceptions.MemoryError: Unable to allocate 7.95 GiB for an array with shape (64800, 16469) and data type float64
Traceback (most recent call last):
File “fit.py”, line 197, in
fcs_raw, fcs_rot = fit_paras(name, sc_nstruct, cs_1, sys_name, prim, supercell, cutoffs, sd[1])
File “fit.py”, line 98, in fit_paras
opt.train()
File “/gpfs/share/home/2201112409/anaconda3/envs/p3/lib/python3.8/site-packages/hiphive/fitting/optimizer.py”, line 102, in train
A_train = self._A[self.train_set, :]
numpy.core.exceptions.MemoryError: Unable to allocate 7.87 GiB for an array with shape (64152, 16469) and data type float64
Traceback (most recent call last):
File “fit.py”, line 197, in
fcs_raw, fcs_rot = fit_paras(name, sc_nstruct, cs_1, sys_name, prim, supercell, cutoffs, sd[1])
File “fit.py”, line 97, in fit_paras
opt = Optimizer(sc.get_fit_data(), train_size=0.99, seed=seed
)
File “/gpfs/share/home/2201112409/anaconda3/envs/p3/lib/python3.8/site-packages/hiphive/structure_container.py”, line 214, in get_fit_data
return np.vstack(M_list), np.hstack(f_list)
File “<array_function internals>”, line 180, in vstack
File “/gpfs/share/home/2201112409/anaconda3/envs/p3/lib/python3.8/site-packages/numpy/core/shape_base.py”, line 282, in vstack
return _nx.concatenate(arrs, 0)
File “<array_function internals>”, line 180, in concatenate
numpy.core.exceptions.MemoryError: Unable to allocate 7.95 GiB for an array with shape (64800, 16469) and data type float64
Traceback (most recent call last):
File “fit.py”, line 197, in
File “fit.py”, line 97, in fit_paras
opt = Optimizer(sc.get_fit_data(), train_size=0.99, seed=seed
)
File “/gpfs/share/home/2201112409/anaconda3/envs/p3/lib/python3.8/site-packages/hiphive/structure_container.py”, line 214, in get_fit_data
return np.vstack(M_list), np.hstack(f_list)
File “<array_function internals>”, line 180, in vstack
File “/gpfs/share/home/2201112409/anaconda3/envs/p3/lib/python3.8/site-packages/numpy/core/shape_base.py”, line 282, in vstack
return _nx.concatenate(arrs, 0)
File “<array_function internals>”, line 180, in concatenate
numpy.core._exceptions.MemoryError: Unable to allocate 7.95 GiB for an array with shape (64800, 16469) and data type float64
Traceback (most recent call last):
File “fit.py”, line 197, in
fcs_raw, fcs_rot = fit_paras(name, sc_nstruct, cs_1, sys_name, prim, supercell, cutoffs, sd[1])
File “fit.py”, line 114, in fit_paras
read_sheng.write_shengBTE_fc4(‘./’+name+‘/’+‘FORCE_CONSTANTS_4TH’, fcs_raw, atoms_unit, 1e-5, cut_4th)
AttributeError: module ‘hiphive.input_output.shengBTE’ has no attribute ‘write_shengBTE_fc4’

I think this might be the problem, you dont have enough memory.

Thank you so much for you positive response. I have resolved the memory issue now to resolve the 4th order issue i comment out in the script.

Luckily when we comment out the 4th order in the python script it provide excellent results without any kind of error.