Linking with LAPACK when compiling LAMMPS

LAMMPS LAMMPS Development
1

Dear LAMMPS users (and devs),
I have written a function that requires inversion, multiplication of complex matrices.
Before including this function in LAMMPS code, I have tested it outside independently, built with LAPACK

#include <complex>
#include <cmath>
#include <algorithm>
#include <iostream>
#include <vector>

using std::complex;
using std::vector;

extern "C" {
void zgesv_(int* n, int* nrhs, complex<double>* a, int* lda, int* ipiv, complex<double>* b, int* ldb, int* info);
void zgemm_(char* transa, char* transb, int* m, int* n, int* k, complex<double>* alpha,
            complex<double>* a, int* lda, complex<double>* b, int* ldb,
            complex<double>* beta, complex<double>* c, int* ldc);
void zgemv_(char* trans, int* m, int* n, complex<double>* alpha,
            complex<double>* a, int* lda, complex<double>* x, int* incx,
            complex<double>* beta, complex<double>* y, int* incy);
}


void calculate_energy_absorption() {
//other parts of code.....

        zgesv_(called);

        if (i == 0) {
            std::copy(p.begin(), p.end(), transfer_matrix.begin());
        } else {
            zgemm_(called);
        }
    
for (int i = 0; i < n_bulk; i++) {
    zgemv_(called);
}
//other parts of code
}


int main() {
//test_variable definition

calculate_energy_absorption();


return 0;
}

it compiles with the following command

g++ -o test test.cpp -std=c++11 -llapack -lopenblas

and I get the results I expect.

But when I migrate this function to the target fix code, am unable to compile it. Heres what I did

  • I included void calculate_energy_absorption(); in that fix class’s private scope in its header file.

  • In the cpp file

using namespace LAMMPS_NS;
  using namespace FixConst;
  using namespace MathConst;
  using std::complex;
  using std::vector;
  using std::cout;
  using std::endl;

  extern "C" {
  void zgesv_(int* n, int* nrhs, complex<double>* a, int* lda, int* ipiv, complex<double>* b, int* ldb, int* info);
  void zgemm_(char* transa, char* transb, int* m, int* n, int* k, complex<double>* alpha,
              complex<double>* a, int* lda, complex<double>* b, int* ldb,
              complex<double>* beta, complex<double>* c, int* ldc);
  void zgemv_(char* trans, int* m, int* n, complex<double>* alpha,
              complex<double>* a, int* lda, complex<double>* x, int* incx,
              complex<double>* beta, complex<double>* y, int* incy);
  }

added the above at the top and defined the function

void FixTTMMod::calculate_energy_absorption() {
//idential to the code i tested outside lammps
}
  • In CMake list, added the package to the condition that searches for LAPACK AND BLAS
if(PKG_MSCG OR PKG_ATC OR PKG_AWPMD OR PKG_ML-QUIP OR PKG_LATTE OR PKG_ELECTRODE OR PKG_EXTRA-FIX[**here**])
  enable_language(C)
  find_package(LAPACK)
  find_package(BLAS)
....

Then compiled as usual, lapack and blas is found

Found BLAS: /usr/lib/x86_64-linux-gnu/libopenblas.so  
-- Looking for cheev_
-- Looking for cheev_ - found
-- Found LAPACK: /usr/lib/x86_64-linux-gnu/libopenblas.so;-lm;-ldl

However, this happens at the end

[ 99%] Built target lammps
[ 99%] Building CXX object CMakeFiles/lmp.dir/home/sabik/lammps_t/src/main.cpp.o
[100%] Linking CXX executable lmp_fxv
/usr/bin/ld: liblammps_fxv.a(fix_ttm_mod.cpp.o): in function `LAMMPS_NS::FixTTMMod::calculate_energy_absorption()':
/home/sabik/lammps_t/src/EXTRA-FIX/fix_ttm_mod.cpp:1641: undefined reference to `zgesv_'
/usr/bin/ld: /home/sabik/lammps_t/src/EXTRA-FIX/fix_ttm_mod.cpp:1646: undefined reference to `zgemm_'
/usr/bin/ld: /home/sabik/lammps_t/src/EXTRA-FIX/fix_ttm_mod.cpp:1659: undefined reference to `zgemv_'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/lmp.dir/build.make:109: lmp_fxv] Error 1
make[1]: *** [CMakeFiles/Makefile2:187: CMakeFiles/lmp.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

Obviously, these function are defined in openblas and works independently. How to solve this linking issue?

2

Got it fixed. Made a new block for cmake list

if(PKG_EXTRA-FIX)
    find_package(LAPACK REQUIRED)
    find_package(BLAS REQUIRED)

    if(LAPACK_FOUND AND BLAS_FOUND)
        include_directories(${LAPACK_INCLUDE_DIRS} ${BLAS_INCLUDE_DIRS})
        target_link_libraries(lammps PRIVATE ${LAPACK_LIBRARIES} ${BLAS_LIBRARIES})
    else()
        message(FATAL_ERROR "LAPACK or BLAS not found")
    endif()
endif()

Now it works.

However, this raises another concern for me.
This particular function was previously written using armadillo. @akohlmey, professor, you informed me that I will not be able to contribute this code in the future if 3rd party library dependency is needed. You told me to use lapack instead. So, I have finally converted the code. I did not completely understand, but you said, lammps comes with a subset of lapack, so I can use that for no dependency on 3rd party. Hence, I added PKG_EXTRA-FIX to existing lapack linking routine in cmake list (it’s a hack job, which you really hate, I know). But, that is not working, as you can see. So I am explicitly linking to those libraries.

Does this make the code unsuitable for submission? There are other bells and whistles I am adding to TTM, like the proper em wave definition using transfer matrix, different type of coupling, spline objects, improving the FDM for handling ablation. One of the work is under review. I am hoping to submit the fix after another work. So, I needed to know.

3

You didn’t tell to link with BLAS and LAPACK anywhere.

This would not be required, since you already enabled the search earlier. Also this enforces the use of an explicit BLAS/LAPACK and is not compatible with the subset in the “lib/linalg” folder, which is required for platforms without a standard BLAS/LAPACK like Windows.

With the code in the base CMakeLists.txt file you can also force using the linalg lib subset with:

cmake -D USE_INTERNAL_LINALG=yes

So all that would be needed to add is:

if(PKG_EXTRA-FIX)
    target_link_libraries(lammps PRIVATE ${LAPACK_LIBRARIES} ${BLAS_LIBRARIES})
endif()