Install python lammps in a non-standard directory

Dear all,

I am trying to install LAMMPS v2Aug2023.update1 in a cluster.
I am using EasyBuild to do this, but it does not succeed to install the python lammps/ directoy into non standard: $LAMMPS_DIR/lib64/python/python3.10/site-package

I use cmake and enable python, but the system does not install the python libraries, either in the install directory neither in the site-package on the python directory.

I also tried with : make install-python , defining exporting the CMAKE_INSTALL_PREFIX to the non-standard directory, but when doing it, it install the lammps libraries into the standard $PYTHON lib directory.

Could any of you help me?

Those are the options I use:
cmake -DCMAKE_INSTALL_PREFIX=/software/easybuild/x86_64/software/LAMMPS/2Aug2023-foss-2022a.update1-kokkos -DBoost_NO_SYSTEM_PATHS=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF -DBOOST_ROOT=/software/easybuild/x86_64/software/Boost/1.79.0-GCC-11.3.0 -DLAMMPS_EXCEPTIONS=yes -DBUILD_TOOLS=on -DBUILD_DOC=off -DWITH_GZIP=yes -DWITH_PNG=yes -DWITH_JPEG=yes -DDOWNLOAD_EIGEN3=no -DEIGEN3_INCLUDE_DIR=/software/easybuild/x86_64/software/Eigen/3.4.0-GCCcore-11.3.0/include/Eigen -DEigen3_DIR=/software/easybuild/x86_64/software/Eigen/3.4.0-GCCcore-11.3.0/share/eigen3/cmake/ -DPKG_AMOEBA=on -DPKG_ASPHERE=on -DPKG_ATC=on -DPKG_AWPMD=on -DPKG_BOCS=on -DPKG_BODY=on -DPKG_BPM=on -DPKG_BROWNIAN=on -DPKG_CG-DNA=on -DPKG_CG-SPICA=on -DPKG_CLASS2=on -DPKG_COLLOID=on -DPKG_COLVARS=on -DPKG_COMPRESS=on -DPKG_CORESHELL=on -DPKG_DIELECTRIC=on -DPKG_DIFFRACTION=on -DPKG_DIPOLE=on -DPKG_DPD-BASIC=on -DPKG_DPD-MESO=on -DPKG_DPD-REACT=on -DPKG_DPD-SMOOTH=on -DPKG_DRUDE=on -DPKG_EFF=on -DPKG_ELECTRODE=on -DPKG_EXTRA-COMPUTE=on -DPKG_EXTRA-DUMP=on -DPKG_EXTRA-FIX=on -DPKG_EXTRA-MOLECULE=on -DPKG_EXTRA-PAIR=on -DPKG_FEP=on -DPKG_GRANULAR=on -DPKG_H5MD=on -DPKG_INTERLAYER=on -DPKG_KIM=on -DPKG_KSPACE=on -DPKG_LATBOLTZ=on -DPKG_LEPTON=on -DPKG_MACHDYN=on -DPKG_MANIFOLD=on -DPKG_MANYBODY=on -DPKG_MC=on -DPKG_MDI=on -DPKG_MEAM=on -DPKG_MGPT=on -DPKG_MISC=on -DPKG_ML-IAP=on -DPKG_ML-PACE=on -DPKG_ML-POD=on -DPKG_ML-RANN=on -DPKG_ML-SNAP=on -DPKG_MOFFF=on -DPKG_MOLECULE=on -DPKG_MOLFILE=on -DPKG_MPIIO=on -DPKG_NETCDF=on -DPKG_OPT=on -DPKG_ORIENT=on -DPKG_PERI=on -DPKG_PHONON=on -DPKG_PLUGIN=on -DPKG_PLUMED=on -DPKG_POEMS=on -DPKG_PTM=on -DPKG_PYTHON=on -DPKG_QEQ=on -DPKG_QTB=on -DPKG_REACTION=on -DPKG_REAXFF=on -DPKG_REPLICA=on -DPKG_RIGID=on -DPKG_SCAFACOS=on -DPKG_SHOCK=on -DPKG_SMTBQ=on -DPKG_SPH=on -DPKG_SPIN=on -DPKG_SRD=on -DPKG_TALLY=on -DPKG_UEF=on -DPKG_VORONOI=on -DPKG_VTK=on -DPKG_YAFF=on -DPKG_INTEL=on -DBUILD_MPI=yes -DBUILD_OMP=yes -DPKG_OPENMP=on -DFFT=FFTW3 -DFFT_PACK=array -DPKG_KOKKOS=on -DKokkos_ENABLE_OPENMP=yes -DKokkos_ARCH_SKX=yes -DCMAKE_INSTALL_LIBDIR=lib64 -DPython_EXECUTABLE=/software/easybuild/x86_64/software/Python/3.10.4-GCCcore-11.3.0/bin/python -DPYTHON_EXECUTABLE=/software/easybuild/x86_64/software/Python/3.10.4-GCCcore-11.3.0/bin/python -DPYTHON_LIBRARY=/software/easybuild/x86_64/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so -DPYTHON_INCLUDE_DIR=/software/easybuild/x86_64/software/Python/3.10.4-GCCcore-11.3.0/include/python3.10 -DBUILD_SHARED_LIBS=ON /dev/shm/easybuild/LAMMPS/2Aug2023/foss-2022a.update1-kokkos/lammps-stable_2Aug2023_update1/cmake

Thank you !

This sounds more like a question for the EasyBuild folks due to a problem with they install/setup scripts/configuration. The LAMMPS developers have no control over what they do and how they orchestrate the LAMMPS installation.

make install-python does not pay attention to any CMake settings. If available, the python module should be already installed into the proper location by make install.

However, after executing it, there should be a wheel file (e.g. lammps-2023.8.2-cp310-cp310-linux_x86_64.whl) in the build folder and you can install that file wherever you want using pip by using the suitable flags (after uninstalling it from the default location).

First of all, thank you for your fast reply !

Sure. I understand this, for that reason I wrote the cmake command and options, just to be sure everything was OK.

FINALLY, I tried a clean installation:

mkdir build (but outside of the lammps directory)
cd  build 
cmake -DCMAKE_INSTALL_PREFIX=/mypath/2Aug2023-foss-2022a.update1-kokkos  [my options and PKG] -DPYTHON_EXECUTABLE=mypthon -DBUILD_SHARED_LIBS=ON  ../lammps-stable_2Aug2023_update1/cmake 
make -j 10
make install

In that point, the installation processes complained when installing the python module saying :

version.h not found...

since it tries to find it in a parent folder of the build directory (i.e. buiddir/../src/version.h)

I have solved it by exporting the variable LAMMPS_VERSION_FILE manually to the correct path where:
lammps-stable_2Aug2023_update1/src/version.h"

I have checked that the python/setup.py has changed from the version 4May2022.
I don’t know if the problem can come from there . Currently, if the LAMMPS_VERSION_FILE is not defined,
LAMMPS_DIR is set as the parent directory of the build directory:

versionfile = os.environ.get("LAMMPS_VERSION_FILE")
if not versionfile:
    # allows installing and building wheel from current directory
    LAMMPS_DIR = os.path.realpath(os.path.join(os.environ['PWD'], '..'))
    versionfile = os.path.join(LAMMPS_DIR, 'src', 'version.h')

Thank you !
quim

Besides, I had to add this lines in cmake/CMakeList.txt that were in the lammps v4May2022:

+###############################################################################
+# Add LAMMPS python module to "install" target. This is taylored for building
+# LAMMPS for package managers and with different prefix settings.
+# This requires either a shared library or that the PYTHON package is included.
+###############################################################################
+if(BUILD_SHARED_LIBS OR PKG_PYTHON)
+  if(CMAKE_VERSION VERSION_LESS 3.12)
+    find_package(PythonInterp) # Deprecated since version 3.12
+    if(PYTHONINTERP_FOUND)
+        set(Python_EXECUTABLE ${PYTHON_EXECUTABLE})
+    endif()
+  else()
+    find_package(Python COMPONENTS Interpreter)
+  endif()
+  if(Python_EXECUTABLE)
+    file(MAKE_DIRECTORY ${CMAKE_BINARY_DIR}/python)
+    install(CODE "execute_process(COMMAND ${Python_EXECUTABLE} setup.py build -b ${CMAKE_BINARY_DIR}/python install --prefix=${CMAKE_INSTALL_PREFIX} --root=\$ENV{DESTDIR}/ WORKING_DIRECTORY ${LAMMPS_PYTHON_DIR})")
+  endif()
+endif()

Thanks for the updates. Now I remember some more details. This was changed to accommodate some packaging scenarios where this process is not tolerated. Please see cmake: remove pip install from install target by rbberger · Pull Request #3689 · lammps/lammps · GitHub. I would assume this was in relation to build RPMs for EPEL and/or LAMMPS with Spack.
So it seems that EasyBuild would have to be adapted to follow what these packaging tools do.

At the moment “make install” is a bit of a step child and not fully supported. It was added primarily to simplify building RPM packages. For a full LAMMPS support, this is missing the definition of a consistent LAMMPS C++ API and changes to the corresponding header files to reside in “lammps/header.h” instead of “header.h”. This has not happened since historically LAMMPS was not “installed” and the regular makefiles do no have an “install” target. Instead people have used LAMMPS “in place” for coupling to other codes etc. and also used the python module via setting PYTHONPATH and LD_LIBRARY_PATH.

As already mentioned in a previous post, a workaround would be to run “make install-python”, then “pip uninstall lammps”, to remove it from the undesired general user or system location and instead manually install the resulting .whl file (aka wheel file) from the build folder into the desired location (either using pip or simply extracting the content to the desired location with “unzip” since the wheel file is basically a zip archive).

Great !
I will try to do it like you recommend.

Thank you for all the informaction and help.

Please also see: 2.2. Installation — LAMMPS documentation where it also provides instructions for building a wheel without installing it.
And User Guide - pip documentation v23.3.1 for how to install local wheel files with pip.