Lammps with Kokkos using SYCL for Intel PVC GPU

LAMMPS LAMMPS Installation
64

Great news, guys! I finally got this running. Here’s what I had to do.

  1. OneAPI toolkit from a container named
    intel/oneapi-hpckit:2025.0.0-0-devel-ubuntu24.04

  2. LAMMPS commit 43fbdc2d9385715ac01f9218defc5beca0afc853
    ( nearest tag: patch_19Nov2024-2-g43fbdc2d93 )

  3. cmake -C ./my-kokkos-sycl-intel.cmake -DPKG_MOLECULE=on -DPKG_RIGID=on -DPKG_KSPACE=on -DPKG_ML-SNAP=on -DFFT=MKL -DFFT_KOKKOS=MKL_GPU -DFFT_SINGLE=on ../cmake

  4. I modified that preset my-kokkos-sycl-intel.cmake to include the settings that generate AOT kernel compiles. I also turned off OpenMP because it’s nothing but trouble.

my-kokkos-sycl-intel.cmake (1.5 KB)

I haven’t tested all the features yet, but I confirmed that the Rhodopsin benchmark runs on a single GPU.

2 Likes
65

@rarensu Good to hear!

66

It turns out that cmake file I uploaded is no good. The hardcoded values for MPI are not correct, so no parallel is possible. Annoyingly, instead of complaining about it, cmake just defaulted to the builtin MPI stubs. After I commented out the line for MPI executable and instead turned on BUILD_MPI to encourage cmake to find MPI on its own, cmake was actually able to find and use it.

In addition to the variables @stamoor suggested (Jan 4), I also needed to do unset ZE_AFFINITY_MASK. For some reason, the latest Intel MPI refuses to do gpu/aware when that variable is set.

After rebuilding and setting all these annoying variables, I can now fully parallelize across multiple PVC GPUs. Even the kspace for Rhodopsin benchmark is working correctly. This is very exciting indeed.

I put my build in a container. If someone you know has PVCs and would like to try my build, I can share it.

1 Like