AMD openCL GPU installation with LAMMPS

ivanpivan · May 5, 2023, 5:01pm

Hello!

I cannot seem to build a version that can can use my GPU. My installation steps:

Download and unzip most recent feature release (28 March, 2023)
Build with cmake (for openCL): cmake -C …/cmake/presets/most.cmake -C …/cmake/presets/nolib.cmake -D PKG_GPU=on -D GPU_API=opencl …/cmake
make

After building is done, ocl_get_devices outputs:
“Found 0 platform(s).”

And when running executable on any lammps script:
“ERROR: Invalid OpenCL platform ID. (src/GPU/gpu_extra.h:77)”

I have a AMD Radeon RX 480. Obviously it seems like LAMMPS cannot identify my GPU. Are there any OpenCL drivers that are required? Is there any README or any further information that I might have missed regarding installation instructions? Thankful for any help!

akohlmey · May 5, 2023, 6:15pm

What specific graphics driver are you using? The standard Linux one or the vendor provided one?

Do you have the “clinfo” program installed? If yes, please provide the output of that command. That can help identify issues, since provides more details as ocl_get_devices.

Generally speaking, to use OpenCL with LAMMPS in the GPU package, you need to install/configure an OpenCL “ICD”. That usually means that with the graphics driver is a file ending in .icd in the folder /etc/OpenCL/verndors/ that contains the path to a .so file with the OpenCL runtime.

As an alternative, you can install AMD ROCm and configure for GPU_API=HIP.

ivanpivan · May 8, 2023, 10:50am

Hello again! Sorry for late response, have not had access to computer during weekend.

Installation

I tried installing AMD ROCm and configuring the build in LAMMPS with it on my OS Ubuntu 22.04.2 LTS (Jammy Jellyfish). I followed the steps at: AMD Documentation - Portal

1:

sudo apt-get update
wget https://repo.radeon.com/amdgpu-install/5.4.3/ubuntu/jammy/amdgpu-install_5.4.50403-1_all.deb 
sudo apt-get install ./amdgpu-install_5.4.50403-1_all.deb

2:
echo 'deb [arch=amd64 signed-by=/etc/apt/trusted.gpg.d/rocm-keyring.gpg] https://repo.radeon.com/amdgpu/5.4.3/ubuntu jammy main' | sudo tee /etc/apt/sources.list.d/amdgpu.list

3:
echo 'deb [arch=amd64 signed-by=/etc/apt/trusted.gpg.d/rocm-keyring.gpg] https://repo.radeon.com/rocm/apt/5.4.3 jammy main' | sudo tee /etc/apt/sources.list.d/rocm.list echo -e 'Package: *\nPin: release o=repo.radeon.com\nPin-Priority: 600' | sudo tee /etc/apt/preferences.d/rocm-pin-600 sudo apt-get update

4:
sudo apt install rocm-hip-sdk

Check post install actions and verifications:

dkms status output:
amdgpu/6.0.5-1581431.22.04, 5.19.0-41-generic, x86_64: installed

rocminfo sucessfully identifies my GPU:

…
Name: gfx803
Uuid: GPU-XX
Marketing Name: AMD Radeon ™ RX 480 Graphics
Vendor Name: AMD
Feature: KERNEL_DISPATCH
Profile: BASE_PROFILE
Float Round Mode: NEAR
Max Queue Number: 128(0x80)
Queue Min Size: 64(0x40)
Queue Max Size: 131072(0x20000)
…

LAMMPS build

I proceed to follow the steps given in 3.7. Packages with extra build options — LAMMPS documentation and 3.6. Include packages in build — LAMMPS documentation

export HIP_PLATFORM=amd
export HIP_PATH=/opt/rocm-5.4.3

cmake -C ../cmake/presets/most.cmake -C ../cmake/presets/nolib.cmake -D PKG_GPU=on GPU_API=HIP -D HIP_ARCH=gfx 803 -D CMAKE_CXX_COMPILER=hipcc ../cmake…

make -j 20

Test build:

in: examples/granular/

mpirun -np 20 lmp -in in.pour.drum works fine.

lmp -in in.pour.drum works fine.

lmp -sf gpu -in in.pour.drum output:
LAMMPS (28 Mar 2023)
ERROR: Invalid OpenCL platform ID. (src/GPU/gpu_extra.h:77)
Last command: package gpu 0

Note

Why does it try to identify an openCL platform when I built with HIP? Also, I had the option to use the following installation for ROcm:
sudo apt install rocm-hip-sdk rocm-opencl-sdk
Was I supposed to use this instead?

I just realized that I didnt include a -D flag between “PKG_GPU=on” and “GPU_API=HIP”. Could it be that it set to building openCL anyways?

I’m really clueless when it comes to all this stuff so I just kinda try my way, google until things “work”. Am I missing something obvious?

ivanpivan · May 8, 2023, 11:15am

Update:

Checking the output from last build it seems to have built with openCL API. This time I built correctly with GPU_API as HIP.

At last step of make when linking static libraries, etc I encounter the following warnings:
readelf: Warning: Unrecognized form: 0x22
readelf: Warning: Unrecognized form: 0x23
readelf: Warning: DIE at offset 0x2124 refers to abbreviation number 427 which does not exist
readelf: Warning: Bogus end-of-siblings marker detected at offset 20d1 in .debug_info section
readelf: Warning: Further warnings about bogus end-of-sibling markers suppressed

Number of warning outputs are a lot.

When running the same test script, all three cases previously mentioned under “Test build” works, and so at least I encounter no error when using the GPU. However, I’m not sure if the GPU is actually used. How can I confirm that it is given for example the LAMMPS logfile?

It seems that LAMMPS identifies the GPU aswell:

hip_get_devices output:
Found 1 platform(s).
Driver Version: 50422.4

Device 0: “AMD Radeon ™ RX 480 Graphics”
Type of device: GPU
Compute capability: 8
Double precision support: Yes
Total amount of global memory: 8 GB
Number of compute units/multiprocessors: 36
Number of cores: 6912
Total amount of constant memory: 2147483647 bytes
Total amount of local/shared memory per block: 65536 bytes
Total number of registers available per block: 65536
Warp size: 64
Maximum number of threads per block: 1024
Maximum group size (# of threads per block) 1024 x 1024 x 1024
Maximum item sizes (# threads for each dim) 2147483647 x 2147483647 x 2147483647
Clock rate: 1.266 GHz
Integrated: No
Compute mode: Default
Concurrent kernel execution: Yes

akohlmey · May 8, 2023, 11:45am

This hints at the GPU driver and libraries being compiled for a different compiler/linker toolkit version that what you have installed on your machine.

This is a rather pointless exercise, since there currently is no acceleration (GPU or otherwise) for the granular pair style at all.

I’ve looked up this GPU. It is rather old and since AMD often used multiple different GPU architectures for the same GPU model is is not easy to pinpoint the exact spects. Because of its age, it may not be fully supported by the GPU driver anymore.

GPU package specific information about the GPU kernels used is only printed to the screen, not log file.

You can check with the command overview: 5. Commands — LAMMPS documentation
sections 5.5 to 5.13. Accelerator package supported styles are indicated with a single character code in parentheses (GPU package support is indicated with ‘g’).

Google can only give you a starting point. The real information is provided in the respective documentation for the respective commands, in this case CMake. There is no way around spending some time reading through the relevant parts of those manuals. Guessing will just lead to incorrect conclusions, eventually. or wasted time.

ivanpivan · May 8, 2023, 1:17pm

You’re right. I’ve not read up enough on the documentation to realize that nothing in my LAMMPS script has any support for GPU. That settles it then…

Yup