Leap + TW: OpenCL on AMD with Mesa 3D + headless amdgpu-pro

English Hardware
1

At last it works.

Trying to get OpenCL with AMD APU Ryzen 3 3200G.
I had many troubles with AMD ROCm.

Uninstalled it and installed headless amdgpu-pro OpenCL drivers:

  1. Download amdgpu-pro from AMD website. I chose amdgpu-pro-20.40-1147287-sle-15.2.tar.xz for Vega64.

  2. Run

amdgpu-pro-install --opencl=legacy,pal --headless

You may add:
“-y” for non-interactive install
“–no-dkms” for non-DKMS install. IDK whether it is needed or not.

  1. Reboot after install.

Amdgpu-pro has OpenCL image support.
OpenCL works 4-8 times faster than with ROCm.
Tested with clpeak and luxmark-linux64-v3.1.
luxmark on CPU works faster than with ROCm - ILL additionally using GPU.
FAHBench 2.3.2 works at last.
SVP 4.5 works nicely with Pro, but with ROCm drops frames and hangs VLC.

Blender benchmark-launcher 2.0.5: requires ‘GLIBC_2.27’.
Run benchmark-launcher-cli or as described here.
On CPU - 100% load and warms up, on GPU (OpenCL) - zero CPU load.
But scene bmw27 takes 8,5 min on CPU compared to 14,3 min on GPU.

I have POCL installed, but still cannot use CPU for OpenCL.

2 
:~> glxinfo | grep -i OpenGL
**OpenGL vendor string: AMD**
OpenGL renderer string: AMD RAVEN (**DRM 3.33.0**, 5.3.18-lp152.57-default, LLVM 9.0.1)
OpenGL core profile version string: 4.6 (Core Profile) Mesa 20.3.2
OpenGL core profile shading language version string: 4.60
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 4.6 (Compatibility Profile) Mesa 20.3.2
OpenGL shading language version string: 4.60
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 3.2 Mesa 20.3.2
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20
OpenGL ES profile extensions:

Previously this was


glxinfo | grep -i opengl
**OpenGL vendor string: X.Org**
OpenGL renderer string: AMD RAVEN (**DRM 3.36.0**, 5.5.5-3.g5157fff-default, LLVM 9.0.1)
OpenGL core profile version string: 4.6 (Core Profile) Mesa 20.0.0
OpenGL core profile shading language version string: 4.60
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 4.6 (Compatibility Profile) Mesa 20.0.0
OpenGL shading language version string: 4.60
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 3.2 Mesa 20.0.0
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20
OpenGL ES profile extensions:

ILL installing amdgpu-pro has changed amdgpu driver from Mesa’s to AMD’s.

Previous tests were conducted with video memory = 1 GiB. Possibly Blender test needs more.

3

Does AMD unofficially support APUs with amdgpu-pro driver?

4

With amdgpu-pro OpenCL headless driver:

:~> clpeak

Platform: AMD Accelerated Parallel Processing
  Device: gfx902
    Driver version  : 3180.7 (PAL,HSAIL) (Linux x64)
    Compute units   : 8
    Clock frequency : 1250 MHz

    Global memory bandwidth (GBPS)
      float   : 42.63
      float2  : 44.12
      float4  : 45.13
      float8  : 46.01
      float16 : 45.59

    Single-precision compute (GFLOPS)
      float   : 1252.46
      float2  : 1251.55
      float4  : 1249.05
      float8  : 1238.10
      float16 : 1225.67

    half-precision compute (GFLOPS)
      half   : 1250.89
      half2  : 2421.75
      half4  : 2409.56
      half8  : 2381.20
      half16 : 2237.11

    Double-precision compute (GFLOPS)
      double   : 79.14
      double2  : 79.10
      double4  : 79.01
      double8  : 78.85
      double16 : 78.41

    Integer compute (GIOPS)
      int   : 252.97
      int2  : 252.96
      int4  : 252.96
      int8  : 252.96
      int16 : 252.94

    Transfer bandwidth (GBPS)
      enqueueWriteBuffer         : 42.38
      enqueueReadBuffer          : 14.94
      enqueueMapBuffer(for read) : 35320.46
        memcpy from mapped ptr   : 14.29
      enqueueUnmap(after write)  : 78951.60
        memcpy to mapped ptr     : 14.34

    Kernel launch latency : 48.96 us

With ROCm driver (v. 3.3 and 3.10):

:~> clpeak

Platform: AMD Accelerated Parallel Processing
  Device: gfx902+xnack
    Driver version  : 3212.0 (HSA1.1,LC) (Linux x64)
    Compute units   : 11
    Clock frequency : 1250 MHz

    Global memory bandwidth (GBPS)
      float   : 12.66
      float2  : 12.75
      float4  : 12.76
      float8  : 12.74
      float16 : 12.20

    Single-precision compute (GFLOPS)
      float   : 202.43
      float2  : 202.28
      float4  : 201.77
      float8  : 200.75
      float16 : 198.48

    half-precision compute (GFLOPS)
      half   : 202.33
      half2  : 396.88
      half4  : 394.08
      half8  : 389.46
      half16 : 384.83

    Double-precision compute (GFLOPS)
      double   : 12.76
      double2  : 12.76
      double4  : 12.71
      double8  : 12.71
      double16 : 12.58

    Integer compute (GIOPS)
      int   : 40.81
      int2  : 40.81
      int4  : 40.81
      int8  : 40.80
      int16 : 40.79

    Transfer bandwidth (GBPS)
      enqueueWriteBuffer         : 2.11
      enqueueReadBuffer          : 5.34
      enqueueMapBuffer(for read) : 33925.49
        memcpy from mapped ptr   : 5.23
      enqueueUnmap(after write)  : 70409.30
        memcpy to mapped ptr     : 5.41

    Kernel launch latency : -500964480.00 us

Legacy = orca drivers are not working with Vega = GCN5.

5

After updating Mesa to 20.3.3:


:~> glxinfo | grep -i OpenGL
OpenGL vendor string: **AMD**
OpenGL renderer string: AMD RAVEN (**DRM 3.40.0**, 5.3.18-lp152.57-default, LLVM 9.0.1)
OpenGL core profile version string: 4.6 (Core Profile) Mesa 20.3.3
OpenGL core profile shading language version string: 4.60
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 4.6 (Compatibility Profile) Mesa 20.3.3
OpenGL shading language version string: 4.60
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 3.2 Mesa 20.3.3
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20
OpenGL ES profile extensions:

Then updating kernel to 5.3.18-lp152.60-default:


:~> glxinfo | grep -i OpenGL
OpenGL vendor string: **AMD**
OpenGL renderer string: AMD RAVEN (**DRM 3.33.0**, 5.3.18-lp152.60-default, LLVM 9.0.1)
OpenGL core profile version string: 4.6 (Core Profile) Mesa 20.3.3
OpenGL core profile shading language version string: 4.60
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 4.6 (Compatibility Profile) Mesa 20.3.3
OpenGL shading language version string: 4.60
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 3.2 Mesa 20.3.3
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20
OpenGL ES profile extensions:

ILL Mesa 3D and amdgpu-pro are fighting with each other.
Amdgpu-pro is using dkms to get in its drivers.

If someone have some clarifications - please, provide them.

6

Minimal install with Leap 15.2 + amdgpu-pro-20.40-1147287-sle-15.2.tar.xz with default Mesa 3D, X11, no DKMS:

Choose only


amdgpu-core
amdgpu-pro-core
amdgpu-pro-versionlist 
amdgpu-versionlist 
clinfo-amdgpu-pro (gives info only)
ocl-icd-amdgpu-pro
ocl-icd-amdgpu-pro-devel (possibly not needed)
opencl-amdgpu-pro-comgr
opencl-amdgpu-pro-icd

To get packages unzip files from AMD archive with driver to a needed folder. Then use folder as repository.
More info at https://en.opensuse.org/SDB:AMDGPU-PRO .
Needs checks after kernel update.

Drivers versions:


:~> glxinfo | grep -i OpenGL
OpenGL vendor string: X.Org
OpenGL renderer string: AMD RAVEN (DRM 3.33.0, 5.3.18-lp152.63-default, LLVM 9.0.1)
OpenGL core profile version string: 4.5 (Core Profile) Mesa 19.3.4
OpenGL core profile shading language version string: 4.50
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 4.5 (Compatibility Profile) Mesa 19.3.4
OpenGL shading language version string: 4.50
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 3.2 Mesa 19.3.4
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20
OpenGL ES profile extensions:
7

Hi
No downgrading of driver here (but I’m on Tumbleweed, plus use multiple GPU’s), I’m using the kernel driver. You need to slow down!!!

NO DKMS in use… hence the --dkms, use default openSUSE Mesa etc. You are on a Raven card, it for sure needs later packages, running on Leap is probably going to be an issue…


switcherooctl launch -g 1 glxinfo | grep -i OpenGL

OpenGL vendor string: AMD
OpenGL renderer string: Radeon RX550/550 Series (POLARIS12, DRM 3.40.0, 5.10.12-1-default, LLVM 11.0.1)
OpenGL core profile version string: 4.6 (Core Profile) Mesa 20.3.4
OpenGL core profile shading language version string: 4.60
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 4.6 (Compatibility Profile) Mesa 20.3.4
OpenGL shading language version string: 4.60
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 3.2 Mesa 20.3.4
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20
OpenGL ES profile extensions:

The only files I have installed are;


 zypper se -si -r 5
Loading repository data...
Reading installed packages...

S  | Name                       | Type    | Version           | Arch   | Repository
---+----------------------------+---------+-------------------+--------+--------------------------------
i  | amdgpu-core                | package | 20.40-1147287     | noarch | AMD amdgpu Pro local repository
i  | amdgpu-pro-core            | package | 20.40-1147287     | noarch | AMD amdgpu Pro local repository
i+ | amdgpu-versionlist         | package | 20.40-1147287     | noarch | AMD amdgpu Pro local repository
i+ | clinfo-amdgpu-pro          | package | 20.40-1147287     | x86_64 | AMD amdgpu Pro local repository
i  | libdrm-amdgpu              | package | 1:2.4.100-1147287 | x86_64 | AMD amdgpu Pro local repository
i  | libdrm-amdgpu-common       | package | 1.0.0-1147287     | noarch | AMD amdgpu Pro local repository
i  | ocl-icd-amdgpu-pro         | package | 20.40-1147287     | x86_64 | AMD amdgpu Pro local repository
i  | opencl-amdgpu-pro-comgr    | package | 20.40-1147287     | x86_64 | AMD amdgpu Pro local repository
i+ | opencl-amdgpu-pro-icd      | package | 20.40-1147287     | x86_64 | AMD amdgpu Pro local repository
i+ | opencl-orca-amdgpu-pro-icd | package | 20.40-1147287     | x86_64 | AMD amdgpu Pro local repository
8

But you are installing libdrm-amdgpu?

I had hangs with FF when switched to X11:XOrg repo.
Using Leap 15.2 kernel is Ok.

9

Hi
Yes, replied in other thread…


ldd /usr/bin/blender | grep drm
    libva-drm.so.2 => /usr/lib64/libva-drm.so.2 (0x00007f2a9ff0b000)
    libdrm.so.2 => /opt/amdgpu/lib64/libdrm.so.2 (0x00007f2a9fcf2000)
10

But for what?
I’m trying right now to use system without this package.

11

Hi
Move the ld config files out and run ldconfig and it will disappear… Like I indicated, you will be hard pressed to come up with a workable solution on Leap with a Raven card.

12

ILL installing libdrm-amdgpu package for Raven APU is causing:

  1. Firefox hangs sometimes during work.
  2. Ark quits if trying to compress files too big in summed volume.

Possibly this is due to wrong video memory volume report for APUs.

Without libdrm-amdgpu GPU is not used by Ark for compression-decompression.
I can select “Unknown AMD GPU” in Blender for OpenCL usage, and it renders pictures, with less heat generation, but slower than with CPU.
So there is no HSA support with Linux + AMD APU (Intel may work).

13

Package libdrm_amdgpu1 came from Mesa 3D, package libdrm-amdgpu came from AMD.
Instead of adding libdrm-amdgpu I decided to replace libdrm_amdgpu1 with libdrm-amdgpu.
In YaST I marked libdrm_amdgpu1 for deletion, YaST added libdrm-amdgpu + libdrm-amdgpu-common.
After applying changes and reboot system looks stable (at least now).
Maybe I shall test it with libdrm-tools.

Have no issuises with FF and Ark right now (see previous message). ILL Ark uses OpenCL - runs ~ the same compared to J7Z (p7zip), but uses only 1 kernel.
Ark can compress big set of files - more in volume than dedicated video RAM.

I already tested OpenCL with Luxmark, Blender, FAHBench.
Please provide suggestions how I can test HSA - i.e. speeding up ordinary applications with GPGPU/OpenCL.
Candidates: archivers, picture browsers (working with jpeg).

14

I am using OpenCL part from amdgpu-pro-20.40 driver.
It is the last with PAL code for OpenCL.
Next drivers are using ROCr from ROCm and may run much slower than PAL code (it needs tests).

OpenCL code from amdgpu-pro needs amdgpu driver.
Linux kernel is using it for GCN3 and newer. For GCN1 and GCN2 use kernel parameters to get amdgpu instead of radeon drivers.
For Terascale 2 and Terascale 3 try to use Mesa 3D OpenCL drivers. Right now they lack image support, so OpenCL in Blender or SVP don’t work.
Older cards are not supported by Mesa 3D OpenCL drivers.

15

The latest drivers from AMD according to GCN generation:

GCN1 - 20.10 https://www.amd.com/en/support/kb/release-notes/rn-amdgpu-unified-linux-20-10
GCN2 - 20.40 https://www.amd.com/en/support/kb/release-notes/rn-amdgpu-unified-linux-20-40
GCN3 - possibly 21.30 https://www.amd.com/en/support/kb/release-notes/rn-amdgpu-unified-linux-21-30

16

The latest drivers from AMD according to GCN generation:

GCN1 - 20.10 https://www.amd.com/en/support/kb/release-notes/rn-amdgpu-unified-linux-20-10
GCN2 and GCN3 - still supported with 21.30 https://www.amd.com/en/support/kb/release-notes/rn-amdgpu-unified-linux-21-30

17

After upgrade from Leap 15.2 to 15.3: OpenCL works OK.

Radeon™ Software for Linux® 21.40.1 driver news:

  1. Unified with ROCm repo.
  2. GCN5 and newer use ROCm for OpenCL.
  3. GCN4 uses PAL OpenCL (but files have “orca” names).
  4. Dropped support for OpenCL with pre-GCN4 chips.
  5. Installer uses AMD repos, stops including files in itself.