Aeon and Rocm installation

[sylthi]

Is there something that i need to know if I want to install Rocm on an aeon machine?
How do I install it if I want to use the GPU of a RX9060XT with Ollama (in a podman container)?
Thanks in advance !

[stendler]

I run the ollama rocm image in podman without issues. And if I remember correctly, I did not need to install rocm directly on the host.

I use it with my own systemd user service (for reference on how to run the container):

# ~/.config/systemd/user/container-ollama.service
# autogenerated by Podman 5.3.2
# Thu Feb 13 15:59:19 CET 2025

[Unit]
Description=Podman container-ollama.service
Documentation=man:podman-generate-systemd(1)
Wants=network-online.target
After=network-online.target
RequiresMountsFor=%t/containers

[Service]
Environment=PODMAN_SYSTEMD_UNIT=%n
Restart=on-failure
TimeoutStopSec=70
Environment=CONTAINER_NAME=ollama
ExecStart=%h/bin/podman-start-or-run.sh \
        --cidfile=%t/%n.ctr-id \
        --cgroups=no-conmon \
        --sdnotify=conmon \
        --pull newer \
        --security-opt label=type:container_runtime_t \
        --group-add keep-groups \
        --device /dev/kfd \
        --device /dev/dri \
        --volume ollama:/root/.ollama \
        --label "io.containers.autoupdate=registry" \
        -p 11434:11434 \
        --name ollama docker.io/ollama/ollama:rocm
ExecStop=/usr/bin/podman stop \
        --ignore -t 10 \
        --cidfile=%t/%n.ctr-id
Type=notify
NotifyAccess=all

[Install]
WantedBy=default.target

where `~/bin/podman-start-or-run.sh` (click to expand)

#!/bin/sh

# either start an existing container or run a new one, if it does not exist already
# issue: podman autoupdate just updates the image and assumes the service stop/start will remove the container and create a new one with the new image
# -> we need to check, if the container exists already, and if its image is still the newest for the tag

run_args="$*"


# we need to know the container name
# Sure, that is defined in the run arguments, but I don't want to parse that here
# Require it to be set as env vars
if [ -z "$CONTAINER_NAME" ]; then
  echo >&2 "Error: No CONTAINER_NAME specified."
  exit 1
fi

container=$(podman container inspect --format '{{.Name}}' "$CONTAINER_NAME")
container_image_name=$(podman container inspect --format '{{.ImageName}}' "$CONTAINER_NAME")
container_image_id=$(podman container inspect --format '{{.Image}}' "$CONTAINER_NAME")
latest_image_id=$(podman image inspect --format '{{.ID}}' "$container_image_name")

set -e # now propagate any error of failing commands
if [ "$container_image_id" = "$latest_image_id" ]; then
  if [ -n "$container" ]; then
    podman start "$CONTAINER_NAME"
    exit 0
  fi
fi

# container does not exist or needs to be re-created to run the latest image
podman run --replace --detach $run_args

You can of course use podman run directly or a quadlet instead.