AMD ROCm support

DLM supports AMD GPUs via ROCm as a Tier 2 backend: LoRA training and inference work, but QLoRA is refused and the CI coverage is weaker than the CUDA path.

What works

• Training: LoRA on bf16-capable AMD GPUs.
• Inference: dlm prompt uses the standard PyTorch path; MLX and bitsandbytes are not involved.
• Export: llama.cpp produces GGUF quantized weights. For ROCm-accelerated quantization, rebuild llama.cpp with HIP (see below); otherwise the default CPU build is used.

Supported GPUs

The bf16-capable allowlist is enforced in dlm.hardware.capabilities based on torch.cuda.get_device_properties(0).gcnArchName. Unsupported arches fall back to fp16 (still functional, just slower per token on weight-heavy layers).

What doesn't work

QLoRA is refused on ROCm. bitsandbytes ROCm builds are upstream-unstable — 4-bit quantized matmuls silently return wrong values on several arch/driver combinations. We refuse the combination rather than risk corrupt gradients. Use adapter: lora in your .dlm frontmatter.

Multi-GPU ROCm is out of scope for this sprint. Sprint 23's multi-GPU work targets CUDA first; ROCm multi-GPU is a follow-on.

Software prerequisites

• ROCm ≥ 5.7; 6.0+ preferred. We test against 6.0 and 6.2.
• PyTorch with HIP build — install via the ROCm wheels from pytorch.org. The torch.version.hip attribute must be non-None.
• FlashAttention 2 (optional): AMD's flash_attn fork is the package name on ROCm. Install for CDNA (MI200/MI300); RDNA3 support is experimental. If flash_attn is not importable or the arch is not on the allowlist, SDPA is used instead.

Determinism posture

The doctor reports determinism_class: best-effort on ROCm. ROCm's deterministic kernels exist but are not as thorough as CUDA's; fp match may drift across PyTorch/ROCm upgrades even with a pinned seed.

Rebuilding llama.cpp with ROCm

The default vendored llama.cpp binary is CPU-only. Build a ROCm version once for faster quantization:

# Set your GPU arch
export AMDGPU_TARGETS="gfx1100"   # RDNA3
# export AMDGPU_TARGETS="gfx90a"  # MI200
# export AMDGPU_TARGETS="gfx942"  # MI300

scripts/build-llama-cpp-rocm.sh

The script writes to vendor/llama.cpp/build-rocm/. To make dlm export prefer this build, point the runner at it:

export DLM_LLAMA_CPP_BUILD=vendor/llama.cpp/build-rocm

DLM_LLAMA_CPP_BUILD is honored by dlm.export.vendoring — when set, it's checked before the default vendor dir for each binary, so the ROCm-accelerated llama-quantize / llama-imatrix win over any CPU-only build left behind from scripts/bump-llama-cpp.sh build.

CI / testing

No default ROCm CI runner exists. Contributors with ROCm hardware can run the gated smoke:

DLM_ENABLE_ROCM_SMOKE=1 uv run pytest tests/integration/hardware/test_rocm_train_smoke.py -v

A scheduled self-hosted runner is the intended deployment; contact the maintainers if you'd like to host one.