Topic Guide

AMD RDNA4 LLM Inference

ZINC supports RDNA4 consumer and workstation GPUs through both Vulkan and ROCm.

Practical Answer

If you want local LLM inference on AMD RDNA4, ZINC offers independently tuned Vulkan and ROCm paths. Vulkan is the broad compatibility route; ROCm uses HIP and dedicated gfx1201 kernels. The important performance split is decode versus prefill: decode is mostly memory and scheduling, while prefill needs batched kernels and model-aware tiling.

When This Page Helps

Use this page for readers choosing AMD hardware or validating whether Vulkan local inference is real on RDNA4. The page should answer the hardware, driver, benchmark, and baseline questions before sending them to deep posts.

Status Snapshot

Best current answer
RDNA4 local inference is real through both Vulkan and ROCm. The practical question is which backend, memory class, and driver stack best fit the workload.
Reader problem
Readers need hardware fit, driver caveats, reproducible benchmark method, and a clear llama.cpp comparison before they trust the result.
Main bottleneck
Decode is usually weight and scheduling bound. Prefill is a different workload and often needs batched kernels plus fewer host-side submits.

Action Plan

  1. Pick the memory budget first The R9700-class 32 GB card changes which Qwen and Gemma models are realistic. RX 9070-class cards share RDNA4 behavior but fit fewer large targets.
  2. Benchmark against llama.cpp Use the same machine, model file, quantization, prompt, output cap, warmup policy, and backend residency. Cross-machine screenshots are not evidence.
  3. Tune the platform before the shader Driver version, GECC, cooperative matrix flags, ASPM, and stale GPU processes can move results enough to hide real kernel work.
  4. Report prefill and decode separately RDNA4 decode can look strong while prefill remains the user-visible latency limiter. A good post shows both phases.

Operator Checklist

  • Record the exact GPU Name the card, VRAM, driver, Mesa version, and Vulkan device. RDNA4 is not a single performance number.
  • Clean the node Stop stale zinc, llama.cpp, and benchmark processes before publishing a comparison.
  • Lock the run shape Use the same model file, quantization, prompt, max tokens, context, warmup, and endpoint across engines.
  • Separate CLI from server CLI decode, raw HTTP completion, and chat completion measure different parts of the stack.

What Matters

  • RDNA4 can run useful local LLMs through either ZINC Vulkan or ZINC ROCm, with each backend measured independently.
  • The R9700 is the strongest ZINC tuning target because 32 GB VRAM changes which Qwen and Gemma models fit.
  • Prefill and decode are different workloads; a fast decode loop does not imply fast time-to-first-token.
  • llama.cpp is the comparison baseline, while ZINC maintains dedicated AMD kernels for its supported backends.

What To Measure

Prompt tok/s
Shows whether prefill kernels and command submission are healthy.
Decode tok/s
Useful only when the model, quantization, prompt, and output length are included.
Latency distribution
For server mode, include TTFT plus p50 and p95 request latency under the stated concurrency.
Platform state
Record driver version, GECC, cooperative matrix flag, ASPM policy, and thermal throttling clues.

Common Traps

  • Blending backend results Vulkan and ROCm have different runtime and kernel behavior. Publish them as separate datasets instead of mixing their best numbers.
  • Comparing dirty runs Background GPU users, cold builds, one-token completions, and changed prompts will swamp the signal.
  • Treating RDNA4 as one number R9700, RX 9070 XT, and future workstation cards share architecture but not memory capacity, clocks, thermals, or deployment fit.

Useful Next Posts

  • RX 9070 XT local LLM guide A 16 GB practical guide for what fits, what needs offload, and what ZINC runs well today.
  • R9700 versus RX 9070 XT A memory-class comparison using the same Qwen and Gemma prompts.
  • RDNA4 benchmark hygiene checklist A reproducibility post covering drivers, stale processes, endpoint choice, and prompt shape.
How we made AMD LLM inference 4x faster on a single GPU The early RDNA4 speedup story and what changed in the decode path. What broke first in local LLM inference on AMD RDNA4 The first correctness failures: attention, KV cache, RoPE, MoE, SSM, and tokenizer drift. The broken Vulkan shaders keeping AMD RDNA4 inference stuck at 4 tok/s How shader debugging moved work back onto the GPU. Why RDNA4 prefill for Qwen3.5-35B is stuck at 25 tok/s Why prefill needs different kernels than decode on AMD. How AMD Qwen decode passed llama.cpp in six weeks The public comparison point against llama.cpp on the R9700.

Latest Research Notes

  1. Qwen 3.8 27B Shipped Yesterday. Here's What ZINC's Day-One Support Actually Took.
  2. The KV cache lever the swarm keeps pulling stops being free below 8 bits
  3. Eight agents share one system prompt, so store the prefix KV once
  4. Free RDNA4 preemption ends where paused KV caches fill the 16 GB card
  5. A long background turn stalls the foreground agent until RDNA4 preempts the decode slot
  6. A deeper queue cannot fix an oversubscribed RDNA4 agent swarm
  7. Turn-length variance, not load, sets an RDNA4 agent swarm's tail latency
  8. Little's Law caps a responsive RDNA4 agent swarm near twelve
  9. The foreground agent decodes seven times faster than you read
  10. First-come scheduling starves an RDNA4 swarm; fair queuing shares the card
  11. The datacenter disaggregates prefill and decode; a single RDNA4 card fuses them
  12. Chunked prefill keeps an RDNA4 swarm decoding while one agent's prompt loads

Run It

Run LLMs on AMD GPUs with ROCm or Vulkan The fastest path from clone to local AMD, Intel Arc, or Apple Silicon inference. AMD RDNA3/RDNA4 GPU Reference Hardware, memory, wave execution, and Vulkan compute details. RDNA4 Tuning Guide Driver, shader, cooperative matrix, and benchmark tuning notes. ZINC Benchmarks Current public measurements across AMD, Metal, and baseline engines.

Common Questions

Should I use ROCm or Vulkan on AMD RDNA4?

Both are supported ZINC backends. ROCm is the primary Radeon path, using HIP with dedicated gfx1201 kernels, and it posts by far the widest prompt-processing margins; Vulkan needs no ROCm install and currently generates tokens faster on several models. Compare the separate backend tabs for the model and phase you care about.

Which RDNA4 GPU is the best target for ZINC?

The Radeon AI PRO R9700 is the primary tuning target because it combines RDNA4 behavior with a 32 GB memory budget. RX 9070-class cards share the architecture but fit fewer large models.

Why compare to llama.cpp?

llama.cpp is the strongest widely used local baseline with Vulkan and Metal support. ZINC compares against it on the same machine to avoid misleading cross-hardware numbers.