AMD Radeon Vulkan
Best measured path today
5/5 decode wins
RDNA4 sweep beats llama.cpp on all headline rows.
Consumer GPU local LLM inference
Inference for the GPUs people actually own
ZINC is a Zig inference engine for GGUF models across consumer GPU backends: AMD Radeon via Vulkan, Intel Arc via Vulkan, Apple Silicon via Metal, and NVIDIA RTX via CUDA experiments.
Current strongest measured path: AMD RDNA4 beats llama.cpp across the five-model headline sweep. Official support: AMD Radeon, Intel Arc, and Apple Silicon. Also measured: RTX 5090 CUDA experiments.
Apple Silicon Metal
Supported
81.7 tok/s top decode
Native MSL kernels and zero-copy unified-memory loading. NVIDIA RTX CUDA
Experimental
145.8 tok/s top decode
RTX 5090/4090 CUDA path is measured and still maturing. Intel Arc Vulkan
Supported
75.3 tok/s top decode
Official Linux Vulkan target with all five catalog rows published.Current public sweep
AMD RDNA4 — Radeon AI PRO R9700
ZINC llama.cpp decode tok/s · higher is better
Qwen 3.6 35B A3B UD Q4K XL 1.54x llama.cpp
Prefill 540 tok/s ZINC · 397 tok/s llama.cpp · Overall 151%
Qwen 3.5 9B Q4K M 1.14x llama.cpp
Prefill 739 tok/s ZINC · 549 tok/s llama.cpp · Overall 115%
Qwen 3.6 27B Dense Q4K M 1.04x llama.cpp
Prefill 213 tok/s ZINC · 184 tok/s llama.cpp · Overall 105%
Gemma 4 26B-A4B MoE Q4K M 1.11x llama.cpp
Prefill 809 tok/s ZINC · 497 tok/s llama.cpp · Overall 115%
Gemma 4 31B Q4K M 1.01x llama.cpp
Prefill 249 tok/s ZINC · 200 tok/s llama.cpp · Overall 103%
Measured with the fair server-vs-server suite: one reusable ZINC server per model, one reusable llama.cpp server per model, same GGUF files, same scenario matrix, same warmup/run count, and server-side timing. We are still cooking: the next work is widening the narrow Gemma 31B margin and extending the sweep across more scenarios.
ZINC is an inference engine for consumer GPUs, not a single-vendor demo. The bet is that local LLMs should run well on the hardware people can actually put in a workstation: AMD Radeon, Apple Silicon, NVIDIA RTX, and Intel Arc. Each backend gets native GPU code instead of being treated as a generic fallback.
The AMD board is the headline today because it is the most mature and currently the fastest measured path in the public suite. Intel Arc is now an official Linux Vulkan target, and the same project keeps Apple Silicon and NVIDIA CUDA in the benchmark data so backend gaps stay visible instead of hidden.
ZINC is not finished productized infrastructure yet. It is an engine under active development, and the best current performance is still RDNA4-first and CLI-first. Intel Arc is supported, but it has less tuning history than RDNA4. The point of the docs is to make those paths explicit instead of making you reverse-engineer them from technical specs.
ZINC takes a different approach. It is written in Zig, loads GGUF directly, and uses backend-specific GPU code: Vulkan compute for AMD and Intel, Metal for Apple Silicon, and an experimental CUDA path for NVIDIA RTX.
Why another engine?
| ZINC | Typical datacenter stack | General local stack | |
|---|---|---|---|
| Design center | Consumer GPU inference | CUDA/ROCm servers | Broad portability |
| Backends | Vulkan, Metal, CUDA experiments | CUDA/ROCm first | CPU plus many GPU backends |
| AMD RDNA | Primary measured path | ROCm support boundary | Works, less RDNA-specific tuning |
| Apple Silicon | Native Metal backend | Not the target | Mature Metal path |
| NVIDIA RTX | Experimental CUDA path | Primary target | Mature CUDA path |
| KV cache compression | TurboQuant roadmap / experimental path | No | No |
| OpenAI API | Native | Native | Via server wrapper |
Other measured targets
The AMD board above is the headline path. We also keep publishing Intel Arc, Apple Silicon, and NVIDIA CUDA measurements with the same benchmark data format, so the gaps are visible instead of hidden. Full breakdown with scenarios and run provenance lives on the benchmarks dashboard.
Apple Silicon — Mac Studio (M4 Max) 40-core GPU · 64 GB unified · 2026-06-13
ZINC llama.cpp decode tok/s · higher is better
Qwen 3.6 35B A3B UD Q4K XL 1.31x llama.cpp
Prefill 97 tok/s ZINC · 299 tok/s llama.cpp
Gemma 4 26B-A4B MoE Q4K M 0.83x llama.cpp
Prefill 335 tok/s ZINC · 412 tok/s llama.cpp
Gemma 4 31B Q4K M 1.00x llama.cpp
Prefill 131 tok/s ZINC · 102 tok/s llama.cpp
Qwen 3.6 27B Dense Q4K M 0.70x llama.cpp
Prefill 16 tok/s ZINC · 105 tok/s llama.cpp
Exact machine above, native Metal backend (2026-06-13 refresh) — not a generic "Apple Silicon" average. The Metal path is younger than the RDNA4 one: decode still trails llama.cpp on most models, but ZINC prefill already leads on Qwen3-8B and Gemma 4 31B. Closing the A3B decode gap is the active Metal target.
What ZINC does
Native GPU backends, one engine. On AMD and Intel: Vulkan compute, with the mature RDNA path using wave64, cooperative matrix, and architecture-aware tiling. On Apple Silicon: native MSL kernels with simdgroup ops, zero-copy mmap, and Metal pipeline tuning. On NVIDIA RTX: experimental CUDA kernels are tracked in the same benchmark matrix. ZINC picks the right backend at build time where that backend is supported.
The API is OpenAI-compatible: POST /v1/chat/completions with SSE streaming, plus a built-in chat UI with thinking mode support. Point your existing client at ZINC and it works. API reference.
Models load from GGUF files via direct memory-map — DMA to GPU VRAM on Vulkan, zero-copy newBufferWithBytesNoCopy on Metal. Supports Q4_K, Q5_K, Q6_K, Q8_0, and F16 quantization. Architectures: Qwen 3 / 3.5 / 3.6 (dense and A3B MoE), Gemma 4, and SSM-hybrid paths.
The whole thing is written in Zig (with a thin Objective-C shim for Metal). No hidden allocations, direct GPU API calls, comptime dispatch tables, single binary.
Architecture
Supported hardware: consumer GPU targets
ZINC targets consumer and prosumer GPUs with native backends. AMD RDNA is the strongest measured path today; Intel Arc and Apple Silicon are supported; NVIDIA CUDA is an active experimental track.
AMD Radeon GPUs — RDNA4 & RDNA3 (Linux)
- RDNA4: RX 9070, RX 9070 XT, Radeon AI PRO R9700 (primary tuning target)
- RDNA3: RX 7900 XTX, RX 7900 XT, RX 7800 XT, RX 7700 XT, RX 7600
Any AMD GPU with Vulkan 1.3 and RADV or AMDVLK drivers should work.
Apple Silicon (macOS)
- M1 through M5: all variants (base, Pro, Max, Ultra)
- Current public reference box: Mac Studio (M4 Max, 40-core GPU, 64 GB unified memory)
- Native Metal shaders, zero-copy model loading, unified memory
- 16 GB for 2B models, 24+ GB for 35B models
NVIDIA RTX GPUs — CUDA (experimental)
- Current public reference: GeForce RTX 5090, with RTX 4090 also used for backend bring-up
- Native CUDA backend work is measured in the dashboard but still younger than the RDNA path
- Primary target: local GGUF inference on consumer RTX cards where Vulkan is not the practical path
Intel Arc — Xe2 / Battlemage (Linux Vulkan)
- Official support target: Arc B-series / Battlemage class GPUs
- Current public benchmark node completes all five managed catalog rows
- Arc-specific performance tuning is still younger than the RDNA4 path
See the full hardware requirements page for details.