ZINC

Benchmarks

Choose a platform and backend, then compare ZINC with llama.cpp on the same GPU, model, prompts, and reusable-server protocol.

Backend runs 5

Unique Models 6

Comparisons 25

Updated Sep 25

Choose a platform

Choose a platform and backend. Each result uses the same model file and reusable-server protocol.

AMD Radeon · AMD RDNA bench node

ROCm backend

Radeon AI PRO R9700 · 32 GB VRAM · 640 GB/s · 3 measured · 1 warmup

Exact configuration

GPU: Radeon AI PRO R9700
Chip: AMD Radeon AI PRO R9700
Memory: 32 GB VRAM · 640 GB/s
OS: Ubuntu Linux
ZINC backend: ROCM
llama.cpp backend: ROCm (ROCm0)
Prompt parity: 24/24 rows read the same prompt tokens in both engines
Build: zig build -Doptimize=ReleaseFast -Dbackend=rocm
ZINC commit: 6e6854251118
llama.cpp build: 0.4.0-dev
Artifact: Sep 25, 2026, 12:13 PM

Methodology & reproducibility

Hardware: one AMD Radeon AI PRO R9700 benchmark node with 32 GB VRAM and 640 GB/s memory bandwidth. ZINC and llama.cpp run on the same Ubuntu host and use the same GGUF file for each model.
ZINC path: sync the current source tree to the RDNA node, build ReleaseFast with the selected rocm backend, then measure generation through one reusable single-slot ZINC server per model.
ZINC RDNA backend: rocm.
Baseline backend: llama.cpp ROCm (ROCm0) on the same node and the same model file.
Baseline path: launch llama.cpp against the same model file, preferring one reusable single-slot llama-server per model across the full scenario matrix and falling back to llama-cli when the server path is unavailable.
Scenarios: Quick Chat, Coding Review, Incident Context, and Long Coding Draft. The prompts are real-world chat, code-review, support-context, and coding-plan workloads instead of synthetic factual completions.
Statistics: one warmup pass is discarded, then 3 measured runs are collected. Published prefill, decode, end-to-end throughput, and latency values are medians.
Execution order: the harness records the full ZINC scenario matrix before starting the llama.cpp baseline phase, so only one inference engine owns the GPU during measurement.
Power policy: before either engine starts, the harness selects PCIe ASPM performance mode and locks the discrete AMD GPU's memory DPM policy high to prevent idle-clock drift between runs.
Run hygiene: published RDNA runs assume a clean node with no competing ZINC, llama.cpp, or other GPU workloads.

Speculative decoding (MTP) Qwen 3.8 27B Dense Q4_K_M runs NextN self-speculation, drafting tokens the full model then verifies. llama.cpp can speculate with the same block by exporting it as a separate draft model, which these runs do not do — so the figures default to both engines without it.

Fastest 99.3 tok/s Gemma 4 26B-A4B MoE Q4K M

Decode 63.6 tok/s average

Overall 112% prefill + decode vs llama.cpp

Choose a workload

Prefill and decode are shown separately below.

Decode 68.8 tok/s 135% decode ratio

Prefill 815.3 tok/s 180% prefill ratio · 56 prompt tok

Fastest Gemma 4 26B-A4B MoE Q4K M 99.3 tok/s

Overall 130% best row: Qwen 3.8 27B Dense Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 60p · 96out Compared

Prefill 188%

ZINC 1360.5 tok/s

llama 722.3 tok/s

Decode 128%

ZINC 99.3 tok/s

llama 77.4 tok/s

131% overall

Gemma 4 31B Q4K M 60p · 96out Compared

Prefill 200%

ZINC 589.7 tok/s

llama 294.6 tok/s

Decode 104%

ZINC 28.5 tok/s

llama 27.4 tok/s

107% overall

Muse Glimmer 30B Q4K M 67p · 96out Compared

Prefill 176%

ZINC 676.2 tok/s

llama 383.9 tok/s

Decode 109%

ZINC 33.3 tok/s

llama 30.7 tok/s

111% overall

Qwen 3.5 9B Q4K M 58p · 96out Compared

Prefill 184%

ZINC 1226.5 tok/s

llama 668.3 tok/s

Decode 119%

ZINC 90.9 tok/s

llama 76.4 tok/s

122% overall

Qwen 3.6 35B A3B UD Q4K XL 35p · 96out Compared

Prefill 107%

ZINC 514.5 tok/s

llama 481.8 tok/s

Decode 128%

ZINC 96.8 tok/s

llama 75.3 tok/s

127% overall

Qwen 3.8 27B Dense Q4K M 58p · 96out Compared

Prefill 225%

ZINC 524.4 tok/s

llama 232.8 tok/s

Decode 113%

ZINC 32.8 tok/s

llama 28.9 tok/s

116% overall

Details

Gemma 4 26B-A4B MoE Q4K M 60 tok prompt · 96 tok output

Compared

Prefill 188%

ZINC 1360.5 tok/s

llama 722.3 tok/s

Decode 128%

ZINC 99.3 tok/s

llama 77.4 tok/s

Overall 131%

ZINC 154.3 tok/s

llama 117.9 tok/s

Prompt+decode. Cold latency: ZINC 1565 ms · llama.cpp 1759 ms

Gemma 4 31B Q4K M 60 tok prompt · 96 tok output

Compared

Prefill 200%

ZINC 589.7 tok/s

llama 294.6 tok/s

Decode 104%

ZINC 28.5 tok/s

llama 27.4 tok/s

Overall 107%

ZINC 45.0 tok/s

llama 42.1 tok/s

Prompt+decode. Cold latency: ZINC 3801 ms · llama.cpp 4146 ms

Muse Glimmer 30B Q4K M 67 tok prompt · 96 tok output

Compared

Prefill 176%

ZINC 676.2 tok/s

llama 383.9 tok/s

Decode 109%

ZINC 33.3 tok/s

llama 30.7 tok/s

Overall 111%

ZINC 54.7 tok/s

llama 49.4 tok/s

Prompt+decode. Cold latency: ZINC 3397 ms · llama.cpp 3676 ms

Qwen 3.5 9B Q4K M 58 tok prompt · 96 tok output

Compared

Prefill 184%

ZINC 1226.5 tok/s

llama 668.3 tok/s

Decode 119%

ZINC 90.9 tok/s

llama 76.4 tok/s

Overall 122%

ZINC 139.5 tok/s

llama 114.6 tok/s

Prompt+decode. Cold latency: ZINC 1376 ms · llama.cpp 1709 ms

Qwen 3.6 35B A3B UD Q4K XL 35 tok prompt · 96 tok output

Compared

Prefill 107%

ZINC 514.5 tok/s

llama 481.8 tok/s

Decode 128%

ZINC 96.8 tok/s

llama 75.3 tok/s

Overall 127%

ZINC 123.6 tok/s

llama 97.3 tok/s

Prompt+decode. Cold latency: ZINC 1577 ms · llama.cpp 1827 ms

Qwen 3.8 27B Dense Q4K M 58 tok prompt · 96 tok output

Compared

Prefill 225%

ZINC 524.4 tok/s

llama 232.8 tok/s

Decode 113%

ZINC 32.8 tok/s

llama 28.9 tok/s

Overall 116%

ZINC 50.1 tok/s

llama 43.1 tok/s

Prompt+decode. Cold latency: ZINC 1872 ms · llama.cpp 3742 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 60 tok prompt · 96 tok output · 3 runs · Compared	Z 1360.5 tok/s 44.1 ms L 722.3 tok/s 83.1 ms 188% +638.3 tok/s	Z 99.3 tok/s 10.1 ms/tok L 77.4 tok/s 12.9 ms/tok 128% +21.9 tok/s	Z 154.3 tok/s prompt+decode L 117.9 tok/s prompt+decode 131% +36.4 tok/s 194 ms faster
Gemma 4 31B Q4K M 60 tok prompt · 96 tok output · 3 runs · Compared	Z 589.7 tok/s 101.7 ms L 294.6 tok/s 203.7 ms 200% +295.1 tok/s	Z 28.5 tok/s 35.1 ms/tok L 27.4 tok/s 36.5 ms/tok 104% +1.1 tok/s	Z 45.0 tok/s prompt+decode L 42.1 tok/s prompt+decode 107% +2.9 tok/s 345 ms faster
Muse Glimmer 30B Q4K M 67 tok prompt · 96 tok output · 3 runs · Compared	Z 676.2 tok/s 99.1 ms L 383.9 tok/s 174.5 ms 176% +292.3 tok/s	Z 33.3 tok/s 30.0 ms/tok L 30.7 tok/s 32.6 ms/tok 109% +2.6 tok/s	Z 54.7 tok/s prompt+decode L 49.4 tok/s prompt+decode 111% +5.4 tok/s 278 ms faster
Qwen 3.5 9B Q4K M 58 tok prompt · 96 tok output · 3 runs · Compared	Z 1226.5 tok/s 47.3 ms L 668.3 tok/s 86.8 ms 184% +558.1 tok/s	Z 90.9 tok/s 11.0 ms/tok L 76.4 tok/s 13.1 ms/tok 119% +14.5 tok/s	Z 139.5 tok/s prompt+decode L 114.6 tok/s prompt+decode 122% +24.9 tok/s 333 ms faster
Qwen 3.6 35B A3B UD Q4K XL 35 tok prompt · 96 tok output · 3 runs · Compared	Z 514.5 tok/s 68.0 ms L 481.8 tok/s 72.7 ms 107% +32.7 tok/s	Z 96.8 tok/s 10.3 ms/tok L 75.3 tok/s 13.3 ms/tok 128% +21.5 tok/s	Z 123.6 tok/s prompt+decode L 97.3 tok/s prompt+decode 127% +26.3 tok/s 250 ms faster
Qwen 3.8 27B Dense Q4K M 58 tok prompt · 96 tok output · 3 runs · Compared	Z 524.4 tok/s 110.6 ms L 232.8 tok/s 249.2 ms 225% +291.6 tok/s	Z 32.8 tok/s 15.7 ms/tok L 28.9 tok/s 34.6 ms/tok 113% +34.8 tok/s	Z 50.1 tok/s prompt+decode L 43.1 tok/s prompt+decode 116% +52.1 tok/s 1871 ms faster

Decode 69.6 tok/s 139% decode ratio

Prefill 1492.1 tok/s 138% prefill ratio · 184 prompt tok

Fastest Gemma 4 26B-A4B MoE Q4K M 98.0 tok/s

Overall 129% best row: Qwen 3.8 27B Dense Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 203p · 160out Compared

Prefill 178%

ZINC 3119.7 tok/s

llama 1755.7 tok/s

Decode 125%

ZINC 98.0 tok/s

llama 78.7 tok/s

127% overall

Gemma 4 31B Q4K M 203p · 160out Compared

Prefill 150%

ZINC 831.0 tok/s

llama 555.0 tok/s

Decode 102%

ZINC 27.9 tok/s

llama 27.2 tok/s

104% overall

Muse Glimmer 30B Q4K M 179p · 160out Compared

Prefill 127%

ZINC 811.5 tok/s

llama 638.3 tok/s

Decode 108%

ZINC 33.0 tok/s

llama 30.7 tok/s

108% overall

Qwen 3.5 9B Q4K M 183p · 160out Compared

Prefill 137%

ZINC 1992.0 tok/s

llama 1453.4 tok/s

Decode 117%

ZINC 89.9 tok/s

llama 76.6 tok/s

118% overall

Qwen 3.6 35B A3B UD Q4K XL 153p · 160out Compared

Prefill 104%

ZINC 1565.4 tok/s

llama 1498.9 tok/s

Decode 127%

ZINC 96.3 tok/s

llama 75.8 tok/s

126% overall

Qwen 3.8 27B Dense Q4K M 183p · 160out Compared

Prefill 134%

ZINC 632.9 tok/s

llama 474.0 tok/s

Decode 112%

ZINC 32.4 tok/s

llama 28.8 tok/s

113% overall

Details

Gemma 4 26B-A4B MoE Q4K M 203 tok prompt · 160 tok output

Compared

Prefill 178%

ZINC 3119.7 tok/s

llama 1755.7 tok/s

Decode 125%

ZINC 98.0 tok/s

llama 78.7 tok/s

Overall 127%

ZINC 213.8 tok/s

llama 168.9 tok/s

Prompt+decode. Cold latency: ZINC 2278 ms · llama.cpp 2533 ms

Gemma 4 31B Q4K M 203 tok prompt · 160 tok output

Compared

Prefill 150%

ZINC 831.0 tok/s

llama 555.0 tok/s

Decode 102%

ZINC 27.9 tok/s

llama 27.2 tok/s

Overall 104%

ZINC 60.6 tok/s

llama 58.1 tok/s

Prompt+decode. Cold latency: ZINC 6439 ms · llama.cpp 6640 ms

Muse Glimmer 30B Q4K M 179 tok prompt · 160 tok output

Compared

Prefill 127%

ZINC 811.5 tok/s

llama 638.3 tok/s

Decode 108%

ZINC 33.0 tok/s

llama 30.7 tok/s

Overall 108%

ZINC 66.9 tok/s

llama 61.7 tok/s

Prompt+decode. Cold latency: ZINC 5410 ms · llama.cpp 6143 ms

Qwen 3.5 9B Q4K M 183 tok prompt · 160 tok output

Compared

Prefill 137%

ZINC 1992.0 tok/s

llama 1453.4 tok/s

Decode 117%

ZINC 89.9 tok/s

llama 76.6 tok/s

Overall 118%

ZINC 183.3 tok/s

llama 154.9 tok/s

Prompt+decode. Cold latency: ZINC 2123 ms · llama.cpp 2469 ms

Qwen 3.6 35B A3B UD Q4K XL 153 tok prompt · 160 tok output

Compared

Prefill 104%

ZINC 1565.4 tok/s

llama 1498.9 tok/s

Decode 127%

ZINC 96.3 tok/s

llama 75.8 tok/s

Overall 126%

ZINC 177.9 tok/s

llama 141.4 tok/s

Prompt+decode. Cold latency: ZINC 2199 ms · llama.cpp 2602 ms

Qwen 3.8 27B Dense Q4K M 183 tok prompt · 160 tok output

Compared

Prefill 134%

ZINC 632.9 tok/s

llama 474.0 tok/s

Decode 112%

ZINC 32.4 tok/s

llama 28.8 tok/s

Overall 113%

ZINC 65.4 tok/s

llama 57.8 tok/s

Prompt+decode. Cold latency: ZINC 2759 ms · llama.cpp 6109 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 203 tok prompt · 160 tok output · 3 runs · Compared	Z 3119.7 tok/s 65.1 ms L 1755.7 tok/s 115.6 ms 178% +1364.0 tok/s	Z 98.0 tok/s 10.2 ms/tok L 78.7 tok/s 12.7 ms/tok 125% +19.3 tok/s	Z 213.8 tok/s prompt+decode L 168.9 tok/s prompt+decode 127% +44.9 tok/s 255 ms faster
Gemma 4 31B Q4K M 203 tok prompt · 160 tok output · 3 runs · Compared	Z 831.0 tok/s 244.3 ms L 555.0 tok/s 365.8 ms 150% +276.1 tok/s	Z 27.9 tok/s 35.9 ms/tok L 27.2 tok/s 36.8 ms/tok 102% +0.7 tok/s	Z 60.6 tok/s prompt+decode L 58.1 tok/s prompt+decode 104% +2.5 tok/s 200 ms faster
Muse Glimmer 30B Q4K M 179 tok prompt · 160 tok output · 3 runs · Compared	Z 811.5 tok/s 220.6 ms L 638.3 tok/s 280.4 ms 127% +173.2 tok/s	Z 33.0 tok/s 30.3 ms/tok L 30.7 tok/s 32.6 ms/tok 108% +2.3 tok/s	Z 66.9 tok/s prompt+decode L 61.7 tok/s prompt+decode 108% +5.2 tok/s 734 ms faster
Qwen 3.5 9B Q4K M 183 tok prompt · 160 tok output · 3 runs · Compared	Z 1992.0 tok/s 91.9 ms L 1453.4 tok/s 125.9 ms 137% +538.6 tok/s	Z 89.9 tok/s 11.1 ms/tok L 76.6 tok/s 13.1 ms/tok 117% +13.3 tok/s	Z 183.3 tok/s prompt+decode L 154.9 tok/s prompt+decode 118% +28.4 tok/s 346 ms faster
Qwen 3.6 35B A3B UD Q4K XL 153 tok prompt · 160 tok output · 3 runs · Compared	Z 1565.4 tok/s 97.7 ms L 1498.9 tok/s 102.1 ms 104% +66.4 tok/s	Z 96.3 tok/s 10.4 ms/tok L 75.8 tok/s 13.2 ms/tok 127% +20.5 tok/s	Z 177.9 tok/s prompt+decode L 141.4 tok/s prompt+decode 126% +36.5 tok/s 403 ms faster
Qwen 3.8 27B Dense Q4K M 183 tok prompt · 160 tok output · 3 runs · Compared	Z 632.9 tok/s 289.2 ms L 474.0 tok/s 386.1 ms 134% +158.9 tok/s	Z 32.4 tok/s 13.7 ms/tok L 28.8 tok/s 34.7 ms/tok 112% +43.9 tok/s	Z 65.4 tok/s prompt+decode L 57.8 tok/s prompt+decode 113% +80.0 tok/s 3349 ms faster

Decode 68.6 tok/s 136% decode ratio

Prefill 1856.2 tok/s 122% prefill ratio · 341 prompt tok

Fastest Gemma 4 26B-A4B MoE Q4K M 96.7 tok/s

Overall 125% best row: Qwen 3.8 27B Dense Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 353p · 128out Compared

Prefill 164%

ZINC 3901.9 tok/s

llama 2379.5 tok/s

Decode 122%

ZINC 96.7 tok/s

llama 79.0 tok/s

125% overall

Gemma 4 31B Q4K M 353p · 128out Compared

Prefill 119%

ZINC 789.0 tok/s

llama 660.8 tok/s

Decode 101%

ZINC 27.4 tok/s

llama 27.2 tok/s

103% overall

Muse Glimmer 30B Q4K M 326p · 128out Compared

Prefill 119%

ZINC 963.4 tok/s

llama 809.7 tok/s

Decode 107%

ZINC 33.0 tok/s

llama 30.7 tok/s

108% overall

Qwen 3.5 9B Q4K M 349p · 128out Compared

Prefill 115%

ZINC 2445.3 tok/s

llama 2124.6 tok/s

Decode 117%

ZINC 89.4 tok/s

llama 76.7 tok/s

116% overall

Qwen 3.6 35B A3B UD Q4K XL 315p · 128out Compared

Prefill 105%

ZINC 2294.8 tok/s

llama 2187.1 tok/s

Decode 125%

ZINC 95.7 tok/s

llama 76.3 tok/s

124% overall

Qwen 3.8 27B Dense Q4K M 349p · 128out Compared

Prefill 109%

ZINC 743.0 tok/s

llama 680.8 tok/s

Decode 112%

ZINC 32.4 tok/s

llama 28.9 tok/s

112% overall

Details

Gemma 4 26B-A4B MoE Q4K M 353 tok prompt · 128 tok output

Compared

Prefill 164%

ZINC 3901.9 tok/s

llama 2379.5 tok/s

Decode 122%

ZINC 96.7 tok/s

llama 79.0 tok/s

Overall 125%

ZINC 340.1 tok/s

llama 271.9 tok/s

Prompt+decode. Cold latency: ZINC 1872 ms · llama.cpp 2070 ms

Gemma 4 31B Q4K M 353 tok prompt · 128 tok output

Compared

Prefill 119%

ZINC 789.0 tok/s

llama 660.8 tok/s

Decode 101%

ZINC 27.4 tok/s

llama 27.2 tok/s

Overall 103%

ZINC 94.1 tok/s

llama 91.7 tok/s

Prompt+decode. Cold latency: ZINC 5567 ms · llama.cpp 5582 ms

Muse Glimmer 30B Q4K M 326 tok prompt · 128 tok output

Compared

Prefill 119%

ZINC 963.4 tok/s

llama 809.7 tok/s

Decode 107%

ZINC 33.0 tok/s

llama 30.7 tok/s

Overall 108%

ZINC 107.5 tok/s

llama 99.3 tok/s

Prompt+decode. Cold latency: ZINC 4730 ms · llama.cpp 5058 ms

Qwen 3.5 9B Q4K M 349 tok prompt · 128 tok output

Compared

Prefill 115%

ZINC 2445.3 tok/s

llama 2124.6 tok/s

Decode 117%

ZINC 89.4 tok/s

llama 76.7 tok/s

Overall 116%

ZINC 302.9 tok/s

llama 260.3 tok/s

Prompt+decode. Cold latency: ZINC 1798 ms · llama.cpp 2157 ms

Qwen 3.6 35B A3B UD Q4K XL 315 tok prompt · 128 tok output

Compared

Prefill 105%

ZINC 2294.8 tok/s

llama 2187.1 tok/s

Decode 125%

ZINC 95.7 tok/s

llama 76.3 tok/s

Overall 124%

ZINC 300.4 tok/s

llama 243.2 tok/s

Prompt+decode. Cold latency: ZINC 1912 ms · llama.cpp 2164 ms

Qwen 3.8 27B Dense Q4K M 349 tok prompt · 128 tok output

Compared

Prefill 109%

ZINC 743.0 tok/s

llama 680.8 tok/s

Decode 112%

ZINC 32.4 tok/s

llama 28.9 tok/s

Overall 112%

ZINC 108.2 tok/s

llama 96.6 tok/s

Prompt+decode. Cold latency: ZINC 2582 ms · llama.cpp 5099 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 353 tok prompt · 128 tok output · 3 runs · Compared	Z 3901.9 tok/s 90.5 ms L 2379.5 tok/s 148.3 ms 164% +1522.3 tok/s	Z 96.7 tok/s 10.3 ms/tok L 79.0 tok/s 12.7 ms/tok 122% +17.7 tok/s	Z 340.1 tok/s prompt+decode L 271.9 tok/s prompt+decode 125% +68.2 tok/s 198 ms faster
Gemma 4 31B Q4K M 353 tok prompt · 128 tok output · 3 runs · Compared	Z 789.0 tok/s 447.4 ms L 660.8 tok/s 534.2 ms 119% +128.2 tok/s	Z 27.4 tok/s 36.4 ms/tok L 27.2 tok/s 36.8 ms/tok 101% +0.3 tok/s	Z 94.1 tok/s prompt+decode L 91.7 tok/s prompt+decode 103% +2.4 tok/s 15.0 ms faster
Muse Glimmer 30B Q4K M 326 tok prompt · 128 tok output · 3 runs · Compared	Z 963.4 tok/s 338.4 ms L 809.7 tok/s 402.6 ms 119% +153.7 tok/s	Z 33.0 tok/s 30.3 ms/tok L 30.7 tok/s 32.6 ms/tok 107% +2.3 tok/s	Z 107.5 tok/s prompt+decode L 99.3 tok/s prompt+decode 108% +8.2 tok/s 328 ms faster
Qwen 3.5 9B Q4K M 349 tok prompt · 128 tok output · 3 runs · Compared	Z 2445.3 tok/s 142.7 ms L 2124.6 tok/s 164.3 ms 115% +320.7 tok/s	Z 89.4 tok/s 11.2 ms/tok L 76.7 tok/s 13.0 ms/tok 117% +12.7 tok/s	Z 302.9 tok/s prompt+decode L 260.3 tok/s prompt+decode 116% +42.6 tok/s 359 ms faster
Qwen 3.6 35B A3B UD Q4K XL 315 tok prompt · 128 tok output · 3 runs · Compared	Z 2294.8 tok/s 137.3 ms L 2187.1 tok/s 144.0 ms 105% +107.7 tok/s	Z 95.7 tok/s 10.4 ms/tok L 76.3 tok/s 13.1 ms/tok 125% +19.4 tok/s	Z 300.4 tok/s prompt+decode L 243.2 tok/s prompt+decode 124% +57.2 tok/s 252 ms faster
Qwen 3.8 27B Dense Q4K M 349 tok prompt · 128 tok output · 3 runs · Compared	Z 743.0 tok/s 469.7 ms L 680.8 tok/s 512.6 ms 109% +62.1 tok/s	Z 32.4 tok/s 14.3 ms/tok L 28.9 tok/s 34.6 ms/tok 112% +40.8 tok/s	Z 108.2 tok/s prompt+decode L 96.6 tok/s prompt+decode 112% +110.3 tok/s 2517 ms faster

Decode 69.4 tok/s 137% decode ratio

Prefill 958.1 tok/s 163% prefill ratio · 79 prompt tok

Fastest Gemma 4 26B-A4B MoE Q4K M 98.6 tok/s

Overall 128% best row: Qwen 3.8 27B Dense Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 81p · 256out Compared

Prefill 176%

ZINC 1643.2 tok/s

llama 931.5 tok/s

Decode 124%

ZINC 98.6 tok/s

llama 79.3 tok/s

125% overall

Gemma 4 31B Q4K M 81p · 256out Compared

Prefill 172%

ZINC 561.4 tok/s

llama 327.2 tok/s

Decode 102%

ZINC 28.0 tok/s

llama 27.3 tok/s

103% overall

Muse Glimmer 30B Q4K M 89p · 256out Compared

Prefill 153%

ZINC 713.4 tok/s

llama 465.1 tok/s

Decode 107%

ZINC 32.9 tok/s

llama 30.7 tok/s

108% overall

Qwen 3.5 9B Q4K M 80p · 256out Compared

Prefill 168%

ZINC 1411.0 tok/s

llama 837.5 tok/s

Decode 116%

ZINC 89.7 tok/s

llama 77.1 tok/s

117% overall

Qwen 3.6 35B A3B UD Q4K XL 63p · 256out Compared

Prefill 108%

ZINC 868.7 tok/s

llama 802.3 tok/s

Decode 126%

ZINC 96.3 tok/s

llama 76.2 tok/s

126% overall

Qwen 3.8 27B Dense Q4K M 80p · 256out Compared

Prefill 201%

ZINC 551.0 tok/s

llama 274.7 tok/s

Decode 112%

ZINC 32.5 tok/s

llama 28.9 tok/s

113% overall

Details

Gemma 4 26B-A4B MoE Q4K M 81 tok prompt · 256 tok output

Compared

Prefill 176%

ZINC 1643.2 tok/s

llama 931.5 tok/s

Decode 124%

ZINC 98.6 tok/s

llama 79.3 tok/s

Overall 125%

ZINC 127.4 tok/s

llama 101.7 tok/s

Prompt+decode. Cold latency: ZINC 3262 ms · llama.cpp 3745 ms

Gemma 4 31B Q4K M 81 tok prompt · 256 tok output

Compared

Prefill 172%

ZINC 561.4 tok/s

llama 327.2 tok/s

Decode 102%

ZINC 28.0 tok/s

llama 27.3 tok/s

Overall 103%

ZINC 36.3 tok/s

llama 35.1 tok/s

Prompt+decode. Cold latency: ZINC 9610 ms · llama.cpp 10005 ms

Muse Glimmer 30B Q4K M 89 tok prompt · 256 tok output

Compared

Prefill 153%

ZINC 713.4 tok/s

llama 465.1 tok/s

Decode 107%

ZINC 32.9 tok/s

llama 30.7 tok/s

Overall 108%

ZINC 43.7 tok/s

llama 40.5 tok/s

Prompt+decode. Cold latency: ZINC 8330 ms · llama.cpp 9001 ms

Qwen 3.5 9B Q4K M 80 tok prompt · 256 tok output

Compared

Prefill 168%

ZINC 1411.0 tok/s

llama 837.5 tok/s

Decode 116%

ZINC 89.7 tok/s

llama 77.1 tok/s

Overall 117%

ZINC 115.5 tok/s

llama 98.3 tok/s

Prompt+decode. Cold latency: ZINC 3291 ms · llama.cpp 3857 ms

Qwen 3.6 35B A3B UD Q4K XL 63 tok prompt · 256 tok output

Compared

Prefill 108%

ZINC 868.7 tok/s

llama 802.3 tok/s

Decode 126%

ZINC 96.3 tok/s

llama 76.2 tok/s

Overall 126%

ZINC 116.8 tok/s

llama 92.8 tok/s

Prompt+decode. Cold latency: ZINC 3107 ms · llama.cpp 3793 ms

Qwen 3.8 27B Dense Q4K M 80 tok prompt · 256 tok output

Compared

Prefill 201%

ZINC 551.0 tok/s

llama 274.7 tok/s

Decode 112%

ZINC 32.5 tok/s

llama 28.9 tok/s

Overall 113%

ZINC 41.6 tok/s

llama 36.7 tok/s

Prompt+decode. Cold latency: ZINC 4034 ms · llama.cpp 9291 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 81 tok prompt · 256 tok output · 3 runs · Compared	Z 1643.2 tok/s 49.3 ms L 931.5 tok/s 87.0 ms 176% +711.7 tok/s	Z 98.6 tok/s 10.1 ms/tok L 79.3 tok/s 12.6 ms/tok 124% +19.3 tok/s	Z 127.4 tok/s prompt+decode L 101.7 tok/s prompt+decode 125% +25.7 tok/s 484 ms faster
Gemma 4 31B Q4K M 81 tok prompt · 256 tok output · 3 runs · Compared	Z 561.4 tok/s 144.3 ms L 327.2 tok/s 247.5 ms 172% +234.2 tok/s	Z 28.0 tok/s 35.7 ms/tok L 27.3 tok/s 36.6 ms/tok 102% +0.6 tok/s	Z 36.3 tok/s prompt+decode L 35.1 tok/s prompt+decode 103% +1.2 tok/s 395 ms faster
Muse Glimmer 30B Q4K M 89 tok prompt · 256 tok output · 3 runs · Compared	Z 713.4 tok/s 124.8 ms L 465.1 tok/s 191.4 ms 153% +248.3 tok/s	Z 32.9 tok/s 30.4 ms/tok L 30.7 tok/s 32.6 ms/tok 107% +2.2 tok/s	Z 43.7 tok/s prompt+decode L 40.5 tok/s prompt+decode 108% +3.2 tok/s 672 ms faster
Qwen 3.5 9B Q4K M 80 tok prompt · 256 tok output · 3 runs · Compared	Z 1411.0 tok/s 56.7 ms L 837.5 tok/s 95.5 ms 168% +573.5 tok/s	Z 89.7 tok/s 11.1 ms/tok L 77.1 tok/s 13.0 ms/tok 116% +12.7 tok/s	Z 115.5 tok/s prompt+decode L 98.3 tok/s prompt+decode 117% +17.2 tok/s 566 ms faster
Qwen 3.6 35B A3B UD Q4K XL 63 tok prompt · 256 tok output · 3 runs · Compared	Z 868.7 tok/s 72.5 ms L 802.3 tok/s 78.5 ms 108% +66.4 tok/s	Z 96.3 tok/s 10.4 ms/tok L 76.2 tok/s 13.1 ms/tok 126% +20.0 tok/s	Z 116.8 tok/s prompt+decode L 92.8 tok/s prompt+decode 126% +23.9 tok/s 686 ms faster
Qwen 3.8 27B Dense Q4K M 80 tok prompt · 256 tok output · 3 runs · Compared	Z 551.0 tok/s 145.2 ms L 274.7 tok/s 291.3 ms 201% +276.3 tok/s	Z 32.5 tok/s 14.1 ms/tok L 28.9 tok/s 34.6 ms/tok 112% +41.9 tok/s	Z 41.6 tok/s prompt+decode L 36.7 tok/s prompt+decode 113% +52.6 tok/s 5257 ms faster

Methodology ZINC server vs llama.cpp server on the same RDNA node · Four-scenario matrix with same-file baselines

3 measured · 1 warmup

Benchmark runner Exact prompts and token caps Prompt matrix test Published JSON

Hardware: one AMD Radeon AI PRO R9700 benchmark node with 32 GB VRAM and 640 GB/s memory bandwidth. ZINC and llama.cpp run on the same Ubuntu host and use the same GGUF file for each model.
ZINC path: sync the current source tree to the RDNA node, build ReleaseFast with the selected rocm backend, then measure generation through one reusable single-slot ZINC server per model.
ZINC RDNA backend: rocm.
Baseline backend: llama.cpp ROCm (ROCm0) on the same node and the same model file.
Baseline path: launch llama.cpp against the same model file, preferring one reusable single-slot llama-server per model across the full scenario matrix and falling back to llama-cli when the server path is unavailable.
Scenarios: Quick Chat, Coding Review, Incident Context, and Long Coding Draft. The prompts are real-world chat, code-review, support-context, and coding-plan workloads instead of synthetic factual completions.
Statistics: one warmup pass is discarded, then 3 measured runs are collected. Published prefill, decode, end-to-end throughput, and latency values are medians.
Execution order: the harness records the full ZINC scenario matrix before starting the llama.cpp baseline phase, so only one inference engine owns the GPU during measurement.
Power policy: before either engine starts, the harness selects PCIe ASPM performance mode and locks the discrete AMD GPU's memory DPM policy high to prevent idle-clock drift between runs.
Run hygiene: published RDNA runs assume a clean node with no competing ZINC, llama.cpp, or other GPU workloads.

AMD Radeon · AMD RDNA bench node

Vulkan backend

Radeon AI PRO R9700 · 32 GB VRAM · 640 GB/s · 3 measured · 1 warmup

Exact configuration

GPU: Radeon AI PRO R9700
Chip: AMD Radeon AI PRO R9700
Memory: 32 GB VRAM · 640 GB/s
OS: Ubuntu Linux
ZINC backend: Vulkan (RADV, coop_matrix)
llama.cpp backend: Vulkan (Vulkan0)
Prompt parity: 24/24 rows read the same prompt tokens in both engines
Build: zig build -Doptimize=ReleaseFast -Dbackend=vulkan
ZINC commit: 44d35047c780
llama.cpp build: 0.4.0-dev
Artifact: Sep 24, 2026, 1:49 AM

Methodology & reproducibility

Hardware: one AMD Radeon AI PRO R9700 benchmark node with 32 GB VRAM and 640 GB/s memory bandwidth. ZINC and llama.cpp run on the same Ubuntu host and use the same GGUF file for each model.
ZINC path: sync the current source tree to the RDNA node, build ReleaseFast with the selected vulkan backend, then measure generation through one reusable single-slot ZINC server per model.
ZINC RDNA backend: vulkan.
Baseline backend: llama.cpp Vulkan (Vulkan0) on the same node and the same model file.
Baseline path: launch llama.cpp against the same model file, preferring one reusable single-slot llama-server per model across the full scenario matrix and falling back to llama-cli when the server path is unavailable.
Scenarios: Quick Chat, Coding Review, Incident Context, and Long Coding Draft. The prompts are real-world chat, code-review, support-context, and coding-plan workloads instead of synthetic factual completions.
Statistics: one warmup pass is discarded, then 3 measured runs are collected. Published prefill, decode, end-to-end throughput, and latency values are medians.
Execution order: the harness records the full ZINC scenario matrix before starting the llama.cpp baseline phase, so only one inference engine owns the GPU during measurement.
Power policy: before either engine starts, the harness selects PCIe ASPM performance mode and locks the discrete AMD GPU's memory DPM policy high to prevent idle-clock drift between runs.
Run hygiene: published RDNA runs assume a clean node with no competing ZINC, llama.cpp, or other GPU workloads.

Fastest 153.4 tok/s Qwen 3.6 35B A3B UD Q4K XL

Decode 82.7 tok/s average

Overall 122% prefill + decode vs llama.cpp

Choose a workload

Prefill and decode are shown separately below.

Decode 86.9 tok/s 150% decode ratio

Prefill 508.5 tok/s 241% prefill ratio · 56 prompt tok

Fastest Qwen 3.6 35B A3B UD Q4K XL 153.4 tok/s

Overall 142% best row: Qwen 3.8 27B Dense Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 60p · 96out Compared

Prefill 398%

ZINC 1102.5 tok/s

llama 276.9 tok/s

Decode 156%

ZINC 139.6 tok/s

llama 89.4 tok/s

174% overall

Gemma 4 31B Q4K M 60p · 96out Compared

Prefill 227%

ZINC 330.3 tok/s

llama 145.3 tok/s

Decode 108%

ZINC 31.1 tok/s

llama 28.7 tok/s

115% overall

Muse Glimmer 30B Q4K M 67p · 96out Compared

Prefill 148%

ZINC 275.4 tok/s

llama 186.2 tok/s

Decode 110%

ZINC 36.0 tok/s

llama 32.8 tok/s

113% overall

Qwen 3.5 9B Q4K M 58p · 96out Compared

Prefill 257%

ZINC 728.8 tok/s

llama 283.5 tok/s

Decode 131%

ZINC 101.9 tok/s

llama 77.7 tok/s

141% overall

Qwen 3.6 35B A3B UD Q4K XL 35p · 96out Compared

Prefill 146%

ZINC 335.1 tok/s

llama 229.1 tok/s

Decode 196%

ZINC 153.4 tok/s

llama 78.2 tok/s

189% overall

Qwen 3.8 27B Dense Q4K M 58p · 96out Compared

Prefill 272%

ZINC 278.8 tok/s

llama 102.6 tok/s

Decode 114%

ZINC 34.3 tok/s

llama 30.0 tok/s

125% overall

Details

Gemma 4 26B-A4B MoE Q4K M 60 tok prompt · 96 tok output

Compared

Prefill 398%

ZINC 1102.5 tok/s

llama 276.9 tok/s

Decode 156%

ZINC 139.6 tok/s

llama 89.4 tok/s

Overall 174%

ZINC 210.3 tok/s

llama 120.9 tok/s

Prompt+decode. Cold latency: ZINC 1287 ms · llama.cpp 1724 ms

Gemma 4 31B Q4K M 60 tok prompt · 96 tok output

Compared

Prefill 227%

ZINC 330.3 tok/s

llama 145.3 tok/s

Decode 108%

ZINC 31.1 tok/s

llama 28.7 tok/s

Overall 115%

ZINC 47.7 tok/s

llama 41.6 tok/s

Prompt+decode. Cold latency: ZINC 3701 ms · llama.cpp 4264 ms

Muse Glimmer 30B Q4K M 67 tok prompt · 96 tok output

Compared

Prefill 148%

ZINC 275.4 tok/s

llama 186.2 tok/s

Decode 110%

ZINC 36.0 tok/s

llama 32.8 tok/s

Overall 113%

ZINC 56.1 tok/s

llama 49.6 tok/s

Prompt+decode. Cold latency: ZINC 3285 ms · llama.cpp 3873 ms

Qwen 3.5 9B Q4K M 58 tok prompt · 96 tok output

Compared

Prefill 257%

ZINC 728.8 tok/s

llama 283.5 tok/s

Decode 131%

ZINC 101.9 tok/s

llama 77.7 tok/s

Overall 141%

ZINC 150.8 tok/s

llama 106.9 tok/s

Prompt+decode. Cold latency: ZINC 1461 ms · llama.cpp 2008 ms

Qwen 3.6 35B A3B UD Q4K XL 35 tok prompt · 96 tok output

Compared

Prefill 146%

ZINC 335.1 tok/s

llama 229.1 tok/s

Decode 196%

ZINC 153.4 tok/s

llama 78.2 tok/s

Overall 189%

ZINC 179.3 tok/s

llama 94.9 tok/s

Prompt+decode. Cold latency: ZINC 1062 ms · llama.cpp 1748 ms

Qwen 3.8 27B Dense Q4K M 58 tok prompt · 96 tok output

Compared

Prefill 272%

ZINC 278.8 tok/s

llama 102.6 tok/s

Decode 114%

ZINC 34.3 tok/s

llama 30.0 tok/s

Overall 125%

ZINC 51.1 tok/s

llama 40.9 tok/s

Prompt+decode. Cold latency: ZINC 2269 ms · llama.cpp 4389 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 60 tok prompt · 96 tok output · 3 runs · Compared	Z 1102.5 tok/s 54.4 ms L 276.9 tok/s 216.7 ms 398% +825.6 tok/s	Z 139.6 tok/s 7.2 ms/tok L 89.4 tok/s 11.2 ms/tok 156% +50.3 tok/s	Z 210.3 tok/s prompt+decode L 120.9 tok/s prompt+decode 174% +89.4 tok/s 437 ms faster
Gemma 4 31B Q4K M 60 tok prompt · 96 tok output · 3 runs · Compared	Z 330.3 tok/s 181.6 ms L 145.3 tok/s 412.9 ms 227% +185.0 tok/s	Z 31.1 tok/s 32.2 ms/tok L 28.7 tok/s 34.8 ms/tok 108% +2.3 tok/s	Z 47.7 tok/s prompt+decode L 41.6 tok/s prompt+decode 115% +6.1 tok/s 563 ms faster
Muse Glimmer 30B Q4K M 67 tok prompt · 96 tok output · 3 runs · Compared	Z 275.4 tok/s 243.3 ms L 186.2 tok/s 359.9 ms 148% +89.2 tok/s	Z 36.0 tok/s 27.7 ms/tok L 32.8 tok/s 30.5 ms/tok 110% +3.3 tok/s	Z 56.1 tok/s prompt+decode L 49.6 tok/s prompt+decode 113% +6.5 tok/s 588 ms faster
Qwen 3.5 9B Q4K M 58 tok prompt · 96 tok output · 3 runs · Compared	Z 728.8 tok/s 79.6 ms L 283.5 tok/s 204.6 ms 257% +445.4 tok/s	Z 101.9 tok/s 9.8 ms/tok L 77.7 tok/s 12.9 ms/tok 131% +24.3 tok/s	Z 150.8 tok/s prompt+decode L 106.9 tok/s prompt+decode 141% +43.9 tok/s 547 ms faster
Qwen 3.6 35B A3B UD Q4K XL 35 tok prompt · 96 tok output · 3 runs · Compared	Z 335.1 tok/s 104.5 ms L 229.1 tok/s 152.8 ms 146% +106.0 tok/s	Z 153.4 tok/s 6.5 ms/tok L 78.2 tok/s 12.8 ms/tok 196% +75.2 tok/s	Z 179.3 tok/s prompt+decode L 94.9 tok/s prompt+decode 189% +84.5 tok/s 686 ms faster
Qwen 3.8 27B Dense Q4K M 58 tok prompt · 96 tok output · 3 runs · Compared	Z 278.8 tok/s 208.1 ms L 102.6 tok/s 565.5 ms 272% +176.2 tok/s	Z 34.3 tok/s 16.9 ms/tok L 30.0 tok/s 33.3 ms/tok 114% +29.2 tok/s	Z 51.1 tok/s prompt+decode L 40.9 tok/s prompt+decode 125% +43.3 tok/s 2119 ms faster

Decode 87.4 tok/s 150% decode ratio

Prefill 847.8 tok/s 160% prefill ratio · 184 prompt tok

Fastest Qwen 3.6 35B A3B UD Q4K XL 153.1 tok/s

Overall 137% best row: Qwen 3.8 27B Dense Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 203p · 160out Compared

Prefill 190%

ZINC 1731.9 tok/s

llama 913.4 tok/s

Decode 148%

ZINC 138.6 tok/s

llama 93.9 tok/s

152% overall

Gemma 4 31B Q4K M 203p · 160out Compared

Prefill 115%

ZINC 312.1 tok/s

llama 271.7 tok/s

Decode 108%

ZINC 30.7 tok/s

llama 28.5 tok/s

109% overall

Muse Glimmer 30B Q4K M 179p · 160out Compared

Prefill 168%

ZINC 518.6 tok/s

llama 307.9 tok/s

Decode 109%

ZINC 35.8 tok/s

llama 32.7 tok/s

114% overall

Qwen 3.5 9B Q4K M 183p · 160out Compared

Prefill 151%

ZINC 1119.0 tok/s

llama 740.8 tok/s

Decode 127%

ZINC 101.3 tok/s

llama 79.9 tok/s

129% overall

Qwen 3.6 35B A3B UD Q4K XL 153p · 160out Compared

Prefill 155%

ZINC 961.8 tok/s

llama 619.8 tok/s

Decode 194%

ZINC 153.1 tok/s

llama 79.0 tok/s

189% overall

Qwen 3.8 27B Dense Q4K M 183p · 160out Compared

Prefill 178%

ZINC 443.6 tok/s

llama 249.0 tok/s

Decode 113%

ZINC 34.0 tok/s

llama 30.0 tok/s

118% overall

Details

Gemma 4 26B-A4B MoE Q4K M 203 tok prompt · 160 tok output

Compared

Prefill 190%

ZINC 1731.9 tok/s

llama 913.4 tok/s

Decode 148%

ZINC 138.6 tok/s

llama 93.9 tok/s

Overall 152%

ZINC 285.4 tok/s

llama 188.4 tok/s

Prompt+decode. Cold latency: ZINC 1725 ms · llama.cpp 2275 ms

Gemma 4 31B Q4K M 203 tok prompt · 160 tok output

Compared

Prefill 115%

ZINC 312.1 tok/s

llama 271.7 tok/s

Decode 108%

ZINC 30.7 tok/s

llama 28.5 tok/s

Overall 109%

ZINC 61.9 tok/s

llama 57.0 tok/s

Prompt+decode. Cold latency: ZINC 6236 ms · llama.cpp 6781 ms

Muse Glimmer 30B Q4K M 179 tok prompt · 160 tok output

Compared

Prefill 168%

ZINC 518.6 tok/s

llama 307.9 tok/s

Decode 109%

ZINC 35.8 tok/s

llama 32.7 tok/s

Overall 114%

ZINC 70.4 tok/s

llama 62.0 tok/s

Prompt+decode. Cold latency: ZINC 5196 ms · llama.cpp 5781 ms

Qwen 3.5 9B Q4K M 183 tok prompt · 160 tok output

Compared

Prefill 151%

ZINC 1119.0 tok/s

llama 740.8 tok/s

Decode 127%

ZINC 101.3 tok/s

llama 79.9 tok/s

Overall 129%

ZINC 196.9 tok/s

llama 152.5 tok/s

Prompt+decode. Cold latency: ZINC 2243 ms · llama.cpp 2786 ms

Qwen 3.6 35B A3B UD Q4K XL 153 tok prompt · 160 tok output

Compared

Prefill 155%

ZINC 961.8 tok/s

llama 619.8 tok/s

Decode 194%

ZINC 153.1 tok/s

llama 79.0 tok/s

Overall 189%

ZINC 259.9 tok/s

llama 137.7 tok/s

Prompt+decode. Cold latency: ZINC 1636 ms · llama.cpp 2747 ms

Qwen 3.8 27B Dense Q4K M 183 tok prompt · 160 tok output

Compared

Prefill 178%

ZINC 443.6 tok/s

llama 249.0 tok/s

Decode 113%

ZINC 34.0 tok/s

llama 30.0 tok/s

Overall 118%

ZINC 66.8 tok/s

llama 56.6 tok/s

Prompt+decode. Cold latency: ZINC 3244 ms · llama.cpp 6556 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 203 tok prompt · 160 tok output · 3 runs · Compared	Z 1731.9 tok/s 117.2 ms L 913.4 tok/s 222.3 ms 190% +818.5 tok/s	Z 138.6 tok/s 7.2 ms/tok L 93.9 tok/s 10.7 ms/tok 148% +44.7 tok/s	Z 285.4 tok/s prompt+decode L 188.4 tok/s prompt+decode 152% +97.0 tok/s 550 ms faster
Gemma 4 31B Q4K M 203 tok prompt · 160 tok output · 3 runs · Compared	Z 312.1 tok/s 650.4 ms L 271.7 tok/s 747.0 ms 115% +40.4 tok/s	Z 30.7 tok/s 32.6 ms/tok L 28.5 tok/s 35.1 ms/tok 108% +2.2 tok/s	Z 61.9 tok/s prompt+decode L 57.0 tok/s prompt+decode 109% +4.9 tok/s 545 ms faster
Muse Glimmer 30B Q4K M 179 tok prompt · 160 tok output · 3 runs · Compared	Z 518.6 tok/s 345.2 ms L 307.9 tok/s 581.4 ms 168% +210.7 tok/s	Z 35.8 tok/s 28.0 ms/tok L 32.7 tok/s 30.6 ms/tok 109% +3.1 tok/s	Z 70.4 tok/s prompt+decode L 62.0 tok/s prompt+decode 114% +8.4 tok/s 585 ms faster
Qwen 3.5 9B Q4K M 183 tok prompt · 160 tok output · 3 runs · Compared	Z 1119.0 tok/s 163.5 ms L 740.8 tok/s 247.0 ms 151% +378.2 tok/s	Z 101.3 tok/s 9.9 ms/tok L 79.9 tok/s 12.5 ms/tok 127% +21.4 tok/s	Z 196.9 tok/s prompt+decode L 152.5 tok/s prompt+decode 129% +44.3 tok/s 543 ms faster
Qwen 3.6 35B A3B UD Q4K XL 153 tok prompt · 160 tok output · 3 runs · Compared	Z 961.8 tok/s 159.1 ms L 619.8 tok/s 246.8 ms 155% +342.0 tok/s	Z 153.1 tok/s 6.5 ms/tok L 79.0 tok/s 12.7 ms/tok 194% +74.1 tok/s	Z 259.9 tok/s prompt+decode L 137.7 tok/s prompt+decode 189% +122.2 tok/s 1111 ms faster
Qwen 3.8 27B Dense Q4K M 183 tok prompt · 160 tok output · 3 runs · Compared	Z 443.6 tok/s 412.5 ms L 249.0 tok/s 735.0 ms 178% +194.7 tok/s	Z 34.0 tok/s 15.4 ms/tok L 30.0 tok/s 33.3 ms/tok 113% +34.9 tok/s	Z 66.8 tok/s prompt+decode L 56.6 tok/s prompt+decode 118% +62.6 tok/s 3313 ms faster

Decode 87.0 tok/s 151% decode ratio

Prefill 1032.2 tok/s 143% prefill ratio · 341 prompt tok

Fastest Qwen 3.6 35B A3B UD Q4K XL 152.7 tok/s

Overall 135% best row: Qwen 3.8 27B Dense Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 353p · 128out Compared

Prefill 164%

ZINC 1894.5 tok/s

llama 1153.5 tok/s

Decode 147%

ZINC 136.2 tok/s

llama 92.6 tok/s

150% overall

Gemma 4 31B Q4K M 353p · 128out Compared

Prefill 120%

ZINC 346.8 tok/s

llama 290.1 tok/s

Decode 108%

ZINC 30.5 tok/s

llama 28.3 tok/s

110% overall

Muse Glimmer 30B Q4K M 326p · 128out Compared

Prefill 126%

ZINC 489.2 tok/s

llama 389.7 tok/s

Decode 109%

ZINC 35.7 tok/s

llama 32.7 tok/s

112% overall

Qwen 3.5 9B Q4K M 349p · 128out Compared

Prefill 147%

ZINC 1478.0 tok/s

llama 1008.7 tok/s

Decode 127%

ZINC 101.1 tok/s

llama 79.7 tok/s

130% overall

Qwen 3.6 35B A3B UD Q4K XL 315p · 128out Compared

Prefill 158%

ZINC 1516.6 tok/s

llama 959.8 tok/s

Decode 198%

ZINC 152.7 tok/s

llama 76.9 tok/s

190% overall

Qwen 3.8 27B Dense Q4K M 349p · 128out Compared

Prefill 142%

ZINC 467.9 tok/s

llama 330.0 tok/s

Decode 113%

ZINC 34.0 tok/s

llama 30.0 tok/s

118% overall

Details

Gemma 4 26B-A4B MoE Q4K M 353 tok prompt · 128 tok output

Compared

Prefill 164%

ZINC 1894.5 tok/s

llama 1153.5 tok/s

Decode 147%

ZINC 136.2 tok/s

llama 92.6 tok/s

Overall 150%

ZINC 427.2 tok/s

llama 284.9 tok/s

Prompt+decode. Cold latency: ZINC 1754 ms · llama.cpp 2065 ms

Gemma 4 31B Q4K M 353 tok prompt · 128 tok output

Compared

Prefill 120%

ZINC 346.8 tok/s

llama 290.1 tok/s

Decode 108%

ZINC 30.5 tok/s

llama 28.3 tok/s

Overall 110%

ZINC 92.2 tok/s

llama 83.8 tok/s

Prompt+decode. Cold latency: ZINC 5768 ms · llama.cpp 6233 ms

Muse Glimmer 30B Q4K M 326 tok prompt · 128 tok output

Compared

Prefill 126%

ZINC 489.2 tok/s

llama 389.7 tok/s

Decode 109%

ZINC 35.7 tok/s

llama 32.7 tok/s

Overall 112%

ZINC 106.9 tok/s

llama 95.6 tok/s

Prompt+decode. Cold latency: ZINC 4681 ms · llama.cpp 5075 ms

Qwen 3.5 9B Q4K M 349 tok prompt · 128 tok output

Compared

Prefill 147%

ZINC 1478.0 tok/s

llama 1008.7 tok/s

Decode 127%

ZINC 101.1 tok/s

llama 79.7 tok/s

Overall 130%

ZINC 317.5 tok/s

llama 244.4 tok/s

Prompt+decode. Cold latency: ZINC 1910 ms · llama.cpp 2549 ms

Qwen 3.6 35B A3B UD Q4K XL 315 tok prompt · 128 tok output

Compared

Prefill 158%

ZINC 1516.6 tok/s

llama 959.8 tok/s

Decode 198%

ZINC 152.7 tok/s

llama 76.9 tok/s

Overall 190%

ZINC 423.6 tok/s

llama 222.4 tok/s

Prompt+decode. Cold latency: ZINC 1621 ms · llama.cpp 2389 ms

Qwen 3.8 27B Dense Q4K M 349 tok prompt · 128 tok output

Compared

Prefill 142%

ZINC 467.9 tok/s

llama 330.0 tok/s

Decode 113%

ZINC 34.0 tok/s

llama 30.0 tok/s

Overall 118%

ZINC 106.0 tok/s

llama 89.7 tok/s

Prompt+decode. Cold latency: ZINC 3099 ms · llama.cpp 5736 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 353 tok prompt · 128 tok output · 3 runs · Compared	Z 1894.5 tok/s 186.3 ms L 1153.5 tok/s 306.0 ms 164% +741.0 tok/s	Z 136.2 tok/s 7.3 ms/tok L 92.6 tok/s 10.8 ms/tok 147% +43.6 tok/s	Z 427.2 tok/s prompt+decode L 284.9 tok/s prompt+decode 150% +142.3 tok/s 311 ms faster
Gemma 4 31B Q4K M 353 tok prompt · 128 tok output · 3 runs · Compared	Z 346.8 tok/s 1018.0 ms L 290.1 tok/s 1216.7 ms 120% +56.6 tok/s	Z 30.5 tok/s 32.8 ms/tok L 28.3 tok/s 35.4 ms/tok 108% +2.2 tok/s	Z 92.2 tok/s prompt+decode L 83.8 tok/s prompt+decode 110% +8.5 tok/s 465 ms faster
Muse Glimmer 30B Q4K M 326 tok prompt · 128 tok output · 3 runs · Compared	Z 489.2 tok/s 666.3 ms L 389.7 tok/s 836.6 ms 126% +99.6 tok/s	Z 35.7 tok/s 28.0 ms/tok L 32.7 tok/s 30.6 ms/tok 109% +3.0 tok/s	Z 106.9 tok/s prompt+decode L 95.6 tok/s prompt+decode 112% +11.3 tok/s 394 ms faster
Qwen 3.5 9B Q4K M 349 tok prompt · 128 tok output · 3 runs · Compared	Z 1478.0 tok/s 236.1 ms L 1008.7 tok/s 346.0 ms 147% +469.3 tok/s	Z 101.1 tok/s 9.9 ms/tok L 79.7 tok/s 12.5 ms/tok 127% +21.3 tok/s	Z 317.5 tok/s prompt+decode L 244.4 tok/s prompt+decode 130% +73.0 tok/s 639 ms faster
Qwen 3.6 35B A3B UD Q4K XL 315 tok prompt · 128 tok output · 3 runs · Compared	Z 1516.6 tok/s 207.7 ms L 959.8 tok/s 328.2 ms 158% +556.7 tok/s	Z 152.7 tok/s 6.5 ms/tok L 76.9 tok/s 13.0 ms/tok 198% +75.8 tok/s	Z 423.6 tok/s prompt+decode L 222.4 tok/s prompt+decode 190% +201.2 tok/s 768 ms faster
Qwen 3.8 27B Dense Q4K M 349 tok prompt · 128 tok output · 3 runs · Compared	Z 467.9 tok/s 745.8 ms L 330.0 tok/s 1057.5 ms 142% +137.9 tok/s	Z 34.0 tok/s 15.2 ms/tok L 30.0 tok/s 33.3 ms/tok 113% +35.6 tok/s	Z 106.0 tok/s prompt+decode L 89.7 tok/s prompt+decode 118% +87.2 tok/s 2637 ms faster

Decode 87.4 tok/s 150% decode ratio

Prefill 568.7 tok/s 204% prefill ratio · 79 prompt tok

Fastest Qwen 3.6 35B A3B UD Q4K XL 153.6 tok/s

Overall 138% best row: Qwen 3.8 27B Dense Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 81p · 256out Compared

Prefill 282%

ZINC 1281.1 tok/s

llama 453.8 tok/s

Decode 145%

ZINC 138.9 tok/s

llama 95.6 tok/s

150% overall

Gemma 4 31B Q4K M 81p · 256out Compared

Prefill 244%

ZINC 328.1 tok/s

llama 134.7 tok/s

Decode 108%

ZINC 30.7 tok/s

llama 28.5 tok/s

112% overall

Muse Glimmer 30B Q4K M 89p · 256out Compared

Prefill 153%

ZINC 342.9 tok/s

llama 223.7 tok/s

Decode 109%

ZINC 35.7 tok/s

llama 32.8 tok/s

110% overall

Qwen 3.5 9B Q4K M 80p · 256out Compared

Prefill 174%

ZINC 674.0 tok/s

llama 386.6 tok/s

Decode 127%

ZINC 101.0 tok/s

llama 79.7 tok/s

129% overall

Qwen 3.6 35B A3B UD Q4K XL 63p · 256out Compared

Prefill 180%

ZINC 536.0 tok/s

llama 297.6 tok/s

Decode 198%

ZINC 153.6 tok/s

llama 77.8 tok/s

196% overall

Qwen 3.8 27B Dense Q4K M 80p · 256out Compared

Prefill 193%

ZINC 250.4 tok/s

llama 129.5 tok/s

Decode 113%

ZINC 33.9 tok/s

llama 30.0 tok/s

116% overall

Details

Gemma 4 26B-A4B MoE Q4K M 81 tok prompt · 256 tok output

Compared

Prefill 282%

ZINC 1281.1 tok/s

llama 453.8 tok/s

Decode 145%

ZINC 138.9 tok/s

llama 95.6 tok/s

Overall 150%

ZINC 176.8 tok/s

llama 118.0 tok/s

Prompt+decode. Cold latency: ZINC 2420 ms · llama.cpp 3376 ms

Gemma 4 31B Q4K M 81 tok prompt · 256 tok output

Compared

Prefill 244%

ZINC 328.1 tok/s

llama 134.7 tok/s

Decode 108%

ZINC 30.7 tok/s

llama 28.5 tok/s

Overall 112%

ZINC 39.3 tok/s

llama 35.2 tok/s

Prompt+decode. Cold latency: ZINC 8987 ms · llama.cpp 10192 ms

Muse Glimmer 30B Q4K M 89 tok prompt · 256 tok output

Compared

Prefill 153%

ZINC 342.9 tok/s

llama 223.7 tok/s

Decode 109%

ZINC 35.7 tok/s

llama 32.8 tok/s

Overall 110%

ZINC 46.4 tok/s

llama 42.0 tok/s

Prompt+decode. Cold latency: ZINC 7837 ms · llama.cpp 8560 ms

Qwen 3.5 9B Q4K M 80 tok prompt · 256 tok output

Compared

Prefill 174%

ZINC 674.0 tok/s

llama 386.6 tok/s

Decode 127%

ZINC 101.0 tok/s

llama 79.7 tok/s

Overall 129%

ZINC 126.7 tok/s

llama 98.3 tok/s

Prompt+decode. Cold latency: ZINC 3058 ms · llama.cpp 3852 ms

Qwen 3.6 35B A3B UD Q4K XL 63 tok prompt · 256 tok output

Compared

Prefill 180%

ZINC 536.0 tok/s

llama 297.6 tok/s

Decode 198%

ZINC 153.6 tok/s

llama 77.8 tok/s

Overall 196%

ZINC 178.8 tok/s

llama 91.1 tok/s

Prompt+decode. Cold latency: ZINC 2248 ms · llama.cpp 3838 ms

Qwen 3.8 27B Dense Q4K M 80 tok prompt · 256 tok output

Compared

Prefill 193%

ZINC 250.4 tok/s

llama 129.5 tok/s

Decode 113%

ZINC 33.9 tok/s

llama 30.0 tok/s

Overall 116%

ZINC 42.6 tok/s

llama 36.8 tok/s

Prompt+decode. Cold latency: ZINC 4662 ms · llama.cpp 9820 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 81 tok prompt · 256 tok output · 3 runs · Compared	Z 1281.1 tok/s 63.2 ms L 453.8 tok/s 178.5 ms 282% +827.3 tok/s	Z 138.9 tok/s 7.2 ms/tok L 95.6 tok/s 10.5 ms/tok 145% +43.3 tok/s	Z 176.8 tok/s prompt+decode L 118.0 tok/s prompt+decode 150% +58.8 tok/s 956 ms faster
Gemma 4 31B Q4K M 81 tok prompt · 256 tok output · 3 runs · Compared	Z 328.1 tok/s 246.8 ms L 134.7 tok/s 601.4 ms 244% +193.5 tok/s	Z 30.7 tok/s 32.6 ms/tok L 28.5 tok/s 35.0 ms/tok 108% +2.2 tok/s	Z 39.3 tok/s prompt+decode L 35.2 tok/s prompt+decode 112% +4.1 tok/s 1204 ms faster
Muse Glimmer 30B Q4K M 89 tok prompt · 256 tok output · 3 runs · Compared	Z 342.9 tok/s 259.6 ms L 223.7 tok/s 397.8 ms 153% +119.1 tok/s	Z 35.7 tok/s 28.0 ms/tok L 32.8 tok/s 30.5 ms/tok 109% +2.9 tok/s	Z 46.4 tok/s prompt+decode L 42.0 tok/s prompt+decode 110% +4.4 tok/s 723 ms faster
Qwen 3.5 9B Q4K M 80 tok prompt · 256 tok output · 3 runs · Compared	Z 674.0 tok/s 118.7 ms L 386.6 tok/s 206.9 ms 174% +287.4 tok/s	Z 101.0 tok/s 9.9 ms/tok L 79.7 tok/s 12.5 ms/tok 127% +21.3 tok/s	Z 126.7 tok/s prompt+decode L 98.3 tok/s prompt+decode 129% +28.4 tok/s 794 ms faster
Qwen 3.6 35B A3B UD Q4K XL 63 tok prompt · 256 tok output · 3 runs · Compared	Z 536.0 tok/s 117.5 ms L 297.6 tok/s 211.7 ms 180% +238.4 tok/s	Z 153.6 tok/s 6.5 ms/tok L 77.8 tok/s 12.9 ms/tok 198% +75.9 tok/s	Z 178.8 tok/s prompt+decode L 91.1 tok/s prompt+decode 196% +87.8 tok/s 1590 ms faster
Qwen 3.8 27B Dense Q4K M 80 tok prompt · 256 tok output · 3 runs · Compared	Z 250.4 tok/s 319.5 ms L 129.5 tok/s 617.8 ms 193% +120.9 tok/s	Z 33.9 tok/s 15.5 ms/tok L 30.0 tok/s 33.3 ms/tok 113% +34.6 tok/s	Z 42.6 tok/s prompt+decode L 36.8 tok/s prompt+decode 116% +41.7 tok/s 5158 ms faster

Methodology ZINC server vs llama.cpp server on the same RDNA node · Four-scenario matrix with same-file baselines

3 measured · 1 warmup

Benchmark runner Exact prompts and token caps Prompt matrix test Published JSON

Hardware: one AMD Radeon AI PRO R9700 benchmark node with 32 GB VRAM and 640 GB/s memory bandwidth. ZINC and llama.cpp run on the same Ubuntu host and use the same GGUF file for each model.
ZINC path: sync the current source tree to the RDNA node, build ReleaseFast with the selected vulkan backend, then measure generation through one reusable single-slot ZINC server per model.
ZINC RDNA backend: vulkan.
Baseline backend: llama.cpp Vulkan (Vulkan0) on the same node and the same model file.
Baseline path: launch llama.cpp against the same model file, preferring one reusable single-slot llama-server per model across the full scenario matrix and falling back to llama-cli when the server path is unavailable.
Scenarios: Quick Chat, Coding Review, Incident Context, and Long Coding Draft. The prompts are real-world chat, code-review, support-context, and coding-plan workloads instead of synthetic factual completions.
Statistics: one warmup pass is discarded, then 3 measured runs are collected. Published prefill, decode, end-to-end throughput, and latency values are medians.
Execution order: the harness records the full ZINC scenario matrix before starting the llama.cpp baseline phase, so only one inference engine owns the GPU during measurement.
Power policy: before either engine starts, the harness selects PCIe ASPM performance mode and locks the discrete AMD GPU's memory DPM policy high to prevent idle-clock drift between runs.
Run hygiene: published RDNA runs assume a clean node with no competing ZINC, llama.cpp, or other GPU workloads.

Intel Arc · Intel Arc bench node

Vulkan backend

Intel(R) Graphics (BMG G31) · n/a · 3 measured · 1 warmup

Exact configuration

GPU: Intel(R) Graphics (BMG G31)
Chip: Intel Arc Pro B70 / BMG G31
Memory: n/a
OS: Ubuntu Linux
ZINC backend: Vulkan
llama.cpp backend: Vulkan (--device Vulkan0)
Prompt parity: 6/16 rows read the same prompt tokens in both engines
Build: zig build -Doptimize=ReleaseFast
ZINC commit: n/a
llama.cpp build: 8863
Artifact: Jul 7, 2026, 8:45 PM

Methodology & reproducibility

Hardware: one Intel Arc Vulkan benchmark node. ZINC and llama.cpp run on the same Ubuntu host and use the same GGUF file for each model.
Baseline backend: llama.cpp Vulkan (--device Vulkan0) on the same node — the same backend as ZINC.
ZINC path: optionally sync the current source tree to the Intel node, build with zig build -Doptimize=ReleaseFast, then measure generation through the ZINC CLI.
Baseline path: launch llama.cpp against the same model file, preferring one reusable llama-server per model across the full scenario matrix and falling back to llama-cli when the server path is unavailable or fails to start.
Scenarios: Quick Chat, Coding Review, Incident Context, and Long Coding Draft. The prompts are real-world chat, code-review, support-context, and coding-plan workloads instead of synthetic factual completions.
Statistics: one warmup pass is discarded, then 3 measured runs are collected. Published prefill, decode, end-to-end throughput, and latency values are medians.
Execution order: the harness records the full ZINC scenario matrix before starting the llama.cpp baseline phase, so only one inference engine owns the GPU during measurement.
Run hygiene: published Intel runs assume a clean node with no competing ZINC, llama.cpp, or other GPU workloads.

No speculative decoding on this backend. The ROCm and Vulkan tab has an MTP on/off switch for the models that support it.

Fastest 75.3 tok/s Qwen 3.6 35B A3B UD Q4K XL

Decode 53.6 tok/s average

Overall 104% prefill + decode vs llama.cpp

Choose a workload

Prefill and decode are shown separately below.

Decode 53.6 tok/s 103% decode ratio

Prefill 249.0 tok/s 164% prefill ratio · 43 prompt tok

Fastest Qwen 3.6 35B A3B UD Q4K XL 75.3 tok/s

Overall 109% best row: Gemma 4 26B-A4B MoE Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 49p · 96out Compared

Prefill 199%

ZINC 493.0 tok/s

llama 247.6 tok/s

Decode 104%

ZINC 65.0 tok/s

llama 62.4 tok/s

111% overall

Gemma 4 31B Q4K M 49p · 96out Compared

Prefill 180%

ZINC 120.8 tok/s

llama 67.2 tok/s

Decode 104%

ZINC 18.0 tok/s

llama 17.4 tok/s

110% overall

Qwen 3.5 9B Q4K M 36p · 96out Compared

Prefill 135%

ZINC 191.1 tok/s

llama 142.0 tok/s

Decode 104%

ZINC 56.0 tok/s

llama 54.0 tok/s

106% overall

Qwen 3.6 35B A3B UD Q4K XL 36p · 96out Compared

Prefill 141%

ZINC 191.2 tok/s

llama 135.4 tok/s

Decode 100%

ZINC 75.3 tok/s

llama 75.1 tok/s

105% overall

Details

Gemma 4 26B-A4B MoE Q4K M 49 tok prompt · 96 tok output

Compared

Prefill 199%

ZINC 493.0 tok/s

llama 247.6 tok/s

Decode 104%

ZINC 65.0 tok/s

llama 62.4 tok/s

Overall 111%

ZINC 92.0 tok/s

llama 84.7 tok/s

Prompt+decode. Cold latency: ZINC 6851 ms · llama.cpp 2070 ms

Gemma 4 31B Q4K M 49 tok prompt · 96 tok output

Compared

Prefill 180%

ZINC 120.8 tok/s

llama 67.2 tok/s

Decode 104%

ZINC 18.0 tok/s

llama 17.4 tok/s

Overall 110%

ZINC 25.3 tok/s

llama 23.5 tok/s

Prompt+decode. Cold latency: ZINC 13500 ms · llama.cpp 6768 ms

Qwen 3.5 9B Q4K M 36 tok prompt · 96 tok output

Compared

Prefill 135%

ZINC 191.1 tok/s

llama 142.0 tok/s

Decode 104%

ZINC 56.0 tok/s

llama 54.0 tok/s

Overall 106%

ZINC 69.4 tok/s

llama 64.7 tok/s

Prompt+decode. Cold latency: ZINC 4503 ms · llama.cpp 2365 ms

Qwen 3.6 35B A3B UD Q4K XL 36 tok prompt · 96 tok output

Compared

Prefill 141%

ZINC 191.2 tok/s

llama 135.4 tok/s

Decode 100%

ZINC 75.3 tok/s

llama 75.1 tok/s

Overall 105%

ZINC 90.2 tok/s

llama 85.2 tok/s

Prompt+decode. Cold latency: ZINC 8485 ms · llama.cpp 1861 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 49 tok prompt · 96 tok output · 3 runs · Compared	Z 493.0 tok/s 99.4 ms L 247.6 tok/s 210.0 ms 199% +245.4 tok/s	Z 65.0 tok/s 15.4 ms/tok L 62.4 tok/s 16.0 ms/tok 104% +2.6 tok/s	Z 92.0 tok/s prompt+decode L 84.7 tok/s prompt+decode 111% +7.3 tok/s 4781 ms slower
Gemma 4 31B Q4K M 49 tok prompt · 96 tok output · 3 runs · Compared	Z 120.8 tok/s 405.5 ms L 67.2 tok/s 773.5 ms 180% +53.6 tok/s	Z 18.0 tok/s 55.5 ms/tok L 17.4 tok/s 57.6 ms/tok 104% +0.6 tok/s	Z 25.3 tok/s prompt+decode L 23.5 tok/s prompt+decode 110% +1.8 tok/s 6732 ms slower
Qwen 3.5 9B Q4K M 36 tok prompt · 96 tok output · 3 runs · Compared	Z 191.1 tok/s 188.4 ms L 142.0 tok/s 246.5 ms 135% +49.1 tok/s	Z 56.0 tok/s 17.9 ms/tok L 54.0 tok/s 18.5 ms/tok 104% +2.0 tok/s	Z 69.4 tok/s prompt+decode L 64.7 tok/s prompt+decode 106% +4.6 tok/s 2137 ms slower
Qwen 3.6 35B A3B UD Q4K XL 36 tok prompt · 96 tok output · 3 runs · Compared	Z 191.2 tok/s 188.3 ms L 135.4 tok/s 258.5 ms 141% +55.8 tok/s	Z 75.3 tok/s 13.3 ms/tok L 75.1 tok/s 13.3 ms/tok 100% +0.2 tok/s	Z 90.2 tok/s prompt+decode L 85.2 tok/s prompt+decode 105% +5.0 tok/s 6624 ms slower

Decode 52.5 tok/s 103% decode ratio

Prefill 506.2 tok/s 117% prefill ratio · 183 prompt tok

Fastest Qwen 3.6 35B A3B UD Q4K XL 74.7 tok/s

Overall 104% best row: Gemma 4 26B-A4B MoE Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 192p · 160out Compared

Prefill 171%

ZINC 1025.9 tok/s

llama 599.8 tok/s

Decode 103%

ZINC 62.3 tok/s

llama 60.2 tok/s

108% overall

Gemma 4 31B Q4K M 192p · 160out Compared

Prefill 113%

ZINC 168.6 tok/s

llama 149.7 tok/s

Decode 105%

ZINC 17.5 tok/s

llama 16.6 tok/s

106% overall

Qwen 3.5 9B Q4K M 174p · 160out Compared

Prefill 119%

ZINC 537.4 tok/s

llama 449.8 tok/s

Decode 104%

ZINC 55.6 tok/s

llama 53.7 tok/s

104% overall

Qwen 3.6 35B A3B UD Q4K XL 174p · 160out Compared

Prefill 64%

ZINC 292.9 tok/s

llama 456.3 tok/s

Decode 100%

ZINC 74.7 tok/s

llama 74.4 tok/s

91% overall

Details

Gemma 4 26B-A4B MoE Q4K M 192 tok prompt · 160 tok output

Compared

Prefill 171%

ZINC 1025.9 tok/s

llama 599.8 tok/s

Decode 103%

ZINC 62.3 tok/s

llama 60.2 tok/s

Overall 108%

ZINC 127.7 tok/s

llama 119.1 tok/s

Prompt+decode. Cold latency: ZINC 7988 ms · llama.cpp 3290 ms

Gemma 4 31B Q4K M 192 tok prompt · 160 tok output

Compared

Prefill 113%

ZINC 168.6 tok/s

llama 149.7 tok/s

Decode 105%

ZINC 17.5 tok/s

llama 16.6 tok/s

Overall 106%

ZINC 34.2 tok/s

llama 32.5 tok/s

Prompt+decode. Cold latency: ZINC 17385 ms · llama.cpp 11254 ms

Qwen 3.5 9B Q4K M 174 tok prompt · 160 tok output

Compared

Prefill 119%

ZINC 537.4 tok/s

llama 449.8 tok/s

Decode 104%

ZINC 55.6 tok/s

llama 53.7 tok/s

Overall 104%

ZINC 104.4 tok/s

llama 94.3 tok/s

Prompt+decode. Cold latency: ZINC 5386 ms · llama.cpp 3682 ms

Qwen 3.6 35B A3B UD Q4K XL 174 tok prompt · 160 tok output

Compared

Prefill 64%

ZINC 292.9 tok/s

llama 456.3 tok/s

Decode 100%

ZINC 74.7 tok/s

llama 74.4 tok/s

Overall 91%

ZINC 122.0 tok/s

llama 126.0 tok/s

Prompt+decode. Cold latency: ZINC 9364 ms · llama.cpp 2849 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 192 tok prompt · 160 tok output · 3 runs · Compared	Z 1025.9 tok/s 187.2 ms L 599.8 tok/s 325.1 ms 171% +426.1 tok/s	Z 62.3 tok/s 16.1 ms/tok L 60.2 tok/s 16.6 ms/tok 103% +2.0 tok/s	Z 127.7 tok/s prompt+decode L 119.1 tok/s prompt+decode 108% +8.6 tok/s 4698 ms slower
Gemma 4 31B Q4K M 192 tok prompt · 160 tok output · 3 runs · Compared	Z 168.6 tok/s 1139.2 ms L 149.7 tok/s 1302.3 ms 113% +18.8 tok/s	Z 17.5 tok/s 57.1 ms/tok L 16.6 tok/s 60.2 ms/tok 105% +0.9 tok/s	Z 34.2 tok/s prompt+decode L 32.5 tok/s prompt+decode 106% +1.8 tok/s 6131 ms slower
Qwen 3.5 9B Q4K M 174 tok prompt · 160 tok output · 3 runs · Compared	Z 537.4 tok/s 323.8 ms L 449.8 tok/s 340.1 ms 119% +87.6 tok/s	Z 55.6 tok/s 18.0 ms/tok L 53.7 tok/s 18.6 ms/tok 104% +1.9 tok/s	Z 104.4 tok/s prompt+decode L 94.3 tok/s prompt+decode 104% +10.1 tok/s 1704 ms slower
Qwen 3.6 35B A3B UD Q4K XL 174 tok prompt · 160 tok output · 3 runs · Compared	Z 292.9 tok/s 594.0 ms L 456.3 tok/s 335.3 ms 64% -163.3 tok/s	Z 74.7 tok/s 13.4 ms/tok L 74.4 tok/s 13.4 ms/tok 100% +0.2 tok/s	Z 122.0 tok/s prompt+decode L 126.0 tok/s prompt+decode 91% -3.9 tok/s 6515 ms slower

Decode 51.7 tok/s 103% decode ratio

Prefill 749.0 tok/s 104% prefill ratio · 334 prompt tok

Fastest Qwen 3.6 35B A3B UD Q4K XL 73.7 tok/s

Overall 100% best row: Gemma 4 26B-A4B MoE Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 346p · 128out Compared

Prefill 132%

ZINC 1122.3 tok/s

llama 853.2 tok/s

Decode 102%

ZINC 60.7 tok/s

llama 59.4 tok/s

106% overall

Gemma 4 31B Q4K M 346p · 128out Compared

Prefill 69%

ZINC 159.2 tok/s

llama 229.2 tok/s

Decode 106%

ZINC 17.3 tok/s

llama 16.3 tok/s

98% overall

Qwen 3.5 9B Q4K M 322p · 128out Compared

Prefill 76%

ZINC 629.2 tok/s

llama 827.0 tok/s

Decode 103%

ZINC 55.3 tok/s

llama 53.5 tok/s

98% overall

Qwen 3.6 35B A3B UD Q4K XL 322p · 128out Compared

Prefill 138%

ZINC 1085.3 tok/s

llama 785.2 tok/s

Decode 99%

ZINC 73.7 tok/s

llama 74.5 tok/s

104% overall

Details

Gemma 4 26B-A4B MoE Q4K M 346 tok prompt · 128 tok output

Compared

Prefill 132%

ZINC 1122.3 tok/s

llama 853.2 tok/s

Decode 102%

ZINC 60.7 tok/s

llama 59.4 tok/s

Overall 106%

ZINC 196.2 tok/s

llama 184.8 tok/s

Prompt+decode. Cold latency: ZINC 7663 ms · llama.cpp 2877 ms

Gemma 4 31B Q4K M 346 tok prompt · 128 tok output

Compared

Prefill 69%

ZINC 159.2 tok/s

llama 229.2 tok/s

Decode 106%

ZINC 17.3 tok/s

llama 16.3 tok/s

Overall 98%

ZINC 49.4 tok/s

llama 50.5 tok/s

Prompt+decode. Cold latency: ZINC 16227 ms · llama.cpp 9693 ms

Qwen 3.5 9B Q4K M 322 tok prompt · 128 tok output

Compared

Prefill 76%

ZINC 629.2 tok/s

llama 827.0 tok/s

Decode 103%

ZINC 55.3 tok/s

llama 53.5 tok/s

Overall 98%

ZINC 159.2 tok/s

llama 159.6 tok/s

Prompt+decode. Cold latency: ZINC 4968 ms · llama.cpp 3062 ms

Qwen 3.6 35B A3B UD Q4K XL 322 tok prompt · 128 tok output

Compared

Prefill 138%

ZINC 1085.3 tok/s

llama 785.2 tok/s

Decode 99%

ZINC 73.7 tok/s

llama 74.5 tok/s

Overall 104%

ZINC 221.2 tok/s

llama 208.9 tok/s

Prompt+decode. Cold latency: ZINC 8688 ms · llama.cpp 2448 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 346 tok prompt · 128 tok output · 3 runs · Compared	Z 1122.3 tok/s 308.3 ms L 853.2 tok/s 404.3 ms 132% +269.1 tok/s	Z 60.7 tok/s 16.5 ms/tok L 59.4 tok/s 16.8 ms/tok 102% +1.3 tok/s	Z 196.2 tok/s prompt+decode L 184.8 tok/s prompt+decode 106% +11.4 tok/s 4786 ms slower
Gemma 4 31B Q4K M 346 tok prompt · 128 tok output · 3 runs · Compared	Z 159.2 tok/s 2174.0 ms L 229.2 tok/s 1505.2 ms 69% -70.1 tok/s	Z 17.3 tok/s 58.0 ms/tok L 16.3 tok/s 61.5 ms/tok 106% +1.0 tok/s	Z 49.4 tok/s prompt+decode L 50.5 tok/s prompt+decode 98% -1.1 tok/s 6533 ms slower
Qwen 3.5 9B Q4K M 322 tok prompt · 128 tok output · 3 runs · Compared	Z 629.2 tok/s 511.7 ms L 827.0 tok/s 380.9 ms 76% -197.8 tok/s	Z 55.3 tok/s 18.1 ms/tok L 53.5 tok/s 18.7 ms/tok 103% +1.8 tok/s	Z 159.2 tok/s prompt+decode L 159.6 tok/s prompt+decode 98% -0.5 tok/s 1906 ms slower
Qwen 3.6 35B A3B UD Q4K XL 322 tok prompt · 128 tok output · 3 runs · Compared	Z 1085.3 tok/s 296.7 ms L 785.2 tok/s 401.2 ms 138% +300.1 tok/s	Z 73.7 tok/s 13.6 ms/tok L 74.5 tok/s 13.4 ms/tok 99% -0.8 tok/s	Z 221.2 tok/s prompt+decode L 208.9 tok/s prompt+decode 104% +12.2 tok/s 6240 ms slower

Decode 52.7 tok/s 103% decode ratio

Prefill 339.1 tok/s 141% prefill ratio · 67 prompt tok

Fastest Qwen 3.6 35B A3B UD Q4K XL 74.5 tok/s

Overall 105% best row: Gemma 4 31B Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 70p · 256out Compared

Prefill 183%

ZINC 612.2 tok/s

llama 335.0 tok/s

Decode 104%

ZINC 63.1 tok/s

llama 61.0 tok/s

106% overall

Gemma 4 31B Q4K M 70p · 256out Compared

Prefill 169%

ZINC 138.8 tok/s

llama 82.1 tok/s

Decode 104%

ZINC 17.6 tok/s

llama 16.9 tok/s

107% overall

Qwen 3.5 9B Q4K M 64p · 256out Compared

Prefill 118%

ZINC 353.3 tok/s

llama 298.5 tok/s

Decode 104%

ZINC 55.5 tok/s

llama 53.6 tok/s

104% overall

Qwen 3.6 35B A3B UD Q4K XL 64p · 256out Compared

Prefill 92%

ZINC 252.2 tok/s

llama 273.6 tok/s

Decode 100%

ZINC 74.5 tok/s

llama 74.4 tok/s

100% overall

Details

Gemma 4 26B-A4B MoE Q4K M 70 tok prompt · 256 tok output

Compared

Prefill 183%

ZINC 612.2 tok/s

llama 335.0 tok/s

Decode 104%

ZINC 63.1 tok/s

llama 61.0 tok/s

Overall 106%

ZINC 78.2 tok/s

llama 74.5 tok/s

Prompt+decode. Cold latency: ZINC 9425 ms · llama.cpp 4763 ms

Gemma 4 31B Q4K M 70 tok prompt · 256 tok output

Compared

Prefill 169%

ZINC 138.8 tok/s

llama 82.1 tok/s

Decode 104%

ZINC 17.6 tok/s

llama 16.9 tok/s

Overall 107%

ZINC 21.6 tok/s

llama 20.5 tok/s

Prompt+decode. Cold latency: ZINC 21662 ms · llama.cpp 16440 ms

Qwen 3.5 9B Q4K M 64 tok prompt · 256 tok output

Compared

Prefill 118%

ZINC 353.3 tok/s

llama 298.5 tok/s

Decode 104%

ZINC 55.5 tok/s

llama 53.6 tok/s

Overall 104%

ZINC 66.8 tok/s

llama 64.0 tok/s

Prompt+decode. Cold latency: ZINC 6930 ms · llama.cpp 5300 ms

Qwen 3.6 35B A3B UD Q4K XL 64 tok prompt · 256 tok output

Compared

Prefill 92%

ZINC 252.2 tok/s

llama 273.6 tok/s

Decode 100%

ZINC 74.5 tok/s

llama 74.4 tok/s

Overall 100%

ZINC 86.7 tok/s

llama 86.9 tok/s

Prompt+decode. Cold latency: ZINC 10554 ms · llama.cpp 3986 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 70 tok prompt · 256 tok output · 3 runs · Compared	Z 612.2 tok/s 114.3 ms L 335.0 tok/s 217.9 ms 183% +277.2 tok/s	Z 63.1 tok/s 15.8 ms/tok L 61.0 tok/s 16.4 ms/tok 104% +2.1 tok/s	Z 78.2 tok/s prompt+decode L 74.5 tok/s prompt+decode 106% +3.7 tok/s 4662 ms slower
Gemma 4 31B Q4K M 70 tok prompt · 256 tok output · 3 runs · Compared	Z 138.8 tok/s 504.4 ms L 82.1 tok/s 889.3 ms 169% +56.7 tok/s	Z 17.6 tok/s 56.9 ms/tok L 16.9 tok/s 59.2 ms/tok 104% +0.7 tok/s	Z 21.6 tok/s prompt+decode L 20.5 tok/s prompt+decode 107% +1.1 tok/s 5222 ms slower
Qwen 3.5 9B Q4K M 64 tok prompt · 256 tok output · 3 runs · Compared	Z 353.3 tok/s 181.2 ms L 298.5 tok/s 211.1 ms 118% +54.8 tok/s	Z 55.5 tok/s 18.0 ms/tok L 53.6 tok/s 18.6 ms/tok 104% +1.9 tok/s	Z 66.8 tok/s prompt+decode L 64.0 tok/s prompt+decode 104% +2.8 tok/s 1630 ms slower
Qwen 3.6 35B A3B UD Q4K XL 64 tok prompt · 256 tok output · 3 runs · Compared	Z 252.2 tok/s 253.8 ms L 273.6 tok/s 230.2 ms 92% -21.5 tok/s	Z 74.5 tok/s 13.4 ms/tok L 74.4 tok/s 13.4 ms/tok 100% +0.1 tok/s	Z 86.7 tok/s prompt+decode L 86.9 tok/s prompt+decode 100% -0.2 tok/s 6568 ms slower

Methodology ZINC CLI vs llama.cpp on the same Intel Arc node · Four-scenario matrix with same-file baselines

3 measured · 1 warmup

Benchmark runner Exact prompts and token caps Prompt matrix test Published JSON

Hardware: one Intel Arc Vulkan benchmark node. ZINC and llama.cpp run on the same Ubuntu host and use the same GGUF file for each model.
Baseline backend: llama.cpp Vulkan (--device Vulkan0) on the same node — the same backend as ZINC.
ZINC path: optionally sync the current source tree to the Intel node, build with zig build -Doptimize=ReleaseFast, then measure generation through the ZINC CLI.
Baseline path: launch llama.cpp against the same model file, preferring one reusable llama-server per model across the full scenario matrix and falling back to llama-cli when the server path is unavailable or fails to start.
Scenarios: Quick Chat, Coding Review, Incident Context, and Long Coding Draft. The prompts are real-world chat, code-review, support-context, and coding-plan workloads instead of synthetic factual completions.
Statistics: one warmup pass is discarded, then 3 measured runs are collected. Published prefill, decode, end-to-end throughput, and latency values are medians.
Execution order: the harness records the full ZINC scenario matrix before starting the llama.cpp baseline phase, so only one inference engine owns the GPU during measurement.
Run hygiene: published Intel runs assume a clean node with no competing ZINC, llama.cpp, or other GPU workloads.

Apple Silicon · Mac Studio

Metal backend

Apple M4 Max GPU · 64 GB · 3 measured · 1 warmup

Exact configuration

GPU: Apple M4 Max GPU
Chip: Apple M4 Max
Memory: 64 GB
OS: Darwin 24.6.0
ZINC backend: Metal
llama.cpp backend: Metal (-ngl 999)
Prompt parity: 4/20 rows read the same prompt tokens in both engines
Build: zig build -Doptimize=ReleaseFast
ZINC commit: 68a03ec6c641
llama.cpp build: 0.4.0-dev
Artifact: Sep 12, 2026, 8:37 PM

Methodology & reproducibility

Hardware: the local Apple Silicon target shown above. ZINC and llama.cpp run on the same machine and use the same GGUF file for each model.
ZINC path: build the current workspace with zig build -Doptimize=ReleaseFast, then measure generation through the ZINC CLI.
Baseline backend: llama.cpp Metal (-ngl 999) on the same Apple Silicon machine — the same backend as ZINC.
Baseline path: launch llama.cpp against the same model file, preferring one reusable llama-server per model across the full scenario matrix and falling back to llama-cli when the server path is unavailable.
Scenarios: Quick Chat, Coding Review, Incident Context, and Long Coding Draft. The prompts are real-world chat, code-review, support-context, and coding-plan workloads instead of synthetic factual completions.
Statistics: one warmup pass is discarded, then 3 measured runs are collected. Published prefill, decode, end-to-end throughput, and latency values are medians.
Model resolution: managed catalog models are loaded from the ZINC model cache when available; explicit GGUF paths are preserved when provided.
Run hygiene: published local runs assume no competing workload is saturating CPU, memory bandwidth, or GPU.

No speculative decoding on this backend. The ROCm and Vulkan tab has an MTP on/off switch for the models that support it.

Fastest 79.8 tok/s Qwen 3.6 35B A3B UD Q4K XL

Decode 51.5 tok/s average

Overall 83% prefill + decode vs llama.cpp

Choose a workload

Prefill and decode are shown separately below.

Decode 51.5 tok/s 95% decode ratio

Prefill 241.7 tok/s 121% prefill ratio · 46 prompt tok

Fastest Qwen 3.6 35B A3B UD Q4K XL 79.8 tok/s

Overall 97% best row: Qwen 3.5 9B Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 49p · 96out Compared

Prefill 80%

ZINC 303.6 tok/s

llama 379.6 tok/s

Decode 84%

ZINC 68.4 tok/s

llama 81.7 tok/s

84% overall

Gemma 4 31B Q4K M 49p · 96out Compared

Prefill 198%

ZINC 123.9 tok/s

llama 62.7 tok/s

Decode 93%

ZINC 20.8 tok/s

llama 22.4 tok/s

102% overall

Qwen 3.5 9B Q4K M 48p · 96out Compared

Prefill 179%

ZINC 546.3 tok/s

llama 305.9 tok/s

Decode 102%

ZINC 67.0 tok/s

llama 65.5 tok/s

106% overall

Qwen 3.6 35B A3B UD Q4K XL 36p · 96out Compared

Prefill 28%

ZINC 91.0 tok/s

llama 319.9 tok/s

Decode 105%

ZINC 79.8 tok/s

llama 76.4 tok/s

86% overall

Qwen 3.8 27B Dense Q4K M 48p · 96out Compared

Prefill 120%

ZINC 143.7 tok/s

llama 119.9 tok/s

Decode 89%

ZINC 21.3 tok/s

llama 23.9 tok/s

98% overall

Details

Gemma 4 26B-A4B MoE Q4K M 49 tok prompt · 96 tok output

Compared

Prefill 80%

ZINC 303.6 tok/s

llama 379.6 tok/s

Decode 84%

ZINC 68.4 tok/s

llama 81.7 tok/s

Overall 84%

ZINC 92.7 tok/s

llama 112.8 tok/s

Prompt+decode. Cold latency: ZINC 10384 ms · llama.cpp 1303 ms

Gemma 4 31B Q4K M 49 tok prompt · 96 tok output

Compared

Prefill 198%

ZINC 123.9 tok/s

llama 62.7 tok/s

Decode 93%

ZINC 20.8 tok/s

llama 22.4 tok/s

Overall 102%

ZINC 29.0 tok/s

llama 29.0 tok/s

Prompt+decode. Cold latency: ZINC 15160 ms · llama.cpp 5066 ms

Qwen 3.5 9B Q4K M 48 tok prompt · 96 tok output

Compared

Prefill 179%

ZINC 546.3 tok/s

llama 305.9 tok/s

Decode 102%

ZINC 67.0 tok/s

llama 65.5 tok/s

Overall 106%

ZINC 94.8 tok/s

llama 87.5 tok/s

Prompt+decode. Cold latency: ZINC 2162 ms · llama.cpp 1599 ms

Qwen 3.6 35B A3B UD Q4K XL 36 tok prompt · 96 tok output

Compared

Prefill 28%

ZINC 91.0 tok/s

llama 319.9 tok/s

Decode 105%

ZINC 79.8 tok/s

llama 76.4 tok/s

Overall 86%

ZINC 82.6 tok/s

llama 95.9 tok/s

Prompt+decode. Cold latency: ZINC 10176 ms · llama.cpp 1363 ms

Qwen 3.8 27B Dense Q4K M 48 tok prompt · 96 tok output

Compared

Prefill 120%

ZINC 143.7 tok/s

llama 119.9 tok/s

Decode 89%

ZINC 21.3 tok/s

llama 23.9 tok/s

Overall 98%

ZINC 29.8 tok/s

llama 38.6 tok/s

Prompt+decode. Cold latency: ZINC 11781 ms · llama.cpp 4703 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 49 tok prompt · 96 tok output · 3 runs · Compared	Z 303.6 tok/s 161.4 ms L 379.6 tok/s 137.0 ms 80% -76.0 tok/s	Z 68.4 tok/s 14.6 ms/tok L 81.7 tok/s 12.2 ms/tok 84% -13.2 tok/s	Z 92.7 tok/s prompt+decode L 112.8 tok/s prompt+decode 84% -20.1 tok/s 9080 ms slower
Gemma 4 31B Q4K M 49 tok prompt · 96 tok output · 3 runs · Compared	Z 123.9 tok/s 395.5 ms L 62.7 tok/s 829.2 ms 198% +61.2 tok/s	Z 20.8 tok/s 48.0 ms/tok L 22.4 tok/s 44.6 ms/tok 93% -1.6 tok/s	Z 29.0 tok/s prompt+decode L 29.0 tok/s prompt+decode 102% +0.0 tok/s 10094 ms slower
Qwen 3.5 9B Q4K M 48 tok prompt · 96 tok output · 3 runs · Compared	Z 546.3 tok/s 87.9 ms L 305.9 tok/s 147.1 ms 179% +240.4 tok/s	Z 67.0 tok/s 14.9 ms/tok L 65.5 tok/s 15.3 ms/tok 102% +1.5 tok/s	Z 94.8 tok/s prompt+decode L 87.5 tok/s prompt+decode 106% +7.3 tok/s 563 ms slower
Qwen 3.6 35B A3B UD Q4K XL 36 tok prompt · 96 tok output · 3 runs · Compared	Z 91.0 tok/s 395.5 ms L 319.9 tok/s 109.4 ms 28% -228.9 tok/s	Z 79.8 tok/s 12.5 ms/tok L 76.4 tok/s 13.1 ms/tok 105% +3.5 tok/s	Z 82.6 tok/s prompt+decode L 95.9 tok/s prompt+decode 86% -13.3 tok/s 8813 ms slower
Qwen 3.8 27B Dense Q4K M 48 tok prompt · 96 tok output · 3 runs · Compared	Z 143.7 tok/s 334.0 ms L 119.9 tok/s 725.7 ms 120% +23.8 tok/s	Z 21.3 tok/s 46.9 ms/tok L 23.9 tok/s 41.9 ms/tok 89% -2.6 tok/s	Z 29.8 tok/s prompt+decode L 38.6 tok/s prompt+decode 98% -8.8 tok/s 7078 ms slower

Decode 55.4 tok/s 91% decode ratio

Prefill 255.4 tok/s 62% prefill ratio · 188 prompt tok

Fastest Qwen 3.6 35B A3B UD Q4K XL 74.9 tok/s

Overall 77% best row: Qwen 3.8 27B Dense Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 192p · 160out Compared

Prefill 9%

ZINC 84.2 tok/s

llama 900.6 tok/s

Decode 75%

ZINC 60.4 tok/s

llama 80.5 tok/s

45% overall

Gemma 4 31B Q4K M 192p · 160out Compared

Prefill 133%

ZINC 183.8 tok/s

llama 138.6 tok/s

Decode 91%

ZINC 19.8 tok/s

llama 21.7 tok/s

96% overall

Qwen 3.5 9B Q4K M 193p · 160out Compared

Prefill 114%

ZINC 648.7 tok/s

llama 567.6 tok/s

Decode 96%

ZINC 62.9 tok/s

llama 65.8 tok/s

96% overall

Qwen 3.6 35B A3B UD Q4K XL 174p · 160out Compared

Prefill 11%

ZINC 96.1 tok/s

llama 870.7 tok/s

Decode 95%

ZINC 74.9 tok/s

llama 78.7 tok/s

56% overall

Qwen 3.8 27B Dense Q4K M 193p · 160out Compared

Prefill 113%

ZINC 192.4 tok/s

llama 170.2 tok/s

Decode 99%

ZINC 23.5 tok/s

llama 23.7 tok/s

102% overall

Details

Gemma 4 26B-A4B MoE Q4K M 192 tok prompt · 160 tok output

Compared

Prefill 9%

ZINC 84.2 tok/s

llama 900.6 tok/s

Decode 75%

ZINC 60.4 tok/s

llama 80.5 tok/s

Overall 45%

ZINC 71.4 tok/s

llama 161.1 tok/s

Prompt+decode. Cold latency: ZINC 13012 ms · llama.cpp 2193 ms

Gemma 4 31B Q4K M 192 tok prompt · 160 tok output

Compared

Prefill 133%

ZINC 183.8 tok/s

llama 138.6 tok/s

Decode 91%

ZINC 19.8 tok/s

llama 21.7 tok/s

Overall 96%

ZINC 38.6 tok/s

llama 40.4 tok/s

Prompt+decode. Cold latency: ZINC 19132 ms · llama.cpp 8768 ms

Qwen 3.5 9B Q4K M 193 tok prompt · 160 tok output

Compared

Prefill 114%

ZINC 648.7 tok/s

llama 567.6 tok/s

Decode 96%

ZINC 62.9 tok/s

llama 65.8 tok/s

Overall 96%

ZINC 124.2 tok/s

llama 120.8 tok/s

Prompt+decode. Cold latency: ZINC 3516 ms · llama.cpp 2719 ms

Qwen 3.6 35B A3B UD Q4K XL 174 tok prompt · 160 tok output

Compared

Prefill 11%

ZINC 96.1 tok/s

llama 870.7 tok/s

Decode 95%

ZINC 74.9 tok/s

llama 78.7 tok/s

Overall 56%

ZINC 84.6 tok/s

llama 141.7 tok/s

Prompt+decode. Cold latency: ZINC 13989 ms · llama.cpp 2197 ms

Qwen 3.8 27B Dense Q4K M 193 tok prompt · 160 tok output

Compared

Prefill 113%

ZINC 192.4 tok/s

llama 170.2 tok/s

Decode 99%

ZINC 23.5 tok/s

llama 23.7 tok/s

Overall 102%

ZINC 45.2 tok/s

llama 46.6 tok/s

Prompt+decode. Cold latency: ZINC 12470 ms · llama.cpp 7945 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 192 tok prompt · 160 tok output · 3 runs · Compared	Z 84.2 tok/s 2281.3 ms L 900.6 tok/s 216.5 ms 9% -816.4 tok/s	Z 60.4 tok/s 16.6 ms/tok L 80.5 tok/s 12.4 ms/tok 75% -20.1 tok/s	Z 71.4 tok/s prompt+decode L 161.1 tok/s prompt+decode 45% -89.7 tok/s 10819 ms slower
Gemma 4 31B Q4K M 192 tok prompt · 160 tok output · 3 runs · Compared	Z 183.8 tok/s 1044.7 ms L 138.6 tok/s 1406.9 ms 133% +45.2 tok/s	Z 19.8 tok/s 50.5 ms/tok L 21.7 tok/s 46.2 ms/tok 91% -1.8 tok/s	Z 38.6 tok/s prompt+decode L 40.4 tok/s prompt+decode 96% -1.8 tok/s 10364 ms slower
Qwen 3.5 9B Q4K M 193 tok prompt · 160 tok output · 3 runs · Compared	Z 648.7 tok/s 297.5 ms L 567.6 tok/s 299.5 ms 114% +81.1 tok/s	Z 62.9 tok/s 15.9 ms/tok L 65.8 tok/s 15.2 ms/tok 96% -2.9 tok/s	Z 124.2 tok/s prompt+decode L 120.8 tok/s prompt+decode 96% +3.4 tok/s 797 ms slower
Qwen 3.6 35B A3B UD Q4K XL 174 tok prompt · 160 tok output · 3 runs · Compared	Z 96.1 tok/s 1810.8 ms L 870.7 tok/s 175.7 ms 11% -774.6 tok/s	Z 74.9 tok/s 13.4 ms/tok L 78.7 tok/s 12.7 ms/tok 95% -3.8 tok/s	Z 84.6 tok/s prompt+decode L 141.7 tok/s prompt+decode 56% -57.1 tok/s 11791 ms slower
Qwen 3.8 27B Dense Q4K M 193 tok prompt · 160 tok output · 3 runs · Compared	Z 192.4 tok/s 1003.0 ms L 170.2 tok/s 1245.6 ms 113% +22.2 tok/s	Z 23.5 tok/s 42.6 ms/tok L 23.7 tok/s 42.1 ms/tok 99% -0.3 tok/s	Z 45.2 tok/s prompt+decode L 46.6 tok/s prompt+decode 102% -1.4 tok/s 4525 ms slower

Decode 54.7 tok/s 91% decode ratio

Prefill 272.9 tok/s 57% prefill ratio · 340 prompt tok

Fastest Qwen 3.6 35B A3B UD Q4K XL 71.7 tok/s

Overall 62% best row: Qwen 3.8 27B Dense Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 346p · 128out Compared

Prefill 7%

ZINC 74.1 tok/s

llama 1131.4 tok/s

Decode 78%

ZINC 63.6 tok/s

llama 81.5 tok/s

28% overall

Gemma 4 31B Q4K M 346p · 128out Compared

Prefill 114%

ZINC 181.6 tok/s

llama 158.6 tok/s

Decode 89%

ZINC 19.2 tok/s

llama 21.7 tok/s

94% overall

Qwen 3.5 9B Q4K M 345p · 128out Compared

Prefill 104%

ZINC 714.6 tok/s

llama 689.6 tok/s

Decode 93%

ZINC 60.4 tok/s

llama 65.2 tok/s

94% overall

Qwen 3.6 35B A3B UD Q4K XL 322p · 128out Compared

Prefill 7%

ZINC 85.0 tok/s

llama 1233.8 tok/s

Decode 91%

ZINC 71.7 tok/s

llama 79.0 tok/s

34% overall

Qwen 3.8 27B Dense Q4K M 345p · 128out Compared

Prefill 111%

ZINC 218.1 tok/s

llama 196.9 tok/s

Decode 101%

ZINC 23.1 tok/s

llama 22.8 tok/s

106% overall

Details

Gemma 4 26B-A4B MoE Q4K M 346 tok prompt · 128 tok output

Compared

Prefill 7%

ZINC 74.1 tok/s

llama 1131.4 tok/s

Decode 78%

ZINC 63.6 tok/s

llama 81.5 tok/s

Overall 28%

ZINC 70.9 tok/s

llama 252.3 tok/s

Prompt+decode. Cold latency: ZINC 14952 ms · llama.cpp 1864 ms

Gemma 4 31B Q4K M 346 tok prompt · 128 tok output

Compared

Prefill 114%

ZINC 181.6 tok/s

llama 158.6 tok/s

Decode 89%

ZINC 19.2 tok/s

llama 21.7 tok/s

Overall 94%

ZINC 55.4 tok/s

llama 58.6 tok/s

Prompt+decode. Cold latency: ZINC 18539 ms · llama.cpp 8018 ms

Qwen 3.5 9B Q4K M 345 tok prompt · 128 tok output

Compared

Prefill 104%

ZINC 714.6 tok/s

llama 689.6 tok/s

Decode 93%

ZINC 60.4 tok/s

llama 65.2 tok/s

Overall 94%

ZINC 181.9 tok/s

llama 189.3 tok/s

Prompt+decode. Cold latency: ZINC 3281 ms · llama.cpp 2437 ms

Qwen 3.6 35B A3B UD Q4K XL 322 tok prompt · 128 tok output

Compared

Prefill 7%

ZINC 85.0 tok/s

llama 1233.8 tok/s

Decode 91%

ZINC 71.7 tok/s

llama 79.0 tok/s

Overall 34%

ZINC 80.7 tok/s

llama 236.1 tok/s

Prompt+decode. Cold latency: ZINC 14403 ms · llama.cpp 1864 ms

Qwen 3.8 27B Dense Q4K M 345 tok prompt · 128 tok output

Compared

Prefill 111%

ZINC 218.1 tok/s

llama 196.9 tok/s

Decode 101%

ZINC 23.1 tok/s

llama 22.8 tok/s

Overall 106%

ZINC 66.3 tok/s

llama 67.1 tok/s

Prompt+decode. Cold latency: ZINC 11963 ms · llama.cpp 7489 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 346 tok prompt · 128 tok output · 3 runs · Compared	Z 74.1 tok/s 4667.7 ms L 1131.4 tok/s 304.9 ms 7% -1057.3 tok/s	Z 63.6 tok/s 15.7 ms/tok L 81.5 tok/s 12.3 ms/tok 78% -17.9 tok/s	Z 70.9 tok/s prompt+decode L 252.3 tok/s prompt+decode 28% -181.4 tok/s 13087 ms slower
Gemma 4 31B Q4K M 346 tok prompt · 128 tok output · 3 runs · Compared	Z 181.6 tok/s 1905.6 ms L 158.6 tok/s 2174.8 ms 114% +23.0 tok/s	Z 19.2 tok/s 52.0 ms/tok L 21.7 tok/s 46.0 ms/tok 89% -2.5 tok/s	Z 55.4 tok/s prompt+decode L 58.6 tok/s prompt+decode 94% -3.3 tok/s 10521 ms slower
Qwen 3.5 9B Q4K M 345 tok prompt · 128 tok output · 3 runs · Compared	Z 714.6 tok/s 482.8 ms L 689.6 tok/s 487.2 ms 104% +25.0 tok/s	Z 60.4 tok/s 16.5 ms/tok L 65.2 tok/s 15.3 ms/tok 93% -4.8 tok/s	Z 181.9 tok/s prompt+decode L 189.3 tok/s prompt+decode 94% -7.5 tok/s 844 ms slower
Qwen 3.6 35B A3B UD Q4K XL 322 tok prompt · 128 tok output · 3 runs · Compared	Z 85.0 tok/s 3787.9 ms L 1233.8 tok/s 255.3 ms 7% -1148.8 tok/s	Z 71.7 tok/s 14.0 ms/tok L 79.0 tok/s 12.7 ms/tok 91% -7.3 tok/s	Z 80.7 tok/s prompt+decode L 236.1 tok/s prompt+decode 34% -155.4 tok/s 12539 ms slower
Qwen 3.8 27B Dense Q4K M 345 tok prompt · 128 tok output · 3 runs · Compared	Z 218.1 tok/s 1581.6 ms L 196.9 tok/s 1920.2 ms 111% +21.2 tok/s	Z 23.1 tok/s 43.4 ms/tok L 22.8 tok/s 43.9 ms/tok 101% +0.3 tok/s	Z 66.3 tok/s prompt+decode L 67.1 tok/s prompt+decode 106% -0.8 tok/s 4474 ms slower

Decode 49.1 tok/s 92% decode ratio

Prefill 246.6 tok/s 88% prefill ratio · 69 prompt tok

Fastest Qwen 3.6 35B A3B UD Q4K XL 76.2 tok/s

Overall 91% best row: Qwen 3.8 27B Dense Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 70p · 256out Compared

Prefill 72%

ZINC 360.1 tok/s

llama 501.6 tok/s

Decode 81%

ZINC 63.2 tok/s

llama 77.9 tok/s

81% overall

Gemma 4 31B Q4K M 70p · 256out Compared

Prefill 140%

ZINC 126.6 tok/s

llama 90.2 tok/s

Decode 85%

ZINC 19.3 tok/s

llama 22.8 tok/s

87% overall

Qwen 3.5 9B Q4K M 70p · 256out Compared

Prefill 109%

ZINC 498.2 tok/s

llama 456.6 tok/s

Decode 96%

ZINC 63.1 tok/s

llama 65.7 tok/s

96% overall

Qwen 3.6 35B A3B UD Q4K XL 64p · 256out Compared

Prefill 18%

ZINC 101.3 tok/s

llama 578.6 tok/s

Decode 97%

ZINC 76.2 tok/s

llama 78.4 tok/s

85% overall

Qwen 3.8 27B Dense Q4K M 70p · 256out Compared

Prefill 99%

ZINC 147.0 tok/s

llama 148.7 tok/s

Decode 99%

ZINC 23.5 tok/s

llama 23.8 tok/s

101% overall

Details

Gemma 4 26B-A4B MoE Q4K M 70 tok prompt · 256 tok output

Compared

Prefill 72%

ZINC 360.1 tok/s

llama 501.6 tok/s

Decode 81%

ZINC 63.2 tok/s

llama 77.9 tok/s

Overall 81%

ZINC 76.7 tok/s

llama 95.9 tok/s

Prompt+decode. Cold latency: ZINC 12659 ms · llama.cpp 3423 ms

Gemma 4 31B Q4K M 70 tok prompt · 256 tok output

Compared

Prefill 140%

ZINC 126.6 tok/s

llama 90.2 tok/s

Decode 85%

ZINC 19.3 tok/s

llama 22.8 tok/s

Overall 87%

ZINC 23.6 tok/s

llama 27.3 tok/s

Prompt+decode. Cold latency: ZINC 23988 ms · llama.cpp 12001 ms

Qwen 3.5 9B Q4K M 70 tok prompt · 256 tok output

Compared

Prefill 109%

ZINC 498.2 tok/s

llama 456.6 tok/s

Decode 96%

ZINC 63.1 tok/s

llama 65.7 tok/s

Overall 96%

ZINC 77.7 tok/s

llama 79.9 tok/s

Prompt+decode. Cold latency: ZINC 4851 ms · llama.cpp 4028 ms

Qwen 3.6 35B A3B UD Q4K XL 64 tok prompt · 256 tok output

Compared

Prefill 18%

ZINC 101.3 tok/s

llama 578.6 tok/s

Decode 97%

ZINC 76.2 tok/s

llama 78.4 tok/s

Overall 85%

ZINC 80.2 tok/s

llama 94.6 tok/s

Prompt+decode. Cold latency: ZINC 12852 ms · llama.cpp 3361 ms

Qwen 3.8 27B Dense Q4K M 70 tok prompt · 256 tok output

Compared

Prefill 99%

ZINC 147.0 tok/s

llama 148.7 tok/s

Decode 99%

ZINC 23.5 tok/s

llama 23.8 tok/s

Overall 101%

ZINC 28.7 tok/s

llama 31.7 tok/s

Prompt+decode. Cold latency: ZINC 16664 ms · llama.cpp 11497 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 70 tok prompt · 256 tok output · 3 runs · Compared	Z 360.1 tok/s 194.4 ms L 501.6 tok/s 145.5 ms 72% -141.5 tok/s	Z 63.2 tok/s 15.8 ms/tok L 77.9 tok/s 12.8 ms/tok 81% -14.8 tok/s	Z 76.7 tok/s prompt+decode L 95.9 tok/s prompt+decode 81% -19.1 tok/s 9236 ms slower
Gemma 4 31B Q4K M 70 tok prompt · 256 tok output · 3 runs · Compared	Z 126.6 tok/s 553.0 ms L 90.2 tok/s 809.5 ms 140% +36.4 tok/s	Z 19.3 tok/s 51.8 ms/tok L 22.8 tok/s 43.9 ms/tok 85% -3.5 tok/s	Z 23.6 tok/s prompt+decode L 27.3 tok/s prompt+decode 87% -3.7 tok/s 11986 ms slower
Qwen 3.5 9B Q4K M 70 tok prompt · 256 tok output · 3 runs · Compared	Z 498.2 tok/s 140.5 ms L 456.6 tok/s 146.7 ms 109% +41.6 tok/s	Z 63.1 tok/s 15.8 ms/tok L 65.7 tok/s 15.2 ms/tok 96% -2.6 tok/s	Z 77.7 tok/s prompt+decode L 79.9 tok/s prompt+decode 96% -2.2 tok/s 823 ms slower
Qwen 3.6 35B A3B UD Q4K XL 64 tok prompt · 256 tok output · 3 runs · Compared	Z 101.3 tok/s 632.0 ms L 578.6 tok/s 108.9 ms 18% -477.3 tok/s	Z 76.2 tok/s 13.1 ms/tok L 78.4 tok/s 12.8 ms/tok 97% -2.2 tok/s	Z 80.2 tok/s prompt+decode L 94.6 tok/s prompt+decode 85% -14.4 tok/s 9490 ms slower
Qwen 3.8 27B Dense Q4K M 70 tok prompt · 256 tok output · 3 runs · Compared	Z 147.0 tok/s 476.1 ms L 148.7 tok/s 732.9 ms 99% -1.7 tok/s	Z 23.5 tok/s 42.6 ms/tok L 23.8 tok/s 42.1 ms/tok 99% -0.3 tok/s	Z 28.7 tok/s prompt+decode L 31.7 tok/s prompt+decode 101% -3.1 tok/s 5167 ms slower

Methodology ZINC CLI vs llama.cpp on the same Apple Silicon machine · Four-scenario matrix with same-file baselines

3 measured · 1 warmup

Benchmark runner Exact prompts and token caps Prompt matrix test Published JSON

Hardware: the local Apple Silicon target shown above. ZINC and llama.cpp run on the same machine and use the same GGUF file for each model.
ZINC path: build the current workspace with zig build -Doptimize=ReleaseFast, then measure generation through the ZINC CLI.
Baseline backend: llama.cpp Metal (-ngl 999) on the same Apple Silicon machine — the same backend as ZINC.
Baseline path: launch llama.cpp against the same model file, preferring one reusable llama-server per model across the full scenario matrix and falling back to llama-cli when the server path is unavailable.
Scenarios: Quick Chat, Coding Review, Incident Context, and Long Coding Draft. The prompts are real-world chat, code-review, support-context, and coding-plan workloads instead of synthetic factual completions.
Statistics: one warmup pass is discarded, then 3 measured runs are collected. Published prefill, decode, end-to-end throughput, and latency values are medians.
Model resolution: managed catalog models are loaded from the ZINC model cache when available; explicit GGUF paths are preserved when provided.
Run hygiene: published local runs assume no competing workload is saturating CPU, memory bandwidth, or GPU.

NVIDIA RTX · NVIDIA CUDA bench node

CUDA backend

GeForce RTX 5090 · 32 GB VRAM · 1792 GB/s · 1–3 measured · 1 warmup

Exact configuration

GPU: GeForce RTX 5090
Chip: NVIDIA GeForce RTX 5090
Memory: 32 GB VRAM · 1792 GB/s
OS: Ubuntu (WSL2)
ZINC backend: CUDA (-Dbackend=cuda, NVRTC)
llama.cpp backend: CUDA (CUDA_VISIBLE_DEVICES (UUID-pinned))
Prompt parity: 6/16 rows read the same prompt tokens in both engines
Build: zig build -Doptimize=ReleaseFast -Dbackend=cuda
ZINC commit: 9766de66
llama.cpp build: 1
Artifact: Jul 3, 2026, 3:30 PM

Methodology & reproducibility

Hardware: one NVIDIA GeForce RTX 5090 (32 GB VRAM, 1792 GB/s) benchmark node. ZINC and llama.cpp run on the same host and use the same GGUF file for each model.
ZINC path: sync the current source tree to the CUDA node, build with zig build -Doptimize=ReleaseFast -Dbackend=cuda, then measure generation through the ZINC CLI. CUDA kernels compiled via nvrtcGetCUBIN (NVRTC internal ptxas) — +30% over PTX+JIT for the visible GPU.
Baseline backend: llama.cpp CUDA on the same host; the target GPU is pinned by UUID via CUDA_VISIBLE_DEVICES for both engines.
GPU selection: the target GPU is pinned by UUID via CUDA_VISIBLE_DEVICES for both engines, because nvidia-smi indices are unreliable on the WSL2 passthrough.
Baseline path: launch llama.cpp against the same model file, preferring one reusable llama-server per model across the full scenario matrix and falling back to llama-cli when the server path is unavailable.
Scenarios: Quick Chat, Coding Review, Incident Context, and Long Coding Draft. The prompts are real-world chat, code-review, support-context, and coding-plan workloads instead of synthetic factual completions.
Statistics: one warmup pass is discarded for every row, then 1–3 measured runs are collected depending on the model. Published prefill, decode, end-to-end throughput, and latency values are medians.
Execution order: the harness records the full ZINC scenario matrix before starting the llama.cpp baseline phase, so only one inference engine owns the GPU during measurement.
Run hygiene: published CUDA runs assume a clean node with no competing ZINC, llama.cpp, or other GPU workloads.

No speculative decoding on this backend. The ROCm and Vulkan tab has an MTP on/off switch for the models that support it.

Fastest 154.1 tok/s Qwen 3.5 9B Q4K M

Decode 101.3 tok/s average

Overall 71% prefill + decode vs llama.cpp

Choose a workload

Prefill and decode are shown separately below.

Decode 101.3 tok/s 80% decode ratio

Prefill 247.1 tok/s 27% prefill ratio · 43 prompt tok

Fastest Qwen 3.5 9B Q4K M 154.1 tok/s

Overall 63% best row: Qwen 3.5 9B Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 49p · 96out Compared

Prefill 4%

ZINC 343.0 tok/s

llama 7978.0 tok/s

Decode 53%

ZINC 81.0 tok/s

llama 154.2 tok/s

47% overall

Gemma 4 31B Q4K M 49p · 96out Compared

Prefill 17%

ZINC 517.0 tok/s

llama 3096.0 tok/s

Decode 72%

ZINC 41.0 tok/s

llama 56.9 tok/s

70% overall

Qwen 3.5 9B Q4K M 36p · 96out Compared

Prefill 87%

ZINC 86.9 tok/s

llama 99.7 tok/s

Decode 93%

ZINC 154.1 tok/s

llama 165.9 tok/s

90% overall

Qwen 3.6 35B A3B UD Q4K XL 36p · 96out Compared

Prefill 1%

ZINC 41.4 tok/s

llama 6535.0 tok/s

Decode 103%

ZINC 129.0 tok/s

llama 125.8 tok/s

48% overall

Details

Gemma 4 26B-A4B MoE Q4K M 49 tok prompt · 96 tok output

Compared

Prefill 4%

ZINC 343.0 tok/s

llama 7978.0 tok/s

Decode 53%

ZINC 81.0 tok/s

llama 154.2 tok/s

Overall 47%

ZINC 109.2 tok/s

llama 235.3 tok/s

Prompt+decode. Cold latency: ZINC 20444 ms · llama.cpp 2125 ms

Gemma 4 31B Q4K M 49 tok prompt · 96 tok output

Compared

Prefill 17%

ZINC 517.0 tok/s

llama 3096.0 tok/s

Decode 72%

ZINC 41.0 tok/s

llama 56.9 tok/s

Overall 70%

ZINC 59.5 tok/s

llama 86.9 tok/s

Prompt+decode. Cold latency: ZINC 25191 ms · llama.cpp 3162 ms

Qwen 3.5 9B Q4K M 36 tok prompt · 96 tok output

Compared

Prefill 87%

ZINC 86.9 tok/s

llama 99.7 tok/s

Decode 93%

ZINC 154.1 tok/s

llama 165.9 tok/s

Overall 90%

ZINC 127.2 tok/s

llama 140.9 tok/s

Prompt+decode. Cold latency: ZINC 53708 ms · llama.cpp 1884 ms

Qwen 3.6 35B A3B UD Q4K XL 36 tok prompt · 96 tok output

Compared

Prefill 1%

ZINC 41.4 tok/s

llama 6535.0 tok/s

Decode 103%

ZINC 129.0 tok/s

llama 125.8 tok/s

Overall 48%

ZINC 81.8 tok/s

llama 170.5 tok/s

Prompt+decode. Cold latency: ZINC 15000 ms · llama.cpp 2163 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 49 tok prompt · 96 tok output · 1 runs · Compared	Z 343.0 tok/s 1492.7 ms L 7978.0 tok/s 64.2 ms 4% -7635.0 tok/s	Z 81.0 tok/s 21.0 ms/tok L 154.2 tok/s 6.5 ms/tok 53% -73.2 tok/s	Z 109.2 tok/s prompt+decode L 235.3 tok/s prompt+decode 47% -126.1 tok/s 18319 ms slower
Gemma 4 31B Q4K M 49 tok prompt · 96 tok output · 1 runs · Compared	Z 517.0 tok/s 990.3 ms L 3096.0 tok/s 165.4 ms 17% -2579.0 tok/s	Z 41.0 tok/s 21.3 ms/tok L 56.9 tok/s 17.6 ms/tok 72% -15.9 tok/s	Z 59.5 tok/s prompt+decode L 86.9 tok/s prompt+decode 70% -27.3 tok/s 22030 ms slower
Qwen 3.5 9B Q4K M 36 tok prompt · 96 tok output · 3 runs · Compared	Z 86.9 tok/s 414.4 ms L 99.7 tok/s 351.1 ms 87% -12.8 tok/s	Z 154.1 tok/s 6.5 ms/tok L 165.9 tok/s 6.0 ms/tok 93% -11.8 tok/s	Z 127.2 tok/s prompt+decode L 140.9 tok/s prompt+decode 90% -13.6 tok/s 51824 ms slower
Qwen 3.6 35B A3B UD Q4K XL 36 tok prompt · 96 tok output · 1 runs · Compared	Z 41.4 tok/s 12367.1 ms L 6535.0 tok/s 78.3 ms 1% -6493.6 tok/s	Z 129.0 tok/s 18.9 ms/tok L 125.8 tok/s 7.9 ms/tok 103% +3.2 tok/s	Z 81.8 tok/s prompt+decode L 170.5 tok/s prompt+decode 48% -88.7 tok/s 12837 ms slower

Decode 99.8 tok/s 83% decode ratio

Prefill 299.0 tok/s 125% prefill ratio · 183 prompt tok

Fastest Qwen 3.5 9B Q4K M 148.1 tok/s

Overall 67% best row: Qwen 3.5 9B Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 192p · 160out Compared

Prefill 4%

ZINC 343.0 tok/s

llama 7978.0 tok/s

Decode 75%

ZINC 81.0 tok/s

llama 107.9 tok/s

59% overall

Gemma 4 31B Q4K M 192p · 160out Compared

Prefill 17%

ZINC 517.0 tok/s

llama 3096.0 tok/s

Decode 82%

ZINC 41.0 tok/s

llama 49.9 tok/s

77% overall

Qwen 3.5 9B Q4K M 174p · 160out Compared

Prefill 480%

ZINC 294.6 tok/s

llama 61.4 tok/s

Decode 97%

ZINC 148.1 tok/s

llama 152.5 tok/s

212% overall

Qwen 3.6 35B A3B UD Q4K XL 174p · 160out Compared

Prefill 1%

ZINC 41.4 tok/s

llama 6535.0 tok/s

Decode 79%

ZINC 129.0 tok/s

llama 164.0 tok/s

18% overall

Details

Gemma 4 26B-A4B MoE Q4K M 192 tok prompt · 160 tok output

Compared

Prefill 4%

ZINC 343.0 tok/s

llama 7978.0 tok/s

Decode 75%

ZINC 81.0 tok/s

llama 107.9 tok/s

Overall 59%

ZINC 138.9 tok/s

llama 235.4 tok/s

Prompt+decode. Cold latency: ZINC 17311 ms · llama.cpp 2965 ms

Gemma 4 31B Q4K M 192 tok prompt · 160 tok output

Compared

Prefill 17%

ZINC 517.0 tok/s

llama 3096.0 tok/s

Decode 82%

ZINC 41.0 tok/s

llama 49.9 tok/s

Overall 77%

ZINC 82.4 tok/s

llama 108.5 tok/s

Prompt+decode. Cold latency: ZINC 27471 ms · llama.cpp 4616 ms

Qwen 3.5 9B Q4K M 174 tok prompt · 160 tok output

Compared

Prefill 480%

ZINC 294.6 tok/s

llama 61.4 tok/s

Decode 97%

ZINC 148.1 tok/s

llama 152.5 tok/s

Overall 212%

ZINC 199.9 tok/s

llama 88.3 tok/s

Prompt+decode. Cold latency: ZINC 54159 ms · llama.cpp 2433 ms

Qwen 3.6 35B A3B UD Q4K XL 174 tok prompt · 160 tok output

Compared

Prefill 1%

ZINC 41.4 tok/s

llama 6535.0 tok/s

Decode 79%

ZINC 129.0 tok/s

llama 164.0 tok/s

Overall 18%

ZINC 61.4 tok/s

llama 313.3 tok/s

Prompt+decode. Cold latency: ZINC 28559 ms · llama.cpp 2378 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 192 tok prompt · 160 tok output · 1 runs · Compared	Z 343.0 tok/s 1492.7 ms L 7978.0 tok/s 64.2 ms 4% -7635.0 tok/s	Z 81.0 tok/s 18.6 ms/tok L 107.9 tok/s 9.3 ms/tok 75% -26.9 tok/s	Z 138.9 tok/s prompt+decode L 235.4 tok/s prompt+decode 59% -96.6 tok/s 14346 ms slower
Gemma 4 31B Q4K M 192 tok prompt · 160 tok output · 1 runs · Compared	Z 517.0 tok/s 990.3 ms L 3096.0 tok/s 165.4 ms 17% -2579.0 tok/s	Z 41.0 tok/s 23.1 ms/tok L 49.9 tok/s 20.1 ms/tok 82% -8.9 tok/s	Z 82.4 tok/s prompt+decode L 108.5 tok/s prompt+decode 77% -26.2 tok/s 22855 ms slower
Qwen 3.5 9B Q4K M 174 tok prompt · 160 tok output · 3 runs · Compared	Z 294.6 tok/s 590.6 ms L 61.4 tok/s 2493.6 ms 480% +233.3 tok/s	Z 148.1 tok/s 6.8 ms/tok L 152.5 tok/s 6.6 ms/tok 97% -4.4 tok/s	Z 199.9 tok/s prompt+decode L 88.3 tok/s prompt+decode 212% +111.5 tok/s 51727 ms slower
Qwen 3.6 35B A3B UD Q4K XL 174 tok prompt · 160 tok output · 1 runs · Compared	Z 41.4 tok/s 12367.1 ms L 6535.0 tok/s 78.3 ms 1% -6493.6 tok/s	Z 129.0 tok/s 24.3 ms/tok L 164.0 tok/s 6.1 ms/tok 79% -35.0 tok/s	Z 61.4 tok/s prompt+decode L 313.3 tok/s prompt+decode 18% -251.9 tok/s 26181 ms slower

Decode 99.3 tok/s 90% decode ratio

Prefill 457.4 tok/s 416% prefill ratio · 334 prompt tok

Fastest Qwen 3.5 9B Q4K M 146.1 tok/s

Overall 69% best row: Qwen 3.5 9B Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 346p · 128out Compared

Prefill 4%

ZINC 343.0 tok/s

llama 7978.0 tok/s

Decode 78%

ZINC 81.0 tok/s

llama 104.5 tok/s

49% overall

Gemma 4 31B Q4K M 346p · 128out Compared

Prefill 17%

ZINC 517.0 tok/s

llama 3096.0 tok/s

Decode 80%

ZINC 41.0 tok/s

llama 51.1 tok/s

69% overall

Qwen 3.5 9B Q4K M 322p · 128out Compared

Prefill 1643%

ZINC 928.2 tok/s

llama 56.5 tok/s

Decode 128%

ZINC 146.1 tok/s

llama 114.5 tok/s

547% overall

Qwen 3.6 35B A3B UD Q4K XL 322p · 128out Compared

Prefill 1%

ZINC 41.4 tok/s

llama 6535.0 tok/s

Decode 76%

ZINC 129.0 tok/s

llama 170.3 tok/s

9% overall

Details

Gemma 4 26B-A4B MoE Q4K M 346 tok prompt · 128 tok output

Compared

Prefill 4%

ZINC 343.0 tok/s

llama 7978.0 tok/s

Decode 78%

ZINC 81.0 tok/s

llama 104.5 tok/s

Overall 49%

ZINC 183.1 tok/s

llama 372.9 tok/s

Prompt+decode. Cold latency: ZINC 17653 ms · llama.cpp 2669 ms

Gemma 4 31B Q4K M 346 tok prompt · 128 tok output

Compared

Prefill 17%

ZINC 517.0 tok/s

llama 3096.0 tok/s

Decode 80%

ZINC 41.0 tok/s

llama 51.1 tok/s

Overall 69%

ZINC 125.0 tok/s

llama 180.7 tok/s

Prompt+decode. Cold latency: ZINC 19227 ms · llama.cpp 3975 ms

Qwen 3.5 9B Q4K M 322 tok prompt · 128 tok output

Compared

Prefill 1643%

ZINC 928.2 tok/s

llama 56.5 tok/s

Decode 128%

ZINC 146.1 tok/s

llama 114.5 tok/s

Overall 547%

ZINC 367.9 tok/s

llama 66.2 tok/s

Prompt+decode. Cold latency: ZINC 54684 ms · llama.cpp 2464 ms

Qwen 3.6 35B A3B UD Q4K XL 322 tok prompt · 128 tok output

Compared

Prefill 1%

ZINC 41.4 tok/s

llama 6535.0 tok/s

Decode 76%

ZINC 129.0 tok/s

llama 170.3 tok/s

Overall 9%

ZINC 51.3 tok/s

llama 553.9 tok/s

Prompt+decode. Cold latency: ZINC 19657 ms · llama.cpp 2093 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 346 tok prompt · 128 tok output · 1 runs · Compared	Z 343.0 tok/s 1492.7 ms L 7978.0 tok/s 64.2 ms 4% -7635.0 tok/s	Z 81.0 tok/s 19.2 ms/tok L 104.5 tok/s 9.6 ms/tok 78% -23.5 tok/s	Z 183.1 tok/s prompt+decode L 372.9 tok/s prompt+decode 49% -189.8 tok/s 14984 ms slower
Gemma 4 31B Q4K M 346 tok prompt · 128 tok output · 1 runs · Compared	Z 517.0 tok/s 990.3 ms L 3096.0 tok/s 165.4 ms 17% -2579.0 tok/s	Z 41.0 tok/s 23.7 ms/tok L 51.1 tok/s 19.6 ms/tok 80% -10.1 tok/s	Z 125.0 tok/s prompt+decode L 180.7 tok/s prompt+decode 69% -55.6 tok/s 15252 ms slower
Qwen 3.5 9B Q4K M 322 tok prompt · 128 tok output · 3 runs · Compared	Z 928.2 tok/s 346.9 ms L 56.5 tok/s 5574.0 ms 1643% +871.7 tok/s	Z 146.1 tok/s 6.8 ms/tok L 114.5 tok/s 8.7 ms/tok 128% +31.6 tok/s	Z 367.9 tok/s prompt+decode L 66.2 tok/s prompt+decode 547% +301.7 tok/s 52220 ms slower
Qwen 3.6 35B A3B UD Q4K XL 322 tok prompt · 128 tok output · 1 runs · Compared	Z 41.4 tok/s 12367.1 ms L 6535.0 tok/s 78.3 ms 1% -6493.6 tok/s	Z 129.0 tok/s 17.4 ms/tok L 170.3 tok/s 5.9 ms/tok 76% -41.3 tok/s	Z 51.3 tok/s prompt+decode L 553.9 tok/s prompt+decode 9% -502.5 tok/s 17564 ms slower

Decode 100.3 tok/s 91% decode ratio

Prefill 266.5 tok/s 83% prefill ratio · 67 prompt tok

Fastest Qwen 3.5 9B Q4K M 150.3 tok/s

Overall 81% best row: Qwen 3.5 9B Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 70p · 256out Compared

Prefill 4%

ZINC 343.0 tok/s

llama 7978.0 tok/s

Decode 77%

ZINC 81.0 tok/s

llama 105.7 tok/s

72% overall

Gemma 4 31B Q4K M 70p · 256out Compared

Prefill 17%

ZINC 517.0 tok/s

llama 3096.0 tok/s

Decode 80%

ZINC 41.0 tok/s

llama 51.4 tok/s

78% overall

Qwen 3.5 9B Q4K M 64p · 256out Compared

Prefill 310%

ZINC 164.7 tok/s

llama 53.1 tok/s

Decode 133%

ZINC 150.3 tok/s

llama 113.1 tok/s

165% overall

Qwen 3.6 35B A3B UD Q4K XL 64p · 256out Compared

Prefill 1%

ZINC 41.4 tok/s

llama 6535.0 tok/s

Decode 77%

ZINC 129.0 tok/s

llama 168.2 tok/s

43% overall

Details

Gemma 4 26B-A4B MoE Q4K M 70 tok prompt · 256 tok output

Compared

Prefill 4%

ZINC 343.0 tok/s

llama 7978.0 tok/s

Decode 77%

ZINC 81.0 tok/s

llama 105.7 tok/s

Overall 72%

ZINC 96.9 tok/s

llama 135.3 tok/s

Prompt+decode. Cold latency: ZINC 18234 ms · llama.cpp 3991 ms

Gemma 4 31B Q4K M 70 tok prompt · 256 tok output

Compared

Prefill 17%

ZINC 517.0 tok/s

llama 3096.0 tok/s

Decode 80%

ZINC 41.0 tok/s

llama 51.4 tok/s

Overall 78%

ZINC 51.1 tok/s

llama 65.8 tok/s

Prompt+decode. Cold latency: ZINC 19929 ms · llama.cpp 6660 ms

Qwen 3.5 9B Q4K M 64 tok prompt · 256 tok output

Compared

Prefill 310%

ZINC 164.7 tok/s

llama 53.1 tok/s

Decode 133%

ZINC 150.3 tok/s

llama 113.1 tok/s

Overall 165%

ZINC 153.0 tok/s

llama 92.5 tok/s

Prompt+decode. Cold latency: ZINC 54602 ms · llama.cpp 3685 ms

Qwen 3.6 35B A3B UD Q4K XL 64 tok prompt · 256 tok output

Compared

Prefill 1%

ZINC 41.4 tok/s

llama 6535.0 tok/s

Decode 77%

ZINC 129.0 tok/s

llama 168.2 tok/s

Overall 43%

ZINC 90.6 tok/s

llama 208.3 tok/s

Prompt+decode. Cold latency: ZINC 29140 ms · llama.cpp 2899 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 70 tok prompt · 256 tok output · 1 runs · Compared	Z 343.0 tok/s 1492.7 ms L 7978.0 tok/s 64.2 ms 4% -7635.0 tok/s	Z 81.0 tok/s 17.4 ms/tok L 105.7 tok/s 9.5 ms/tok 77% -24.7 tok/s	Z 96.9 tok/s prompt+decode L 135.3 tok/s prompt+decode 72% -38.4 tok/s 14242 ms slower
Gemma 4 31B Q4K M 70 tok prompt · 256 tok output · 1 runs · Compared	Z 517.0 tok/s 990.3 ms L 3096.0 tok/s 165.4 ms 17% -2579.0 tok/s	Z 41.0 tok/s 21.6 ms/tok L 51.4 tok/s 19.4 ms/tok 80% -10.4 tok/s	Z 51.1 tok/s prompt+decode L 65.8 tok/s prompt+decode 78% -14.7 tok/s 13269 ms slower
Qwen 3.5 9B Q4K M 64 tok prompt · 256 tok output · 3 runs · Compared	Z 164.7 tok/s 388.6 ms L 53.1 tok/s 1186.3 ms 310% +111.6 tok/s	Z 150.3 tok/s 6.7 ms/tok L 113.1 tok/s 8.8 ms/tok 133% +37.2 tok/s	Z 153.0 tok/s prompt+decode L 92.5 tok/s prompt+decode 165% +60.5 tok/s 50918 ms slower
Qwen 3.6 35B A3B UD Q4K XL 64 tok prompt · 256 tok output · 1 runs · Compared	Z 41.4 tok/s 12367.1 ms L 6535.0 tok/s 78.3 ms 1% -6493.6 tok/s	Z 129.0 tok/s 26.0 ms/tok L 168.2 tok/s 5.9 ms/tok 77% -39.2 tok/s	Z 90.6 tok/s prompt+decode L 208.3 tok/s prompt+decode 43% -117.7 tok/s 26241 ms slower

Methodology ZINC CLI vs llama.cpp on the same CUDA node · Four-scenario matrix with same-file baselines

1–3 measured · 1 warmup

Benchmark runner Exact prompts and token caps Prompt matrix test Published JSON

Hardware: one NVIDIA GeForce RTX 5090 (32 GB VRAM, 1792 GB/s) benchmark node. ZINC and llama.cpp run on the same host and use the same GGUF file for each model.
ZINC path: sync the current source tree to the CUDA node, build with zig build -Doptimize=ReleaseFast -Dbackend=cuda, then measure generation through the ZINC CLI. CUDA kernels compiled via nvrtcGetCUBIN (NVRTC internal ptxas) — +30% over PTX+JIT for the visible GPU.
Baseline backend: llama.cpp CUDA on the same host; the target GPU is pinned by UUID via CUDA_VISIBLE_DEVICES for both engines.
GPU selection: the target GPU is pinned by UUID via CUDA_VISIBLE_DEVICES for both engines, because nvidia-smi indices are unreliable on the WSL2 passthrough.
Baseline path: launch llama.cpp against the same model file, preferring one reusable llama-server per model across the full scenario matrix and falling back to llama-cli when the server path is unavailable.
Scenarios: Quick Chat, Coding Review, Incident Context, and Long Coding Draft. The prompts are real-world chat, code-review, support-context, and coding-plan workloads instead of synthetic factual completions.
Statistics: one warmup pass is discarded for every row, then 1–3 measured runs are collected depending on the model. Published prefill, decode, end-to-end throughput, and latency values are medians.
Execution order: the harness records the full ZINC scenario matrix before starting the llama.cpp baseline phase, so only one inference engine owns the GPU during measurement.
Run hygiene: published CUDA runs assume a clean node with no competing ZINC, llama.cpp, or other GPU workloads.

Benchmarks

Quick Chat

Coding Review

Incident Context

Long Coding Draft

Quick Chat

Coding Review

Incident Context

Long Coding Draft

Quick Chat

Coding Review

Incident Context

Long Coding Draft

Quick Chat

Coding Review

Incident Context

Long Coding Draft

Quick Chat

Coding Review

Incident Context

Long Coding Draft