ZINC

Benchmarks

ZINC vs llama.cpp on the same machines.

Targets 4

Unique Models 5

Comparisons 19

Updated Jun 12

AMD RDNA bench node

RDNA

Radeon AI PRO R9700 · 32 GB VRAM · 576 GB/s · 3 measured · 1 warmup

Run info

GPU: Radeon AI PRO R9700
Chip: AMD Radeon AI PRO R9700
Memory: 32 GB VRAM · 576 GB/s
Artifact: Jun 6, 2026, 7:33 PM
ZINC: e1551bff0d33
llama.cpp: 513

Fastest 109.3 tok/s Qwen 3.6 35B A3B UD Q4K XL

Decode 69.5 tok/s average

Overall 108% prefill + decode vs llama.cpp

Decode 69.5 tok/s 96% decode ratio

Prefill 133.5 tok/s 35% prefill ratio · 41 prompt tok

Fastest Qwen 3.6 35B A3B UD Q4K XL 109.3 tok/s

Overall 82% best row: Qwen 3.6 35B A3B UD Q4K XL

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 49p · 96out Compared

Prefill 18%

ZINC 89.1 tok/s

llama 502.2 tok/s

Decode 89%

ZINC 91.0 tok/s

llama 102.3 tok/s

65% overall

Gemma 4 31B Q4K M 49p · 96out Compared

Prefill 21%

ZINC 41.8 tok/s

llama 202.6 tok/s

Decode 86%

ZINC 24.6 tok/s

llama 28.6 tok/s

71% overall

Qwen 3.5 9B Q4K M 36p · 96out Compared

Prefill 32%

ZINC 176.0 tok/s

llama 554.2 tok/s

Decode 109%

ZINC 93.0 tok/s

llama 85.6 tok/s

96% overall

Qwen 3.6 27B Dense Q4K M 36p · 96out Compared

Prefill 26%

ZINC 49.1 tok/s

llama 185.9 tok/s

Decode 95%

ZINC 29.4 tok/s

llama 30.8 tok/s

83% overall

Qwen 3.6 35B A3B UD Q4K XL 36p · 96out Compared

Prefill 77%

ZINC 311.3 tok/s

llama 404.7 tok/s

Decode 100%

ZINC 109.3 tok/s

llama 109.0 tok/s

98% overall

Details

Gemma 4 26B-A4B MoE Q4K M 49 tok prompt · 96 tok output

Compared

Prefill 18%

ZINC 89.1 tok/s

llama 502.2 tok/s

Decode 89%

ZINC 91.0 tok/s

llama 102.3 tok/s

Overall 65%

ZINC 90.4 tok/s

llama 139.9 tok/s

Prompt+decode. Cold latency: ZINC 9394 ms · llama.cpp 1165 ms

Gemma 4 31B Q4K M 49 tok prompt · 96 tok output

Compared

Prefill 21%

ZINC 41.8 tok/s

llama 202.6 tok/s

Decode 86%

ZINC 24.6 tok/s

llama 28.6 tok/s

Overall 71%

ZINC 28.6 tok/s

llama 40.3 tok/s

Prompt+decode. Cold latency: ZINC 10859 ms · llama.cpp 3728 ms

Qwen 3.5 9B Q4K M 36 tok prompt · 96 tok output

Compared

Prefill 32%

ZINC 176.0 tok/s

llama 554.2 tok/s

Decode 109%

ZINC 93.0 tok/s

llama 85.6 tok/s

Overall 96%

ZINC 106.7 tok/s

llama 111.2 tok/s

Prompt+decode. Cold latency: ZINC 3862 ms · llama.cpp 1302 ms

Qwen 3.6 27B Dense Q4K M 36 tok prompt · 96 tok output

Compared

Prefill 26%

ZINC 49.1 tok/s

llama 185.9 tok/s

Decode 95%

ZINC 29.4 tok/s

llama 30.8 tok/s

Overall 83%

ZINC 33.0 tok/s

llama 39.8 tok/s

Prompt+decode. Cold latency: ZINC 9739 ms · llama.cpp 3454 ms

Qwen 3.6 35B A3B UD Q4K XL 36 tok prompt · 96 tok output

Compared

Prefill 77%

ZINC 311.3 tok/s

llama 404.7 tok/s

Decode 100%

ZINC 109.3 tok/s

llama 109.0 tok/s

Overall 98%

ZINC 132.8 tok/s

llama 136.1 tok/s

Prompt+decode. Cold latency: ZINC 8129 ms · llama.cpp 1106 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 49 tok prompt · 96 tok output · 3 runs · Compared	Z 89.1 tok/s 550.0 ms L 502.2 tok/s 101.6 ms 18% -413.1 tok/s	Z 91.0 tok/s 11.0 ms/tok L 102.3 tok/s 9.8 ms/tok 89% -11.3 tok/s	Z 90.4 tok/s prompt+decode L 139.9 tok/s prompt+decode 65% -49.6 tok/s 8228 ms slower
Gemma 4 31B Q4K M 49 tok prompt · 96 tok output · 3 runs · Compared	Z 41.8 tok/s 1171.9 ms L 202.6 tok/s 251.7 ms 21% -160.8 tok/s	Z 24.6 tok/s 40.6 ms/tok L 28.6 tok/s 34.9 ms/tok 86% -4.0 tok/s	Z 28.6 tok/s prompt+decode L 40.3 tok/s prompt+decode 71% -11.7 tok/s 7131 ms slower
Qwen 3.5 9B Q4K M 36 tok prompt · 96 tok output · 3 runs · Compared	Z 176.0 tok/s 204.6 ms L 554.2 tok/s 63.2 ms 32% -378.2 tok/s	Z 93.0 tok/s 10.8 ms/tok L 85.6 tok/s 11.7 ms/tok 109% +7.4 tok/s	Z 106.7 tok/s prompt+decode L 111.2 tok/s prompt+decode 96% -4.5 tok/s 2560 ms slower
Qwen 3.6 27B Dense Q4K M 36 tok prompt · 96 tok output · 3 runs · Compared	Z 49.1 tok/s 733.5 ms L 185.9 tok/s 188.3 ms 26% -136.8 tok/s	Z 29.4 tok/s 34.1 ms/tok L 30.8 tok/s 32.5 ms/tok 95% -1.4 tok/s	Z 33.0 tok/s prompt+decode L 39.8 tok/s prompt+decode 83% -6.9 tok/s 6286 ms slower
Qwen 3.6 35B A3B UD Q4K XL 36 tok prompt · 96 tok output · 3 runs · Compared	Z 311.3 tok/s 115.6 ms L 404.7 tok/s 86.5 ms 77% -93.4 tok/s	Z 109.3 tok/s 9.1 ms/tok L 109.0 tok/s 9.2 ms/tok 100% +0.4 tok/s	Z 132.8 tok/s prompt+decode L 136.1 tok/s prompt+decode 98% -3.3 tok/s 7022 ms slower

Decode 73.5 tok/s 99% decode ratio

Prefill 397.2 tok/s 248% prefill ratio · 181 prompt tok

Fastest Qwen 3.6 35B A3B UD Q4K XL 128.9 tok/s

Overall 117% best row: Qwen 3.6 35B A3B UD Q4K XL

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 192p · 160out Compared

Prefill 47%

ZINC 90.5 tok/s

llama 193.4 tok/s

Decode 90%

ZINC 91.7 tok/s

llama 101.4 tok/s

67% overall

Gemma 4 31B Q4K M 192p · 160out Compared

Prefill 146%

ZINC 73.6 tok/s

llama 50.3 tok/s

Decode 85%

ZINC 24.1 tok/s

llama 28.3 tok/s

103% overall

Qwen 3.5 9B Q4K M 174p · 160out Compared

Prefill 421%

ZINC 874.1 tok/s

llama 207.7 tok/s

Decode 110%

ZINC 94.2 tok/s

llama 85.4 tok/s

143% overall

Qwen 3.6 27B Dense Q4K M 174p · 160out Compared

Prefill 261%

ZINC 194.7 tok/s

llama 74.5 tok/s

Decode 93%

ZINC 28.4 tok/s

llama 30.6 tok/s

116% overall

Qwen 3.6 35B A3B UD Q4K XL 174p · 160out Compared

Prefill 365%

ZINC 753.3 tok/s

llama 206.1 tok/s

Decode 119%

ZINC 128.9 tok/s

llama 108.8 tok/s

157% overall

Details

Gemma 4 26B-A4B MoE Q4K M 192 tok prompt · 160 tok output

Compared

Prefill 47%

ZINC 90.5 tok/s

llama 193.4 tok/s

Decode 90%

ZINC 91.7 tok/s

llama 101.4 tok/s

Overall 67%

ZINC 91.1 tok/s

llama 136.9 tok/s

Prompt+decode. Cold latency: ZINC 11534 ms · llama.cpp 1726 ms

Gemma 4 31B Q4K M 192 tok prompt · 160 tok output

Compared

Prefill 146%

ZINC 73.6 tok/s

llama 50.3 tok/s

Decode 85%

ZINC 24.1 tok/s

llama 28.3 tok/s

Overall 103%

ZINC 38.1 tok/s

llama 37.1 tok/s

Prompt+decode. Cold latency: ZINC 15240 ms · llama.cpp 5886 ms

Qwen 3.5 9B Q4K M 174 tok prompt · 160 tok output

Compared

Prefill 421%

ZINC 874.1 tok/s

llama 207.7 tok/s

Decode 110%

ZINC 94.2 tok/s

llama 85.4 tok/s

Overall 143%

ZINC 176.0 tok/s

llama 123.2 tok/s

Prompt+decode. Cold latency: ZINC 4921 ms · llama.cpp 1998 ms

Qwen 3.6 27B Dense Q4K M 174 tok prompt · 160 tok output

Compared

Prefill 261%

ZINC 194.7 tok/s

llama 74.5 tok/s

Decode 93%

ZINC 28.4 tok/s

llama 30.6 tok/s

Overall 116%

ZINC 51.1 tok/s

llama 44.1 tok/s

Prompt+decode. Cold latency: ZINC 12175 ms · llama.cpp 5470 ms

Qwen 3.6 35B A3B UD Q4K XL 174 tok prompt · 160 tok output

Compared

Prefill 365%

ZINC 753.3 tok/s

llama 206.1 tok/s

Decode 119%

ZINC 128.9 tok/s

llama 108.8 tok/s

Overall 157%

ZINC 226.9 tok/s

llama 144.3 tok/s

Prompt+decode. Cold latency: ZINC 8973 ms · llama.cpp 1613 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 192 tok prompt · 160 tok output · 3 runs · Compared	Z 90.5 tok/s 2120.3 ms L 193.4 tok/s 1002.9 ms 47% -102.9 tok/s	Z 91.7 tok/s 10.9 ms/tok L 101.4 tok/s 9.9 ms/tok 90% -9.7 tok/s	Z 91.1 tok/s prompt+decode L 136.9 tok/s prompt+decode 67% -45.9 tok/s 9809 ms slower
Gemma 4 31B Q4K M 192 tok prompt · 160 tok output · 3 runs · Compared	Z 73.6 tok/s 2607.5 ms L 50.3 tok/s 3856.8 ms 146% +23.3 tok/s	Z 24.1 tok/s 41.4 ms/tok L 28.3 tok/s 35.4 ms/tok 85% -4.1 tok/s	Z 38.1 tok/s prompt+decode L 37.1 tok/s prompt+decode 103% +1.0 tok/s 9354 ms slower
Qwen 3.5 9B Q4K M 174 tok prompt · 160 tok output · 3 runs · Compared	Z 874.1 tok/s 199.1 ms L 207.7 tok/s 736.5 ms 421% +666.3 tok/s	Z 94.2 tok/s 10.6 ms/tok L 85.4 tok/s 11.7 ms/tok 110% +8.8 tok/s	Z 176.0 tok/s prompt+decode L 123.2 tok/s prompt+decode 143% +52.8 tok/s 2923 ms slower
Qwen 3.6 27B Dense Q4K M 174 tok prompt · 160 tok output · 3 runs · Compared	Z 194.7 tok/s 893.6 ms L 74.5 tok/s 2054.5 ms 261% +120.2 tok/s	Z 28.4 tok/s 35.3 ms/tok L 30.6 tok/s 32.7 ms/tok 93% -2.2 tok/s	Z 51.1 tok/s prompt+decode L 44.1 tok/s prompt+decode 116% +7.0 tok/s 6705 ms slower
Qwen 3.6 35B A3B UD Q4K XL 174 tok prompt · 160 tok output · 3 runs · Compared	Z 753.3 tok/s 231.0 ms L 206.1 tok/s 742.2 ms 365% +547.2 tok/s	Z 128.9 tok/s 7.8 ms/tok L 108.8 tok/s 9.2 ms/tok 119% +20.2 tok/s	Z 226.9 tok/s prompt+decode L 144.3 tok/s prompt+decode 157% +82.6 tok/s 7359 ms slower

Decode 73.5 tok/s 99% decode ratio

Prefill 422.3 tok/s 262% prefill ratio · 332 prompt tok

Fastest Qwen 3.6 35B A3B UD Q4K XL 131.8 tok/s

Overall 139% best row: Qwen 3.6 35B A3B UD Q4K XL

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 346p · 128out Compared

Prefill 53%

ZINC 94.4 tok/s

llama 177.8 tok/s

Decode 88%

ZINC 88.8 tok/s

llama 100.9 tok/s

63% overall

Gemma 4 31B Q4K M 346p · 128out Compared

Prefill 160%

ZINC 75.8 tok/s

llama 47.2 tok/s

Decode 85%

ZINC 23.9 tok/s

llama 28.2 tok/s

120% overall

Qwen 3.5 9B Q4K M 322p · 128out Compared

Prefill 441%

ZINC 958.8 tok/s

llama 217.2 tok/s

Decode 111%

ZINC 94.7 tok/s

llama 85.4 tok/s

177% overall

Qwen 3.6 27B Dense Q4K M 322p · 128out Compared

Prefill 285%

ZINC 217.9 tok/s

llama 76.4 tok/s

Decode 93%

ZINC 28.3 tok/s

llama 30.6 tok/s

140% overall

Qwen 3.6 35B A3B UD Q4K XL 322p · 128out Compared

Prefill 370%

ZINC 764.8 tok/s

llama 206.6 tok/s

Decode 121%

ZINC 131.8 tok/s

llama 108.9 tok/s

196% overall

Details

Gemma 4 26B-A4B MoE Q4K M 346 tok prompt · 128 tok output

Compared

Prefill 53%

ZINC 94.4 tok/s

llama 177.8 tok/s

Decode 88%

ZINC 88.8 tok/s

llama 100.9 tok/s

Overall 63%

ZINC 92.8 tok/s

llama 147.4 tok/s

Prompt+decode. Cold latency: ZINC 12600 ms · llama.cpp 1423 ms

Gemma 4 31B Q4K M 346 tok prompt · 128 tok output

Compared

Prefill 160%

ZINC 75.8 tok/s

llama 47.2 tok/s

Decode 85%

ZINC 23.9 tok/s

llama 28.2 tok/s

Overall 120%

ZINC 47.8 tok/s

llama 39.9 tok/s

Prompt+decode. Cold latency: ZINC 15876 ms · llama.cpp 4774 ms

Qwen 3.5 9B Q4K M 322 tok prompt · 128 tok output

Compared

Prefill 441%

ZINC 958.8 tok/s

llama 217.2 tok/s

Decode 111%

ZINC 94.7 tok/s

llama 85.4 tok/s

Overall 177%

ZINC 266.6 tok/s

llama 150.9 tok/s

Prompt+decode. Cold latency: ZINC 4941 ms · llama.cpp 1642 ms

Qwen 3.6 27B Dense Q4K M 322 tok prompt · 128 tok output

Compared

Prefill 285%

ZINC 217.9 tok/s

llama 76.4 tok/s

Decode 93%

ZINC 28.3 tok/s

llama 30.6 tok/s

Overall 140%

ZINC 75.1 tok/s

llama 53.6 tok/s

Prompt+decode. Cold latency: ZINC 11858 ms · llama.cpp 4365 ms

Qwen 3.6 35B A3B UD Q4K XL 322 tok prompt · 128 tok output

Compared

Prefill 370%

ZINC 764.8 tok/s

llama 206.6 tok/s

Decode 121%

ZINC 131.8 tok/s

llama 108.9 tok/s

Overall 196%

ZINC 323.3 tok/s

llama 164.6 tok/s

Prompt+decode. Cold latency: ZINC 8710 ms · llama.cpp 1320 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 346 tok prompt · 128 tok output · 3 runs · Compared	Z 94.4 tok/s 3663.6 ms L 177.8 tok/s 1934.7 ms 53% -83.4 tok/s	Z 88.8 tok/s 11.3 ms/tok L 100.9 tok/s 9.9 ms/tok 88% -12.1 tok/s	Z 92.8 tok/s prompt+decode L 147.4 tok/s prompt+decode 63% -54.6 tok/s 11178 ms slower
Gemma 4 31B Q4K M 346 tok prompt · 128 tok output · 3 runs · Compared	Z 75.8 tok/s 4565.9 ms L 47.2 tok/s 7280.7 ms 160% +28.5 tok/s	Z 23.9 tok/s 41.8 ms/tok L 28.2 tok/s 35.5 ms/tok 85% -4.3 tok/s	Z 47.8 tok/s prompt+decode L 39.9 tok/s prompt+decode 120% +7.8 tok/s 11102 ms slower
Qwen 3.5 9B Q4K M 322 tok prompt · 128 tok output · 3 runs · Compared	Z 958.8 tok/s 335.8 ms L 217.2 tok/s 1450.4 ms 441% +741.6 tok/s	Z 94.7 tok/s 10.6 ms/tok L 85.4 tok/s 11.7 ms/tok 111% +9.2 tok/s	Z 266.6 tok/s prompt+decode L 150.9 tok/s prompt+decode 177% +115.6 tok/s 3299 ms slower
Qwen 3.6 27B Dense Q4K M 322 tok prompt · 128 tok output · 3 runs · Compared	Z 217.9 tok/s 1477.5 ms L 76.4 tok/s 4123.3 ms 285% +141.5 tok/s	Z 28.3 tok/s 35.3 ms/tok L 30.6 tok/s 32.7 ms/tok 93% -2.3 tok/s	Z 75.1 tok/s prompt+decode L 53.6 tok/s prompt+decode 140% +21.5 tok/s 7492 ms slower
Qwen 3.6 35B A3B UD Q4K XL 322 tok prompt · 128 tok output · 3 runs · Compared	Z 764.8 tok/s 421.0 ms L 206.6 tok/s 1524.7 ms 370% +558.2 tok/s	Z 131.8 tok/s 7.6 ms/tok L 108.9 tok/s 9.2 ms/tok 121% +23.0 tok/s	Z 323.3 tok/s prompt+decode L 164.6 tok/s prompt+decode 196% +158.7 tok/s 7390 ms slower

Decode 74.6 tok/s 101% decode ratio

Prefill 295.2 tok/s 47% prefill ratio · 66 prompt tok

Fastest Qwen 3.6 35B A3B UD Q4K XL 134.0 tok/s

Overall 94% best row: Qwen 3.6 35B A3B UD Q4K XL

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 70p · 256out Compared

Prefill 14%

ZINC 94.0 tok/s

llama 659.6 tok/s

Decode 91%

ZINC 92.2 tok/s

llama 101.5 tok/s

75% overall

Gemma 4 31B Q4K M 70p · 256out Compared

Prefill 20%

ZINC 49.5 tok/s

llama 242.4 tok/s

Decode 86%

ZINC 24.3 tok/s

llama 28.3 tok/s

78% overall

Qwen 3.5 9B Q4K M 64p · 256out Compared

Prefill 92%

ZINC 800.5 tok/s

llama 869.3 tok/s

Decode 111%

ZINC 94.4 tok/s

llama 85.2 tok/s

110% overall

Qwen 3.6 27B Dense Q4K M 64p · 256out Compared

Prefill 29%

ZINC 74.5 tok/s

llama 256.6 tok/s

Decode 93%

ZINC 28.3 tok/s

llama 30.5 tok/s

87% overall

Qwen 3.6 35B A3B UD Q4K XL 64p · 256out Compared

Prefill 78%

ZINC 457.6 tok/s

llama 584.2 tok/s

Decode 125%

ZINC 134.0 tok/s

llama 107.1 tok/s

122% overall

Details

Gemma 4 26B-A4B MoE Q4K M 70 tok prompt · 256 tok output

Compared

Prefill 14%

ZINC 94.0 tok/s

llama 659.6 tok/s

Decode 91%

ZINC 92.2 tok/s

llama 101.5 tok/s

Overall 75%

ZINC 92.6 tok/s

llama 124.1 tok/s

Prompt+decode. Cold latency: ZINC 10500 ms · llama.cpp 2755 ms

Gemma 4 31B Q4K M 70 tok prompt · 256 tok output

Compared

Prefill 20%

ZINC 49.5 tok/s

llama 242.4 tok/s

Decode 86%

ZINC 24.3 tok/s

llama 28.3 tok/s

Overall 78%

ZINC 27.3 tok/s

llama 34.9 tok/s

Prompt+decode. Cold latency: ZINC 17737 ms · llama.cpp 9537 ms

Qwen 3.5 9B Q4K M 64 tok prompt · 256 tok output

Compared

Prefill 92%

ZINC 800.5 tok/s

llama 869.3 tok/s

Decode 111%

ZINC 94.4 tok/s

llama 85.2 tok/s

Overall 110%

ZINC 114.6 tok/s

llama 104.0 tok/s

Prompt+decode. Cold latency: ZINC 5890 ms · llama.cpp 3212 ms

Qwen 3.6 27B Dense Q4K M 64 tok prompt · 256 tok output

Compared

Prefill 29%

ZINC 74.5 tok/s

llama 256.6 tok/s

Decode 93%

ZINC 28.3 tok/s

llama 30.5 tok/s

Overall 87%

ZINC 32.3 tok/s

llama 37.1 tok/s

Prompt+decode. Cold latency: ZINC 15875 ms · llama.cpp 8799 ms

Qwen 3.6 35B A3B UD Q4K XL 64 tok prompt · 256 tok output

Compared

Prefill 78%

ZINC 457.6 tok/s

llama 584.2 tok/s

Decode 125%

ZINC 134.0 tok/s

llama 107.1 tok/s

Overall 122%

ZINC 156.0 tok/s

llama 128.1 tok/s

Prompt+decode. Cold latency: ZINC 9550 ms · llama.cpp 2649 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 70 tok prompt · 256 tok output · 3 runs · Compared	Z 94.0 tok/s 744.5 ms L 659.6 tok/s 109.2 ms 14% -565.5 tok/s	Z 92.2 tok/s 10.8 ms/tok L 101.5 tok/s 9.8 ms/tok 91% -9.3 tok/s	Z 92.6 tok/s prompt+decode L 124.1 tok/s prompt+decode 75% -31.5 tok/s 7746 ms slower
Gemma 4 31B Q4K M 70 tok prompt · 256 tok output · 3 runs · Compared	Z 49.5 tok/s 1413.5 ms L 242.4 tok/s 297.1 ms 20% -192.9 tok/s	Z 24.3 tok/s 41.2 ms/tok L 28.3 tok/s 35.4 ms/tok 86% -4.0 tok/s	Z 27.3 tok/s prompt+decode L 34.9 tok/s prompt+decode 78% -7.6 tok/s 8200 ms slower
Qwen 3.5 9B Q4K M 64 tok prompt · 256 tok output · 3 runs · Compared	Z 800.5 tok/s 80.0 ms L 869.3 tok/s 72.5 ms 92% -68.9 tok/s	Z 94.4 tok/s 10.6 ms/tok L 85.2 tok/s 11.7 ms/tok 111% +9.2 tok/s	Z 114.6 tok/s prompt+decode L 104.0 tok/s prompt+decode 110% +10.6 tok/s 2678 ms slower
Qwen 3.6 27B Dense Q4K M 64 tok prompt · 256 tok output · 3 runs · Compared	Z 74.5 tok/s 859.3 ms L 256.6 tok/s 245.5 ms 29% -182.1 tok/s	Z 28.3 tok/s 35.4 ms/tok L 30.5 tok/s 32.8 ms/tok 93% -2.2 tok/s	Z 32.3 tok/s prompt+decode L 37.1 tok/s prompt+decode 87% -4.8 tok/s 7077 ms slower
Qwen 3.6 35B A3B UD Q4K XL 64 tok prompt · 256 tok output · 3 runs · Compared	Z 457.6 tok/s 139.9 ms L 584.2 tok/s 107.8 ms 78% -126.6 tok/s	Z 134.0 tok/s 7.5 ms/tok L 107.1 tok/s 9.3 ms/tok 125% +26.8 tok/s	Z 156.0 tok/s prompt+decode L 128.1 tok/s prompt+decode 122% +28.0 tok/s 6902 ms slower

Methodology ZINC CLI vs llama.cpp on the same RDNA node · Four-scenario matrix with same-file baselines

3 measured · 1 warmup

Benchmark runner Exact prompts and token caps Prompt matrix test Published JSON

Hardware: one AMD Radeon AI PRO R9700 benchmark node with 32 GB VRAM and 576 GB/s memory bandwidth. ZINC and llama.cpp run on the same Ubuntu host and use the same GGUF file for each model.
ZINC path: sync the current source tree to the RDNA node, build with zig build -Doptimize=ReleaseFast, then measure generation through the ZINC CLI with RADV cooperative matrix support enabled.
ZINC RDNA backend: vulkan. Published RDNA runs default to Vulkan; zinc_rt is opt-in because it is a separate bring-up runtime.
Baseline path: launch llama.cpp against the same model file, preferring one reusable llama-server per model across the full scenario matrix and falling back to llama-cli when the server path is unavailable.
Scenarios: Quick Chat, Coding Review, Incident Context, and Long Coding Draft. The prompts are real-world chat, code-review, support-context, and coding-plan workloads instead of synthetic factual completions.
Statistics: one warmup pass is discarded, then three measured runs are collected. Published prefill, decode, end-to-end throughput, and latency values are medians.
Execution order: the harness records the full ZINC scenario matrix before starting the llama.cpp baseline phase, so only one inference engine owns the GPU during measurement.
Run hygiene: published RDNA runs assume a clean node with no competing ZINC, llama.cpp, or other GPU workloads.

Mac Studio

Metal

Apple M4 Max GPU · 64 GB · 3 measured · 1 warmup

Run info

GPU: Apple M4 Max GPU
Chip: Apple M4 Max
Memory: 64 GB
Artifact: Jun 11, 2026, 4:11 PM
ZINC: ce9f0c8f1efd
llama.cpp: 607

Fastest 82.5 tok/s Qwen 3.6 35B A3B UD Q4K XL

Decode 35.3 tok/s average

Overall 89% prefill + decode vs llama.cpp

Decode 35.3 tok/s 123% decode ratio

Prefill 151.5 tok/s 196% prefill ratio · 41 prompt tok

Fastest Qwen 3.6 35B A3B UD Q4K XL 82.5 tok/s

Overall 120% best row: Qwen 3.6 35B A3B UD Q4K XL

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 49p · 96out ZINC Only

Prefill n/a

ZINC 350.1 tok/s

llama n/a

Decode n/a

ZINC 32.1 tok/s

llama n/a

n/a overall

Gemma 4 31B Q4K M 49p · 96out Compared

Prefill 408%

ZINC 131.3 tok/s

llama 32.2 tok/s

Decode 108%

ZINC 9.2 tok/s

llama 8.5 tok/s

118% overall

Qwen 3.5 9B Q4K M 36p · 96out Compared

Prefill 80%

ZINC 27.3 tok/s

llama 34.1 tok/s

Decode 62%

ZINC 17.3 tok/s

llama 27.8 tok/s

66% overall

Qwen 3.6 27B Dense Q4K M 35p · 96out Run Failed

Prefill n/a

ZINC n/a

llama 28.7 tok/s

Decode n/a

ZINC n/a

llama 23.1 tok/s

n/a overall

Qwen 3.6 35B A3B UD Q4K XL 36p · 96out Compared

Prefill 101%

ZINC 97.2 tok/s

llama 96.4 tok/s

Decode 199%

ZINC 82.5 tok/s

llama 41.6 tok/s

175% overall

Details

Gemma 4 26B-A4B MoE Q4K M 49 tok prompt · 96 tok output

ZINC Only

Prefill n/a

ZINC 350.1 tok/s

llama n/a

Decode n/a

ZINC 32.1 tok/s

llama n/a

Overall n/a

ZINC 46.4 tok/s

llama n/a

Prompt+decode. Cold latency: ZINC 8202 ms · llama.cpp n/a

Gemma 4 31B Q4K M 49 tok prompt · 96 tok output

Compared

Prefill 408%

ZINC 131.3 tok/s

llama 32.2 tok/s

Decode 108%

ZINC 9.2 tok/s

llama 8.5 tok/s

Overall 118%

ZINC 13.4 tok/s

llama 11.3 tok/s

Prompt+decode. Cold latency: ZINC 15606 ms · llama.cpp 12565 ms

Qwen 3.5 9B Q4K M 36 tok prompt · 96 tok output

Compared

Prefill 80%

ZINC 27.3 tok/s

llama 34.1 tok/s

Decode 62%

ZINC 17.3 tok/s

llama 27.8 tok/s

Overall 66%

ZINC 19.2 tok/s

llama 29.3 tok/s

Prompt+decode. Cold latency: ZINC 7348 ms · llama.cpp 3574 ms

Qwen 3.6 27B Dense Q4K M 35 tok prompt · 96 tok output

Run Failed

Prefill n/a

ZINC n/a

llama 28.7 tok/s

Decode n/a

ZINC n/a

llama 23.1 tok/s

Overall n/a

ZINC n/a

llama 24.4 tok/s

Prompt+decode. Cold latency: ZINC n/a · llama.cpp 4302 ms

Qwen 3.6 35B A3B UD Q4K XL 36 tok prompt · 96 tok output

Compared

Prefill 101%

ZINC 97.2 tok/s

llama 96.4 tok/s

Decode 199%

ZINC 82.5 tok/s

llama 41.6 tok/s

Overall 175%

ZINC 86.1 tok/s

llama 49.2 tok/s

Prompt+decode. Cold latency: ZINC 3591 ms · llama.cpp 2352 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 49 tok prompt · 96 tok output · 3 runs · ZINC Only	Z 350.1 tok/s 140.0 ms L n/a n/a Comparison unavailable Prefill telemetry is missing on one side.	Z 32.1 tok/s 31.1 ms/tok L n/a n/a Comparison unavailable No same-machine decode ratio for this row yet.	Z 46.4 tok/s prompt+decode L n/a prompt+decode Directional only No end-to-end comparison for this row.
Gemma 4 31B Q4K M 49 tok prompt · 96 tok output · 3 runs · Compared	Z 131.3 tok/s 373.1 ms L 32.2 tok/s 1584.8 ms 408% +99.1 tok/s	Z 9.2 tok/s 109.1 ms/tok L 8.5 tok/s 118.0 ms/tok 108% +0.7 tok/s	Z 13.4 tok/s prompt+decode L 11.3 tok/s prompt+decode 118% +2.1 tok/s 3041 ms slower
Qwen 3.5 9B Q4K M 36 tok prompt · 96 tok output · 3 runs · Compared	Z 27.3 tok/s 1317.1 ms L 34.1 tok/s 1026.2 ms 80% -6.8 tok/s	Z 17.3 tok/s 57.7 ms/tok L 27.8 tok/s 36.0 ms/tok 62% -10.5 tok/s	Z 19.2 tok/s prompt+decode L 29.3 tok/s prompt+decode 66% -10.0 tok/s 3774 ms slower
Qwen 3.6 27B Dense Q4K M 35 tok prompt · 96 tok output · 3 runs · Run Failed	Z n/a n/a L 28.7 tok/s 1219.0 ms Comparison unavailable Prefill telemetry is missing on one side.	Z n/a n/a L 23.1 tok/s 43.3 ms/tok Comparison unavailable No same-machine decode ratio for this row yet.	Z n/a prompt+decode L 24.4 tok/s prompt+decode Directional only No end-to-end comparison for this row.
Qwen 3.6 35B A3B UD Q4K XL 36 tok prompt · 96 tok output · 3 runs · Compared	Z 97.2 tok/s 370.4 ms L 96.4 tok/s 363.2 ms 101% +0.8 tok/s	Z 82.5 tok/s 12.1 ms/tok L 41.6 tok/s 24.1 ms/tok 199% +41.0 tok/s	Z 86.1 tok/s prompt+decode L 49.2 tok/s prompt+decode 175% +36.9 tok/s 1239 ms slower

Decode 29.3 tok/s 62% decode ratio

Prefill 49.8 tok/s 66% prefill ratio · 167 prompt tok

Fastest Qwen 3.6 35B A3B UD Q4K XL 41.5 tok/s

Overall 63% best row: Qwen 3.5 9B Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M n/ap · 160out Run Failed

Prefill n/a

ZINC n/a

llama n/a

Decode n/a

ZINC n/a

llama n/a

n/a overall

Gemma 4 31B Q4K M n/ap · 160out Run Failed

Prefill n/a

ZINC n/a

llama n/a

Decode n/a

ZINC n/a

llama n/a

n/a overall

Qwen 3.5 9B Q4K M 174p · 160out Compared

Prefill 66%

ZINC 22.6 tok/s

llama 34.5 tok/s

Decode 62%

ZINC 17.1 tok/s

llama 27.7 tok/s

63% overall

Qwen 3.6 27B Dense Q4K M 153p · 160out Run Failed

Prefill n/a

ZINC n/a

llama 28.7 tok/s

Decode n/a

ZINC n/a

llama 23.0 tok/s

n/a overall

Qwen 3.6 35B A3B UD Q4K XL 174p · 160out ZINC Only

Prefill n/a

ZINC 76.9 tok/s

llama n/a

Decode n/a

ZINC 41.5 tok/s

llama n/a

n/a overall

Details

Gemma 4 26B-A4B MoE Q4K M n/a · 160 tok output

Run Failed

Prefill n/a

ZINC n/a

llama n/a

Decode n/a

ZINC n/a

llama n/a

Overall n/a

ZINC n/a

llama n/a

Prompt+decode. Cold latency: ZINC n/a · llama.cpp n/a

Gemma 4 31B Q4K M n/a · 160 tok output

Run Failed

Prefill n/a

ZINC n/a

llama n/a

Decode n/a

ZINC n/a

llama n/a

Overall n/a

ZINC n/a

llama n/a

Prompt+decode. Cold latency: ZINC n/a · llama.cpp n/a

Qwen 3.5 9B Q4K M 174 tok prompt · 160 tok output

Compared

Prefill 66%

ZINC 22.6 tok/s

llama 34.5 tok/s

Decode 62%

ZINC 17.1 tok/s

llama 27.7 tok/s

Overall 63%

ZINC 19.6 tok/s

llama 30.8 tok/s

Prompt+decode. Cold latency: ZINC 17614 ms · llama.cpp 5893 ms

Qwen 3.6 27B Dense Q4K M 153 tok prompt · 160 tok output

Run Failed

Prefill n/a

ZINC n/a

llama 28.7 tok/s

Decode n/a

ZINC n/a

llama 23.0 tok/s

Overall n/a

ZINC n/a

llama 25.5 tok/s

Prompt+decode. Cold latency: ZINC n/a · llama.cpp 7104 ms

Qwen 3.6 35B A3B UD Q4K XL 174 tok prompt · 160 tok output

ZINC Only

Prefill n/a

ZINC 76.9 tok/s

llama n/a

Decode n/a

ZINC 41.5 tok/s

llama n/a

Overall n/a

ZINC 54.6 tok/s

llama n/a

Prompt+decode. Cold latency: ZINC 8098 ms · llama.cpp n/a

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M n/a · 160 tok output · 0 runs · Run Failed	Telemetry pending No prefill metrics were published for this run.	Run missing No decode result was published for this row.	Telemetry pending No end-to-end or latency metrics were published.
Gemma 4 31B Q4K M n/a · 160 tok output · 0 runs · Run Failed	Telemetry pending No prefill metrics were published for this run.	Run missing No decode result was published for this row.	Telemetry pending No end-to-end or latency metrics were published.
Qwen 3.5 9B Q4K M 174 tok prompt · 160 tok output · 3 runs · Compared	Z 22.6 tok/s 7694.4 ms L 34.5 tok/s 4440.5 ms 66% -11.9 tok/s	Z 17.1 tok/s 58.5 ms/tok L 27.7 tok/s 36.1 ms/tok 62% -10.6 tok/s	Z 19.6 tok/s prompt+decode L 30.8 tok/s prompt+decode 63% -11.3 tok/s 11721 ms slower
Qwen 3.6 27B Dense Q4K M 153 tok prompt · 160 tok output · 3 runs · Run Failed	Z n/a n/a L 28.7 tok/s 5327.5 ms Comparison unavailable Prefill telemetry is missing on one side.	Z n/a n/a L 23.0 tok/s 43.5 ms/tok Comparison unavailable No same-machine decode ratio for this row yet.	Z n/a prompt+decode L 25.5 tok/s prompt+decode Directional only No end-to-end comparison for this row.
Qwen 3.6 35B A3B UD Q4K XL 174 tok prompt · 160 tok output · 3 runs · ZINC Only	Z 76.9 tok/s 2262.1 ms L n/a n/a Comparison unavailable Prefill telemetry is missing on one side.	Z 41.5 tok/s 24.1 ms/tok L n/a n/a Comparison unavailable No same-machine decode ratio for this row yet.	Z 54.6 tok/s prompt+decode L n/a prompt+decode Directional only No end-to-end comparison for this row.

Decode 10.5 tok/s 46% decode ratio

Prefill 11.6 tok/s 40% prefill ratio · 319 prompt tok

Fastest Qwen 3.6 27B Dense Q4K M 10.5 tok/s

Overall 42% best row: Qwen 3.6 27B Dense Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M n/ap · 128out Run Failed

Prefill n/a

ZINC n/a

llama n/a

Decode n/a

ZINC n/a

llama n/a

n/a overall

Gemma 4 31B Q4K M n/ap · 128out Run Failed

Prefill n/a

ZINC n/a

llama n/a

Decode n/a

ZINC n/a

llama n/a

n/a overall

Qwen 3.5 9B Q4K M n/ap · 128out Run Failed

Prefill n/a

ZINC n/a

llama n/a

Decode n/a

ZINC n/a

llama n/a

n/a overall

Qwen 3.6 27B Dense Q4K M 322p · 128out Compared

Prefill 40%

ZINC 11.6 tok/s

llama 28.6 tok/s

Decode 46%

ZINC 10.5 tok/s

llama 23.0 tok/s

42% overall

Qwen 3.6 35B A3B UD Q4K XL 315p · 128out Run Failed

Prefill n/a

ZINC n/a

llama 90.7 tok/s

Decode n/a

ZINC n/a

llama 42.1 tok/s

n/a overall

Details

Gemma 4 26B-A4B MoE Q4K M n/a · 128 tok output

Run Failed

Prefill n/a

ZINC n/a

llama n/a

Decode n/a

ZINC n/a

llama n/a

Overall n/a

ZINC n/a

llama n/a

Prompt+decode. Cold latency: ZINC n/a · llama.cpp n/a

Gemma 4 31B Q4K M n/a · 128 tok output

Run Failed

Prefill n/a

ZINC n/a

llama n/a

Decode n/a

ZINC n/a

llama n/a

Overall n/a

ZINC n/a

llama n/a

Prompt+decode. Cold latency: ZINC n/a · llama.cpp n/a

Qwen 3.5 9B Q4K M n/a · 128 tok output

Run Failed

Prefill n/a

ZINC n/a

llama n/a

Decode n/a

ZINC n/a

llama n/a

Overall n/a

ZINC n/a

llama n/a

Prompt+decode. Cold latency: ZINC n/a · llama.cpp n/a

Qwen 3.6 27B Dense Q4K M 322 tok prompt · 128 tok output

Compared

Prefill 40%

ZINC 11.6 tok/s

llama 28.6 tok/s

Decode 46%

ZINC 10.5 tok/s

llama 23.0 tok/s

Overall 42%

ZINC 11.3 tok/s

llama 26.8 tok/s

Prompt+decode. Cold latency: ZINC 41281 ms · llama.cpp 5703 ms

Qwen 3.6 35B A3B UD Q4K XL 315 tok prompt · 128 tok output

Run Failed

Prefill n/a

ZINC n/a

llama 90.7 tok/s

Decode n/a

ZINC n/a

llama 42.1 tok/s

Overall n/a

ZINC n/a

llama 68.0 tok/s

Prompt+decode. Cold latency: ZINC n/a · llama.cpp 3082 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M n/a · 128 tok output · 0 runs · Run Failed	Telemetry pending No prefill metrics were published for this run.	Run missing No decode result was published for this row.	Telemetry pending No end-to-end or latency metrics were published.
Gemma 4 31B Q4K M n/a · 128 tok output · 0 runs · Run Failed	Telemetry pending No prefill metrics were published for this run.	Run missing No decode result was published for this row.	Telemetry pending No end-to-end or latency metrics were published.
Qwen 3.5 9B Q4K M n/a · 128 tok output · 0 runs · Run Failed	Telemetry pending No prefill metrics were published for this run.	Run missing No decode result was published for this row.	Telemetry pending No end-to-end or latency metrics were published.
Qwen 3.6 27B Dense Q4K M 322 tok prompt · 128 tok output · 3 runs · Compared	Z 11.6 tok/s 27839.4 ms L 28.6 tok/s 10995.6 ms 40% -17.0 tok/s	Z 10.5 tok/s 95.4 ms/tok L 23.0 tok/s 43.4 ms/tok 46% -12.5 tok/s	Z 11.3 tok/s prompt+decode L 26.8 tok/s prompt+decode 42% -15.5 tok/s 35578 ms slower
Qwen 3.6 35B A3B UD Q4K XL 315 tok prompt · 128 tok output · 3 runs · Run Failed	Z n/a n/a L 90.7 tok/s 3471.7 ms Comparison unavailable Prefill telemetry is missing on one side.	Z n/a n/a L 42.1 tok/s 23.8 ms/tok Comparison unavailable No same-machine decode ratio for this row yet.	Z n/a prompt+decode L 68.0 tok/s prompt+decode Directional only No end-to-end comparison for this row.

Decode 34.3 tok/s 81% decode ratio

Prefill 159.6 tok/s 77% prefill ratio · 65 prompt tok

Fastest Qwen 3.6 35B A3B UD Q4K XL 47.5 tok/s

Overall 80% best row: Qwen 3.6 35B A3B UD Q4K XL

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 70p · 256out ZINC Only

Prefill n/a

ZINC 365.2 tok/s

llama n/a

Decode n/a

ZINC 44.7 tok/s

llama n/a

n/a overall

Gemma 4 31B Q4K M n/ap · 256out Run Failed

Prefill n/a

ZINC n/a

llama n/a

Decode n/a

ZINC n/a

llama n/a

n/a overall

Qwen 3.5 9B Q4K M 63p · 256out Run Failed

Prefill n/a

ZINC n/a

llama 34.4 tok/s

Decode n/a

ZINC n/a

llama 27.5 tok/s

n/a overall

Qwen 3.6 27B Dense Q4K M 64p · 256out Compared

Prefill 40%

ZINC 11.6 tok/s

llama 28.7 tok/s

Decode 46%

ZINC 10.6 tok/s

llama 22.9 tok/s

45% overall

Qwen 3.6 35B A3B UD Q4K XL 64p · 256out Compared

Prefill 113%

ZINC 102.0 tok/s

llama 90.1 tok/s

Decode 116%

ZINC 47.5 tok/s

llama 41.1 tok/s

115% overall

Details

Gemma 4 26B-A4B MoE Q4K M 70 tok prompt · 256 tok output

ZINC Only

Prefill n/a

ZINC 365.2 tok/s

llama n/a

Decode n/a

ZINC 44.7 tok/s

llama n/a

Overall n/a

ZINC 55.1 tok/s

llama n/a

Prompt+decode. Cold latency: ZINC 11193 ms · llama.cpp n/a

Gemma 4 31B Q4K M n/a · 256 tok output

Run Failed

Prefill n/a

ZINC n/a

llama n/a

Decode n/a

ZINC n/a

llama n/a

Overall n/a

ZINC n/a

llama n/a

Prompt+decode. Cold latency: ZINC n/a · llama.cpp n/a

Qwen 3.5 9B Q4K M 63 tok prompt · 256 tok output

Run Failed

Prefill n/a

ZINC n/a

llama 34.4 tok/s

Decode n/a

ZINC n/a

llama 27.5 tok/s

Overall n/a

ZINC n/a

llama 28.7 tok/s

Prompt+decode. Cold latency: ZINC n/a · llama.cpp 9408 ms

Qwen 3.6 27B Dense Q4K M 64 tok prompt · 256 tok output

Compared

Prefill 40%

ZINC 11.6 tok/s

llama 28.7 tok/s

Decode 46%

ZINC 10.6 tok/s

llama 22.9 tok/s

Overall 45%

ZINC 10.8 tok/s

llama 23.9 tok/s

Prompt+decode. Cold latency: ZINC 31023 ms · llama.cpp 11304 ms

Qwen 3.6 35B A3B UD Q4K XL 64 tok prompt · 256 tok output

Compared

Prefill 113%

ZINC 102.0 tok/s

llama 90.1 tok/s

Decode 116%

ZINC 47.5 tok/s

llama 41.1 tok/s

Overall 115%

ZINC 53.2 tok/s

llama 46.1 tok/s

Prompt+decode. Cold latency: ZINC 8250 ms · llama.cpp 6277 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 70 tok prompt · 256 tok output · 3 runs · ZINC Only	Z 365.2 tok/s 191.7 ms L n/a n/a Comparison unavailable Prefill telemetry is missing on one side.	Z 44.7 tok/s 22.4 ms/tok L n/a n/a Comparison unavailable No same-machine decode ratio for this row yet.	Z 55.1 tok/s prompt+decode L n/a prompt+decode Directional only No end-to-end comparison for this row.
Gemma 4 31B Q4K M n/a · 256 tok output · 0 runs · Run Failed	Telemetry pending No prefill metrics were published for this run.	Run missing No decode result was published for this row.	Telemetry pending No end-to-end or latency metrics were published.
Qwen 3.5 9B Q4K M 63 tok prompt · 256 tok output · 3 runs · Run Failed	Z n/a n/a L 34.4 tok/s 1833.6 ms Comparison unavailable Prefill telemetry is missing on one side.	Z n/a n/a L 27.5 tok/s 36.3 ms/tok Comparison unavailable No same-machine decode ratio for this row yet.	Z n/a prompt+decode L 28.7 tok/s prompt+decode Directional only No end-to-end comparison for this row.
Qwen 3.6 27B Dense Q4K M 64 tok prompt · 256 tok output · 3 runs · Compared	Z 11.6 tok/s 5506.6 ms L 28.7 tok/s 2197.3 ms 40% -17.1 tok/s	Z 10.6 tok/s 94.6 ms/tok L 22.9 tok/s 43.6 ms/tok 46% -12.4 tok/s	Z 10.8 tok/s prompt+decode L 23.9 tok/s prompt+decode 45% -13.1 tok/s 19719 ms slower
Qwen 3.6 35B A3B UD Q4K XL 64 tok prompt · 256 tok output · 3 runs · Compared	Z 102.0 tok/s 627.2 ms L 90.1 tok/s 699.5 ms 113% +11.9 tok/s	Z 47.5 tok/s 21.0 ms/tok L 41.1 tok/s 24.3 ms/tok 116% +6.4 tok/s	Z 53.2 tok/s prompt+decode L 46.1 tok/s prompt+decode 115% +7.1 tok/s 1974 ms slower

Methodology ZINC CLI vs llama.cpp on the same Apple Silicon machine · Four-scenario matrix with same-file baselines

3 measured · 1 warmup

Benchmark runner Exact prompts and token caps Prompt matrix test Published JSON

Hardware: the local Apple Silicon target shown above. ZINC and llama.cpp run on the same machine and use the same GGUF file for each model.
ZINC path: build the current workspace with zig build -Doptimize=ReleaseFast, then measure generation through the ZINC CLI.
Baseline path: launch llama.cpp against the same model file, preferring one reusable llama-server per model across the full scenario matrix and falling back to llama-cli when the server path is unavailable.
Scenarios: Quick Chat, Coding Review, Incident Context, and Long Coding Draft. The prompts are real-world chat, code-review, support-context, and coding-plan workloads instead of synthetic factual completions.
Statistics: one warmup pass is discarded, then three measured runs are collected. Published prefill, decode, end-to-end throughput, and latency values are medians.
Model resolution: managed catalog models are loaded from the ZINC model cache when available; explicit GGUF paths are preserved when provided.
Run hygiene: published local runs assume no competing workload is saturating CPU, memory bandwidth, or GPU.

Intel Arc bench node

Intel Arc

Intel(R) Graphics (BMG G31) · n/a · 3 measured · 1 warmup

Run info

GPU: Intel(R) Graphics (BMG G31)
Chip: Intel Arc Pro B70 / BMG G31
Memory: n/a
Artifact: Jun 7, 2026, 2:09 PM
ZINC: 0e1c7135b67f-dirty
llama.cpp: 8863

Fastest 104.0 tok/s Qwen 3.6 35B A3B Q4K XL

Decode 42.2 tok/s average

Overall 75% prefill + decode vs llama.cpp

Decode 26.7 tok/s 80% decode ratio

Prefill 30.7 tok/s 34% prefill ratio · 43 prompt tok

Fastest Qwen 3.5 9B Q4K M 56.7 tok/s

Overall 67% best row: Qwen 3.5 9B Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 49p · 96out Compared

Prefill 6%

ZINC 17.5 tok/s

llama 270.1 tok/s

Decode 26%

ZINC 16.1 tok/s

llama 62.7 tok/s

20% overall

Gemma 4 31B Q4K M 49p · 96out Compared

Prefill 27%

ZINC 17.8 tok/s

llama 65.9 tok/s

Decode 93%

ZINC 16.0 tok/s

llama 17.3 tok/s

72% overall

Qwen 3.5 9B Q4K M 36p · 96out Compared

Prefill 48%

ZINC 67.8 tok/s

llama 142.2 tok/s

Decode 106%

ZINC 56.7 tok/s

llama 53.4 tok/s

92% overall

Qwen 3.6 27B Q4K M 36p · 96out Compared

Prefill 54%

ZINC 19.8 tok/s

llama 36.9 tok/s

Decode 95%

ZINC 18.1 tok/s

llama 19.1 tok/s

84% overall

Qwen 3.6 35B A3B Q4K XL 36p · 2out/96cap Compared

Prefill 51%

ZINC 68.0 tok/s

llama 132.3 tok/s

Decode 140%

ZINC 104.0 tok/s

llama 74.5 tok/s

54% overall

Details

Gemma 4 26B-A4B MoE Q4K M 49 tok prompt · 96 tok output

Compared

Prefill 6%

ZINC 17.5 tok/s

llama 270.1 tok/s

Decode 26%

ZINC 16.1 tok/s

llama 62.7 tok/s

Overall 20%

ZINC 16.6 tok/s

llama 84.7 tok/s

Prompt+decode. Cold latency: ZINC 61223 ms · llama.cpp 1997 ms

Gemma 4 31B Q4K M 49 tok prompt · 96 tok output

Compared

Prefill 27%

ZINC 17.8 tok/s

llama 65.9 tok/s

Decode 93%

ZINC 16.0 tok/s

llama 17.3 tok/s

Overall 72%

ZINC 16.5 tok/s

llama 23.0 tok/s

Prompt+decode. Cold latency: ZINC 15542 ms · llama.cpp 6636 ms

Qwen 3.5 9B Q4K M 36 tok prompt · 96 tok output

Compared

Prefill 48%

ZINC 67.8 tok/s

llama 142.2 tok/s

Decode 106%

ZINC 56.7 tok/s

llama 53.4 tok/s

Overall 92%

ZINC 59.3 tok/s

llama 64.3 tok/s

Prompt+decode. Cold latency: ZINC 4435 ms · llama.cpp 2300 ms

Qwen 3.6 27B Q4K M 36 tok prompt · 96 tok output

Compared

Prefill 54%

ZINC 19.8 tok/s

llama 36.9 tok/s

Decode 95%

ZINC 18.1 tok/s

llama 19.1 tok/s

Overall 84%

ZINC 18.5 tok/s

llama 22.0 tok/s

Prompt+decode. Cold latency: ZINC 16372 ms · llama.cpp 6261 ms

Qwen 3.6 35B A3B Q4K XL 36 tok prompt · 2 tok output/96 tok cap

Compared

Prefill 51%

ZINC 68.0 tok/s

llama 132.3 tok/s

Decode 140%

ZINC 104.0 tok/s

llama 74.5 tok/s

Overall 54%

ZINC 69.3 tok/s

llama 127.1 tok/s

Prompt+decode. Cold latency: ZINC 11442 ms · llama.cpp 1860 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 49 tok prompt · 96 tok output · 3 runs · Compared	Z 17.5 tok/s 2794.5 ms L 270.1 tok/s 192.5 ms 6% -252.6 tok/s	Z 16.1 tok/s 62.0 ms/tok L 62.7 tok/s 16.0 ms/tok 26% -46.6 tok/s	Z 16.6 tok/s prompt+decode L 84.7 tok/s prompt+decode 20% -68.1 tok/s 59227 ms slower
Gemma 4 31B Q4K M 49 tok prompt · 96 tok output · 3 runs · Compared	Z 17.8 tok/s 2751.0 ms L 65.9 tok/s 789.1 ms 27% -48.1 tok/s	Z 16.0 tok/s 62.6 ms/tok L 17.3 tok/s 58.0 ms/tok 93% -1.3 tok/s	Z 16.5 tok/s prompt+decode L 23.0 tok/s prompt+decode 72% -6.4 tok/s 8905 ms slower
Qwen 3.5 9B Q4K M 36 tok prompt · 96 tok output · 3 runs · Compared	Z 67.8 tok/s 531.3 ms L 142.2 tok/s 246.1 ms 48% -74.4 tok/s	Z 56.7 tok/s 17.6 ms/tok L 53.4 tok/s 18.7 ms/tok 106% +3.3 tok/s	Z 59.3 tok/s prompt+decode L 64.3 tok/s prompt+decode 92% -5.0 tok/s 2135 ms slower
Qwen 3.6 27B Q4K M 36 tok prompt · 96 tok output · 3 runs · Compared	Z 19.8 tok/s 1822.9 ms L 36.9 tok/s 948.7 ms 54% -17.1 tok/s	Z 18.1 tok/s 55.2 ms/tok L 19.1 tok/s 52.4 ms/tok 95% -1.0 tok/s	Z 18.5 tok/s prompt+decode L 22.0 tok/s prompt+decode 84% -3.5 tok/s 10111 ms slower
Qwen 3.6 35B A3B Q4K XL 36 tok prompt · 2 tok output/96 tok cap · 3 runs · Compared	Z 68.0 tok/s 529.4 ms L 132.3 tok/s 264.5 ms 51% -64.3 tok/s	Z 104.0 tok/s 9.6 ms/tok L 74.5 tok/s 13.4 ms/tok 140% +29.5 tok/s	Z 69.3 tok/s prompt+decode L 127.1 tok/s prompt+decode 54% -57.9 tok/s 9582 ms slower

Decode 39.6 tok/s 91% decode ratio

Prefill 48.1 tok/s 68% prefill ratio · 181 prompt tok

Fastest Qwen 3.6 35B A3B Q4K XL 64.5 tok/s

Overall 78% best row: Qwen 3.5 9B Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 192p · 160out Compared

Prefill 64%

ZINC 62.8 tok/s

llama 98.5 tok/s

Decode 74%

ZINC 44.5 tok/s

llama 59.9 tok/s

69% overall

Gemma 4 31B Q4K M 192p · 160out Compared

Prefill 68%

ZINC 17.5 tok/s

llama 25.9 tok/s

Decode 95%

ZINC 15.8 tok/s

llama 16.5 tok/s

81% overall

Qwen 3.5 9B Q4K M 174p · 160out Compared

Prefill 74%

ZINC 70.3 tok/s

llama 94.5 tok/s

Decode 105%

ZINC 55.5 tok/s

llama 52.6 tok/s

91% overall

Qwen 3.6 27B Q4K M 174p · 160out Compared

Prefill 63%

ZINC 20.0 tok/s

llama 32.0 tok/s

Decode 95%

ZINC 17.9 tok/s

llama 18.9 tok/s

79% overall

Qwen 3.6 35B A3B Q4K XL 174p · 5out/160cap Compared

Prefill 70%

ZINC 70.0 tok/s

llama 100.7 tok/s

Decode 88%

ZINC 64.5 tok/s

llama 73.6 tok/s

70% overall

Details

Gemma 4 26B-A4B MoE Q4K M 192 tok prompt · 160 tok output

Compared

Prefill 64%

ZINC 62.8 tok/s

llama 98.5 tok/s

Decode 74%

ZINC 44.5 tok/s

llama 59.9 tok/s

Overall 69%

ZINC 52.9 tok/s

llama 76.1 tok/s

Prompt+decode. Cold latency: ZINC 12082 ms · llama.cpp 2908 ms

Gemma 4 31B Q4K M 192 tok prompt · 160 tok output

Compared

Prefill 68%

ZINC 17.5 tok/s

llama 25.9 tok/s

Decode 95%

ZINC 15.8 tok/s

llama 16.5 tok/s

Overall 81%

ZINC 16.7 tok/s

llama 20.6 tok/s

Prompt+decode. Cold latency: ZINC 27690 ms · llama.cpp 10149 ms

Qwen 3.5 9B Q4K M 174 tok prompt · 160 tok output

Compared

Prefill 74%

ZINC 70.3 tok/s

llama 94.5 tok/s

Decode 105%

ZINC 55.5 tok/s

llama 52.6 tok/s

Overall 91%

ZINC 62.3 tok/s

llama 68.4 tok/s

Prompt+decode. Cold latency: ZINC 7505 ms · llama.cpp 3380 ms

Qwen 3.6 27B Q4K M 174 tok prompt · 160 tok output

Compared

Prefill 63%

ZINC 20.0 tok/s

llama 32.0 tok/s

Decode 95%

ZINC 17.9 tok/s

llama 18.9 tok/s

Overall 79%

ZINC 18.9 tok/s

llama 24.1 tok/s

Prompt+decode. Cold latency: ZINC 23829 ms · llama.cpp 8914 ms

Qwen 3.6 35B A3B Q4K XL 174 tok prompt · 5 tok output/160 tok cap

Compared

Prefill 70%

ZINC 70.0 tok/s

llama 100.7 tok/s

Decode 88%

ZINC 64.5 tok/s

llama 73.6 tok/s

Overall 70%

ZINC 69.9 tok/s

llama 99.7 tok/s

Prompt+decode. Cold latency: ZINC 10462 ms · llama.cpp 2480 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 192 tok prompt · 160 tok output · 3 runs · Compared	Z 62.8 tok/s 3059.6 ms L 98.5 tok/s 1979.8 ms 64% -35.7 tok/s	Z 44.5 tok/s 22.5 ms/tok L 59.9 tok/s 16.7 ms/tok 74% -15.3 tok/s	Z 52.9 tok/s prompt+decode L 76.1 tok/s prompt+decode 69% -23.2 tok/s 9173 ms slower
Gemma 4 31B Q4K M 192 tok prompt · 160 tok output · 3 runs · Compared	Z 17.5 tok/s 10965.3 ms L 25.9 tok/s 7533.5 ms 68% -8.4 tok/s	Z 15.8 tok/s 63.5 ms/tok L 16.5 tok/s 60.6 ms/tok 95% -0.7 tok/s	Z 16.7 tok/s prompt+decode L 20.6 tok/s prompt+decode 81% -3.9 tok/s 17541 ms slower
Qwen 3.5 9B Q4K M 174 tok prompt · 160 tok output · 3 runs · Compared	Z 70.3 tok/s 2476.9 ms L 94.5 tok/s 1618.4 ms 74% -24.3 tok/s	Z 55.5 tok/s 18.0 ms/tok L 52.6 tok/s 19.0 ms/tok 105% +2.8 tok/s	Z 62.3 tok/s prompt+decode L 68.4 tok/s prompt+decode 91% -6.1 tok/s 4125 ms slower
Qwen 3.6 27B Q4K M 174 tok prompt · 160 tok output · 3 runs · Compared	Z 20.0 tok/s 8688.9 ms L 32.0 tok/s 4774.8 ms 63% -12.0 tok/s	Z 17.9 tok/s 55.9 ms/tok L 18.9 tok/s 52.8 ms/tok 95% -1.0 tok/s	Z 18.9 tok/s prompt+decode L 24.1 tok/s prompt+decode 79% -5.1 tok/s 14915 ms slower
Qwen 3.6 35B A3B Q4K XL 174 tok prompt · 5 tok output/160 tok cap · 3 runs · Compared	Z 70.0 tok/s 2485.1 ms L 100.7 tok/s 1519.3 ms 70% -30.7 tok/s	Z 64.5 tok/s 15.5 ms/tok L 73.6 tok/s 13.6 ms/tok 88% -9.1 tok/s	Z 69.9 tok/s prompt+decode L 99.7 tok/s prompt+decode 70% -29.8 tok/s 7982 ms slower

Decode 36.8 tok/s 88% decode ratio

Prefill 48.2 tok/s 71% prefill ratio · 332 prompt tok

Fastest Qwen 3.5 9B Q4K M 55.0 tok/s

Overall 76% best row: Qwen 3.5 9B Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 346p · 128out Compared

Prefill 66%

ZINC 62.6 tok/s

llama 94.8 tok/s

Decode 74%

ZINC 43.7 tok/s

llama 59.2 tok/s

69% overall

Gemma 4 31B Q4K M 346p · 128out Compared

Prefill 73%

ZINC 17.4 tok/s

llama 23.8 tok/s

Decode 96%

ZINC 15.6 tok/s

llama 16.2 tok/s

80% overall

Qwen 3.5 9B Q4K M 322p · 128out Compared

Prefill 80%

ZINC 70.5 tok/s

llama 88.2 tok/s

Decode 105%

ZINC 55.0 tok/s

llama 52.5 tok/s

88% overall

Qwen 3.6 27B Q4K M 322p · 128out Compared

Prefill 62%

ZINC 20.0 tok/s

llama 32.1 tok/s

Decode 94%

ZINC 17.8 tok/s

llama 18.9 tok/s

72% overall

Qwen 3.6 35B A3B Q4K XL 322p · 128out Compared

Prefill 72%

ZINC 70.3 tok/s

llama 97.6 tok/s

Decode 71%

ZINC 51.8 tok/s

llama 73.4 tok/s

72% overall

Details

Gemma 4 26B-A4B MoE Q4K M 346 tok prompt · 128 tok output

Compared

Prefill 66%

ZINC 62.6 tok/s

llama 94.8 tok/s

Decode 74%

ZINC 43.7 tok/s

llama 59.2 tok/s

Overall 69%

ZINC 56.1 tok/s

llama 81.6 tok/s

Prompt+decode. Cold latency: ZINC 13847 ms · llama.cpp 2470 ms

Gemma 4 31B Q4K M 346 tok prompt · 128 tok output

Compared

Prefill 73%

ZINC 17.4 tok/s

llama 23.8 tok/s

Decode 96%

ZINC 15.6 tok/s

llama 16.2 tok/s

Overall 80%

ZINC 16.9 tok/s

llama 21.1 tok/s

Prompt+decode. Cold latency: ZINC 34975 ms · llama.cpp 8390 ms

Qwen 3.5 9B Q4K M 322 tok prompt · 128 tok output

Compared

Prefill 80%

ZINC 70.5 tok/s

llama 88.2 tok/s

Decode 105%

ZINC 55.0 tok/s

llama 52.5 tok/s

Overall 88%

ZINC 65.3 tok/s

llama 73.9 tok/s

Prompt+decode. Cold latency: ZINC 9027 ms · llama.cpp 2717 ms

Qwen 3.6 27B Q4K M 322 tok prompt · 128 tok output

Compared

Prefill 62%

ZINC 20.0 tok/s

llama 32.1 tok/s

Decode 94%

ZINC 17.8 tok/s

llama 18.9 tok/s

Overall 72%

ZINC 19.3 tok/s

llama 26.8 tok/s

Prompt+decode. Cold latency: ZINC 30135 ms · llama.cpp 7218 ms

Qwen 3.6 35B A3B Q4K XL 322 tok prompt · 128 tok output

Compared

Prefill 72%

ZINC 70.3 tok/s

llama 97.6 tok/s

Decode 71%

ZINC 51.8 tok/s

llama 73.4 tok/s

Overall 72%

ZINC 63.8 tok/s

llama 89.2 tok/s

Prompt+decode. Cold latency: ZINC 14642 ms · llama.cpp 2032 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 346 tok prompt · 128 tok output · 3 runs · Compared	Z 62.6 tok/s 5526.6 ms L 94.8 tok/s 3639.0 ms 66% -32.2 tok/s	Z 43.7 tok/s 22.9 ms/tok L 59.2 tok/s 16.9 ms/tok 74% -15.5 tok/s	Z 56.1 tok/s prompt+decode L 81.6 tok/s prompt+decode 69% -25.5 tok/s 11378 ms slower
Gemma 4 31B Q4K M 346 tok prompt · 128 tok output · 3 runs · Compared	Z 17.4 tok/s 19856.1 ms L 23.8 tok/s 14499.5 ms 73% -6.4 tok/s	Z 15.6 tok/s 64.3 ms/tok L 16.2 tok/s 61.8 ms/tok 96% -0.6 tok/s	Z 16.9 tok/s prompt+decode L 21.1 tok/s prompt+decode 80% -4.2 tok/s 26585 ms slower
Qwen 3.5 9B Q4K M 322 tok prompt · 128 tok output · 3 runs · Compared	Z 70.5 tok/s 4567.8 ms L 88.2 tok/s 3571.7 ms 80% -17.7 tok/s	Z 55.0 tok/s 18.2 ms/tok L 52.5 tok/s 19.0 ms/tok 105% +2.5 tok/s	Z 65.3 tok/s prompt+decode L 73.9 tok/s prompt+decode 88% -8.7 tok/s 6310 ms slower
Qwen 3.6 27B Q4K M 322 tok prompt · 128 tok output · 3 runs · Compared	Z 20.0 tok/s 16110.9 ms L 32.1 tok/s 9811.9 ms 62% -12.1 tok/s	Z 17.8 tok/s 56.1 ms/tok L 18.9 tok/s 53.0 ms/tok 94% -1.1 tok/s	Z 19.3 tok/s prompt+decode L 26.8 tok/s prompt+decode 72% -7.4 tok/s 22917 ms slower
Qwen 3.6 35B A3B Q4K XL 322 tok prompt · 128 tok output · 3 runs · Compared	Z 70.3 tok/s 4580.4 ms L 97.6 tok/s 3227.9 ms 72% -27.3 tok/s	Z 51.8 tok/s 19.3 ms/tok L 73.4 tok/s 13.6 ms/tok 71% -21.6 tok/s	Z 63.8 tok/s prompt+decode L 89.2 tok/s prompt+decode 72% -25.4 tok/s 12610 ms slower

Decode 37.2 tok/s 87% decode ratio

Prefill 47.9 tok/s 23% prefill ratio · 66 prompt tok

Fastest Qwen 3.5 9B Q4K M 55.4 tok/s

Overall 76% best row: Qwen 3.5 9B Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 70p · 256out Compared

Prefill 19%

ZINC 63.3 tok/s

llama 331.1 tok/s

Decode 74%

ZINC 44.7 tok/s

llama 60.8 tok/s

65% overall

Gemma 4 31B Q4K M 70p · 256out Compared

Prefill 21%

ZINC 17.7 tok/s

llama 84.5 tok/s

Decode 94%

ZINC 15.8 tok/s

llama 16.8 tok/s

80% overall

Qwen 3.5 9B Q4K M 64p · 256out Compared

Prefill 24%

ZINC 69.1 tok/s

llama 287.3 tok/s

Decode 105%

ZINC 55.4 tok/s

llama 52.8 tok/s

91% overall

Qwen 3.6 27B Q4K M 64p · 256out Compared

Prefill 25%

ZINC 19.9 tok/s

llama 80.6 tok/s

Decode 94%

ZINC 17.9 tok/s

llama 18.9 tok/s

82% overall

Qwen 3.6 35B A3B Q4K XL 64p · 256out Compared

Prefill 26%

ZINC 69.3 tok/s

llama 271.0 tok/s

Decode 70%

ZINC 52.1 tok/s

llama 74.2 tok/s

63% overall

Details

Gemma 4 26B-A4B MoE Q4K M 70 tok prompt · 256 tok output

Compared

Prefill 19%

ZINC 63.3 tok/s

llama 331.1 tok/s

Decode 74%

ZINC 44.7 tok/s

llama 60.8 tok/s

Overall 65%

ZINC 47.7 tok/s

llama 73.7 tok/s

Prompt+decode. Cold latency: ZINC 12290 ms · llama.cpp 4670 ms

Gemma 4 31B Q4K M 70 tok prompt · 256 tok output

Compared

Prefill 21%

ZINC 17.7 tok/s

llama 84.5 tok/s

Decode 94%

ZINC 15.8 tok/s

llama 16.8 tok/s

Overall 80%

ZINC 16.1 tok/s

llama 20.2 tok/s

Prompt+decode. Cold latency: ZINC 27118 ms · llama.cpp 16368 ms

Qwen 3.5 9B Q4K M 64 tok prompt · 256 tok output

Compared

Prefill 24%

ZINC 69.1 tok/s

llama 287.3 tok/s

Decode 105%

ZINC 55.4 tok/s

llama 52.8 tok/s

Overall 91%

ZINC 57.6 tok/s

llama 63.1 tok/s

Prompt+decode. Cold latency: ZINC 7767 ms · llama.cpp 5318 ms

Qwen 3.6 27B Q4K M 64 tok prompt · 256 tok output

Compared

Prefill 25%

ZINC 19.9 tok/s

llama 80.6 tok/s

Decode 94%

ZINC 17.9 tok/s

llama 18.9 tok/s

Overall 82%

ZINC 18.2 tok/s

llama 22.3 tok/s

Prompt+decode. Cold latency: ZINC 24230 ms · llama.cpp 14621 ms

Qwen 3.6 35B A3B Q4K XL 64 tok prompt · 256 tok output

Compared

Prefill 26%

ZINC 69.3 tok/s

llama 271.0 tok/s

Decode 70%

ZINC 52.1 tok/s

llama 74.2 tok/s

Overall 63%

ZINC 54.8 tok/s

llama 86.8 tok/s

Prompt+decode. Cold latency: ZINC 13455 ms · llama.cpp 3996 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 70 tok prompt · 256 tok output · 3 runs · Compared	Z 63.3 tok/s 1106.1 ms L 331.1 tok/s 220.5 ms 19% -267.8 tok/s	Z 44.7 tok/s 22.4 ms/tok L 60.8 tok/s 16.5 ms/tok 74% -16.1 tok/s	Z 47.7 tok/s prompt+decode L 73.7 tok/s prompt+decode 65% -26.0 tok/s 7621 ms slower
Gemma 4 31B Q4K M 70 tok prompt · 256 tok output · 3 runs · Compared	Z 17.7 tok/s 3956.2 ms L 84.5 tok/s 863.7 ms 21% -66.8 tok/s	Z 15.8 tok/s 63.4 ms/tok L 16.8 tok/s 59.7 ms/tok 94% -1.0 tok/s	Z 16.1 tok/s prompt+decode L 20.2 tok/s prompt+decode 80% -4.1 tok/s 10749 ms slower
Qwen 3.5 9B Q4K M 64 tok prompt · 256 tok output · 3 runs · Compared	Z 69.1 tok/s 926.4 ms L 287.3 tok/s 219.3 ms 24% -218.2 tok/s	Z 55.4 tok/s 18.1 ms/tok L 52.8 tok/s 18.9 ms/tok 105% +2.5 tok/s	Z 57.6 tok/s prompt+decode L 63.1 tok/s prompt+decode 91% -5.5 tok/s 2449 ms slower
Qwen 3.6 27B Q4K M 64 tok prompt · 256 tok output · 3 runs · Compared	Z 19.9 tok/s 3217.7 ms L 80.6 tok/s 781.2 ms 25% -60.8 tok/s	Z 17.9 tok/s 56.0 ms/tok L 18.9 tok/s 52.9 ms/tok 94% -1.0 tok/s	Z 18.2 tok/s prompt+decode L 22.3 tok/s prompt+decode 82% -4.1 tok/s 9609 ms slower
Qwen 3.6 35B A3B Q4K XL 64 tok prompt · 256 tok output · 3 runs · Compared	Z 69.3 tok/s 923.3 ms L 271.0 tok/s 232.4 ms 26% -201.7 tok/s	Z 52.1 tok/s 19.2 ms/tok L 74.2 tok/s 13.5 ms/tok 70% -22.1 tok/s	Z 54.8 tok/s prompt+decode L 86.8 tok/s prompt+decode 63% -32.0 tok/s 9459 ms slower

Methodology ZINC CLI vs llama.cpp on the same Intel Arc node · Four-scenario matrix with same-file baselines

3 measured · 1 warmup

Benchmark runner Exact prompts and token caps Prompt matrix test Published JSON

Hardware: one Intel Arc Vulkan benchmark node. ZINC and llama.cpp run on the same Ubuntu host and use the same GGUF file for each model.
ZINC path: optionally sync the current source tree to the Intel node, build with zig build -Doptimize=ReleaseFast, then measure generation through the ZINC CLI.
Baseline path: launch llama.cpp against the same model file, preferring one reusable llama-server per model across the full scenario matrix and falling back to llama-cli when the server path is unavailable or fails to start.
Scenarios: Quick Chat, Coding Review, Incident Context, and Long Coding Draft. The prompts are real-world chat, code-review, support-context, and coding-plan workloads instead of synthetic factual completions.
Statistics: one warmup pass is discarded, then three measured runs are collected. Published prefill, decode, end-to-end throughput, and latency values are medians.
Execution order: the harness records the full ZINC scenario matrix before starting the llama.cpp baseline phase, so only one inference engine owns the GPU during measurement.
Run hygiene: published Intel runs assume a clean node with no competing ZINC, llama.cpp, or other GPU workloads.

NVIDIA CUDA bench node

RTX 5090

GeForce RTX 5090 · 32 GB VRAM · 1792 GB/s · 3 measured · 1 warmup

Run info

GPU: GeForce RTX 5090
Chip: NVIDIA GeForce RTX 5090
Memory: 32 GB VRAM · 1792 GB/s
Artifact: Jun 12, 2026, 1:46 PM
ZINC: dcc7cc01
llama.cpp: 1

Fastest 85.5 tok/s Qwen 3.5 9B Q4K M

Decode 51.0 tok/s average

Overall 78% prefill + decode vs llama.cpp

Decode 51.0 tok/s 60% decode ratio

Prefill 48.5 tok/s 78% prefill ratio · 41 prompt tok

Fastest Qwen 3.5 9B Q4K M 85.5 tok/s

Overall 64% best row: Qwen 3.6 27B Dense Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 49p · 96out Compared

Prefill 11%

ZINC 40.8 tok/s

llama 382.6 tok/s

Decode 39%

ZINC 42.4 tok/s

llama 109.4 tok/s

29% overall

Gemma 4 31B Q4K M 49p · 96out Compared

Prefill 13%

ZINC 36.7 tok/s

llama 291.7 tok/s

Decode 75%

ZINC 39.8 tok/s

llama 53.0 tok/s

53% overall

Qwen 3.5 9B Q4K M 36p · 96out Compared

Prefill 155%

ZINC 89.3 tok/s

llama 57.5 tok/s

Decode 74%

ZINC 85.5 tok/s

llama 115.7 tok/s

95% overall

Qwen 3.6 27B Dense Q4K M 36p · 96out Compared

Prefill 139%

ZINC 41.0 tok/s

llama 29.6 tok/s

Decode 79%

ZINC 45.4 tok/s

llama 57.6 tok/s

96% overall

Qwen 3.6 35B A3B UD Q4K XL 36p · 96out Compared

Prefill 71%

ZINC 34.6 tok/s

llama 48.8 tok/s

Decode 33%

ZINC 42.0 tok/s

llama 128.4 tok/s

45% overall

Details

Gemma 4 26B-A4B MoE Q4K M 49 tok prompt · 96 tok output

Compared

Prefill 11%

ZINC 40.8 tok/s

llama 382.6 tok/s

Decode 39%

ZINC 42.4 tok/s

llama 109.4 tok/s

Overall 29%

ZINC 41.9 tok/s

llama 144.2 tok/s

Prompt+decode. Cold latency: ZINC 18473 ms · llama.cpp 2300 ms

Gemma 4 31B Q4K M 49 tok prompt · 96 tok output

Compared

Prefill 13%

ZINC 36.7 tok/s

llama 291.7 tok/s

Decode 75%

ZINC 39.8 tok/s

llama 53.0 tok/s

Overall 53%

ZINC 38.7 tok/s

llama 73.3 tok/s

Prompt+decode. Cold latency: ZINC 20510 ms · llama.cpp 3403 ms

Qwen 3.5 9B Q4K M 36 tok prompt · 96 tok output

Compared

Prefill 155%

ZINC 89.3 tok/s

llama 57.5 tok/s

Decode 74%

ZINC 85.5 tok/s

llama 115.7 tok/s

Overall 95%

ZINC 86.5 tok/s

llama 90.7 tok/s

Prompt+decode. Cold latency: ZINC 11115 ms · llama.cpp 2248 ms

Qwen 3.6 27B Dense Q4K M 36 tok prompt · 96 tok output

Compared

Prefill 139%

ZINC 41.0 tok/s

llama 29.6 tok/s

Decode 79%

ZINC 45.4 tok/s

llama 57.6 tok/s

Overall 96%

ZINC 44.1 tok/s

llama 45.8 tok/s

Prompt+decode. Cold latency: ZINC 19392 ms · llama.cpp 3258 ms

Qwen 3.6 35B A3B UD Q4K XL 36 tok prompt · 96 tok output

Compared

Prefill 71%

ZINC 34.6 tok/s

llama 48.8 tok/s

Decode 33%

ZINC 42.0 tok/s

llama 128.4 tok/s

Overall 45%

ZINC 39.7 tok/s

llama 88.8 tok/s

Prompt+decode. Cold latency: ZINC 22444 ms · llama.cpp 2265 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 49 tok prompt · 96 tok output · 3 runs · Compared	Z 40.8 tok/s 1201.4 ms L 382.6 tok/s 135.9 ms 11% -341.8 tok/s	Z 42.4 tok/s 23.6 ms/tok L 109.4 tok/s 9.1 ms/tok 39% -67.0 tok/s	Z 41.9 tok/s prompt+decode L 144.2 tok/s prompt+decode 29% -102.4 tok/s 16173 ms slower
Gemma 4 31B Q4K M 49 tok prompt · 96 tok output · 3 runs · Compared	Z 36.7 tok/s 1336.5 ms L 291.7 tok/s 178.2 ms 13% -255.1 tok/s	Z 39.8 tok/s 25.1 ms/tok L 53.0 tok/s 18.9 ms/tok 75% -13.2 tok/s	Z 38.7 tok/s prompt+decode L 73.3 tok/s prompt+decode 53% -34.6 tok/s 17107 ms slower
Qwen 3.5 9B Q4K M 36 tok prompt · 96 tok output · 3 runs · Compared	Z 89.3 tok/s 403.3 ms L 57.5 tok/s 608.5 ms 155% +31.7 tok/s	Z 85.5 tok/s 11.7 ms/tok L 115.7 tok/s 8.6 ms/tok 74% -30.1 tok/s	Z 86.5 tok/s prompt+decode L 90.7 tok/s prompt+decode 95% -4.1 tok/s 8868 ms slower
Qwen 3.6 27B Dense Q4K M 36 tok prompt · 96 tok output · 3 runs · Compared	Z 41.0 tok/s 878.7 ms L 29.6 tok/s 1183.3 ms 139% +11.4 tok/s	Z 45.4 tok/s 22.1 ms/tok L 57.6 tok/s 17.4 ms/tok 79% -12.2 tok/s	Z 44.1 tok/s prompt+decode L 45.8 tok/s prompt+decode 96% -1.7 tok/s 16134 ms slower
Qwen 3.6 35B A3B UD Q4K XL 36 tok prompt · 96 tok output · 3 runs · Compared	Z 34.6 tok/s 1040.3 ms L 48.8 tok/s 717.5 ms 71% -14.2 tok/s	Z 42.0 tok/s 23.8 ms/tok L 128.4 tok/s 7.8 ms/tok 33% -86.4 tok/s	Z 39.7 tok/s prompt+decode L 88.8 tok/s prompt+decode 45% -49.2 tok/s 20179 ms slower

Decode 53.4 tok/s 63% decode ratio

Prefill 54.2 tok/s 85% prefill ratio · 181 prompt tok

Fastest Qwen 3.5 9B Q4K M 93.0 tok/s

Overall 73% best row: Qwen 3.6 27B Dense Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 192p · 160out Compared

Prefill 3%

ZINC 46.9 tok/s

llama 1456.6 tok/s

Decode 45%

ZINC 48.0 tok/s

llama 107.8 tok/s

22% overall

Gemma 4 31B Q4K M 192p · 160out Compared

Prefill 4%

ZINC 39.8 tok/s

llama 1006.4 tok/s

Decode 74%

ZINC 38.6 tok/s

llama 51.9 tok/s

36% overall

Qwen 3.5 9B Q4K M 174p · 160out Compared

Prefill 154%

ZINC 93.8 tok/s

llama 60.8 tok/s

Decode 81%

ZINC 93.0 tok/s

llama 115.4 tok/s

119% overall

Qwen 3.6 27B Dense Q4K M 174p · 160out Compared

Prefill 173%

ZINC 48.3 tok/s

llama 28.0 tok/s

Decode 82%

ZINC 46.1 tok/s

llama 56.5 tok/s

128% overall

Qwen 3.6 35B A3B UD Q4K XL 174p · 160out Compared

Prefill 90%

ZINC 42.2 tok/s

llama 47.2 tok/s

Decode 34%

ZINC 41.4 tok/s

llama 122.9 tok/s

63% overall

Details

Gemma 4 26B-A4B MoE Q4K M 192 tok prompt · 160 tok output

Compared

Prefill 3%

ZINC 46.9 tok/s

llama 1456.6 tok/s

Decode 45%

ZINC 48.0 tok/s

llama 107.8 tok/s

Overall 22%

ZINC 47.4 tok/s

llama 217.8 tok/s

Prompt+decode. Cold latency: ZINC 22668 ms · llama.cpp 2908 ms

Gemma 4 31B Q4K M 192 tok prompt · 160 tok output

Compared

Prefill 4%

ZINC 39.8 tok/s

llama 1006.4 tok/s

Decode 74%

ZINC 38.6 tok/s

llama 51.9 tok/s

Overall 36%

ZINC 39.2 tok/s

llama 107.5 tok/s

Prompt+decode. Cold latency: ZINC 26646 ms · llama.cpp 4644 ms

Qwen 3.5 9B Q4K M 174 tok prompt · 160 tok output

Compared

Prefill 154%

ZINC 93.8 tok/s

llama 60.8 tok/s

Decode 81%

ZINC 93.0 tok/s

llama 115.4 tok/s

Overall 119%

ZINC 93.4 tok/s

llama 78.6 tok/s

Prompt+decode. Cold latency: ZINC 13260 ms · llama.cpp 2932 ms

Qwen 3.6 27B Dense Q4K M 174 tok prompt · 160 tok output

Compared

Prefill 173%

ZINC 48.3 tok/s

llama 28.0 tok/s

Decode 82%

ZINC 46.1 tok/s

llama 56.5 tok/s

Overall 128%

ZINC 47.2 tok/s

llama 36.9 tok/s

Prompt+decode. Cold latency: ZINC 23038 ms · llama.cpp 4339 ms

Qwen 3.6 35B A3B UD Q4K XL 174 tok prompt · 160 tok output

Compared

Prefill 90%

ZINC 42.2 tok/s

llama 47.2 tok/s

Decode 34%

ZINC 41.4 tok/s

llama 122.9 tok/s

Overall 63%

ZINC 41.8 tok/s

llama 66.9 tok/s

Prompt+decode. Cold latency: ZINC 26685 ms · llama.cpp 3095 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 192 tok prompt · 160 tok output · 3 runs · Compared	Z 46.9 tok/s 4097.6 ms L 1456.6 tok/s 133.9 ms 3% -1409.8 tok/s	Z 48.0 tok/s 20.8 ms/tok L 107.8 tok/s 9.3 ms/tok 45% -59.8 tok/s	Z 47.4 tok/s prompt+decode L 217.8 tok/s prompt+decode 22% -170.5 tok/s 19759 ms slower
Gemma 4 31B Q4K M 192 tok prompt · 160 tok output · 3 runs · Compared	Z 39.8 tok/s 4828.0 ms L 1006.4 tok/s 193.8 ms 4% -966.6 tok/s	Z 38.6 tok/s 25.9 ms/tok L 51.9 tok/s 19.3 ms/tok 74% -13.3 tok/s	Z 39.2 tok/s prompt+decode L 107.5 tok/s prompt+decode 36% -68.3 tok/s 22002 ms slower
Qwen 3.5 9B Q4K M 174 tok prompt · 160 tok output · 3 runs · Compared	Z 93.8 tok/s 1855.2 ms L 60.8 tok/s 2515.4 ms 154% +33.0 tok/s	Z 93.0 tok/s 10.8 ms/tok L 115.4 tok/s 8.7 ms/tok 81% -22.4 tok/s	Z 93.4 tok/s prompt+decode L 78.6 tok/s prompt+decode 119% +14.8 tok/s 10328 ms slower
Qwen 3.6 27B Dense Q4K M 174 tok prompt · 160 tok output · 3 runs · Compared	Z 48.3 tok/s 3605.2 ms L 28.0 tok/s 5471.4 ms 173% +20.3 tok/s	Z 46.1 tok/s 21.7 ms/tok L 56.5 tok/s 17.7 ms/tok 82% -10.4 tok/s	Z 47.2 tok/s prompt+decode L 36.9 tok/s prompt+decode 128% +10.3 tok/s 18699 ms slower
Qwen 3.6 35B A3B UD Q4K XL 174 tok prompt · 160 tok output · 3 runs · Compared	Z 42.2 tok/s 4119.3 ms L 47.2 tok/s 3243.9 ms 90% -4.9 tok/s	Z 41.4 tok/s 24.1 ms/tok L 122.9 tok/s 8.1 ms/tok 34% -81.5 tok/s	Z 41.8 tok/s prompt+decode L 66.9 tok/s prompt+decode 63% -25.1 tok/s 23590 ms slower

Decode 51.8 tok/s 60% decode ratio

Prefill 53.7 tok/s 130% prefill ratio · 332 prompt tok

Fastest Qwen 3.5 9B Q4K M 88.8 tok/s

Overall 110% best row: Qwen 3.6 27B Dense Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 346p · 128out Compared

Prefill 104%

ZINC 48.8 tok/s

llama 46.8 tok/s

Decode 41%

ZINC 43.9 tok/s

llama 107.4 tok/s

86% overall

Gemma 4 31B Q4K M 346p · 128out Compared

Prefill 155%

ZINC 40.2 tok/s

llama 25.9 tok/s

Decode 71%

ZINC 37.5 tok/s

llama 53.0 tok/s

131% overall

Qwen 3.5 9B Q4K M 322p · 128out Compared

Prefill 144%

ZINC 89.2 tok/s

llama 62.0 tok/s

Decode 77%

ZINC 88.8 tok/s

llama 115.9 tok/s

125% overall

Qwen 3.6 27B Dense Q4K M 322p · 128out Compared

Prefill 159%

ZINC 48.3 tok/s

llama 30.3 tok/s

Decode 79%

ZINC 45.5 tok/s

llama 57.8 tok/s

135% overall

Qwen 3.6 35B A3B UD Q4K XL 322p · 128out Compared

Prefill 90%

ZINC 42.0 tok/s

llama 46.7 tok/s

Decode 33%

ZINC 43.1 tok/s

llama 130.2 tok/s

74% overall

Details

Gemma 4 26B-A4B MoE Q4K M 346 tok prompt · 128 tok output

Compared

Prefill 104%

ZINC 48.8 tok/s

llama 46.8 tok/s

Decode 41%

ZINC 43.9 tok/s

llama 107.4 tok/s

Overall 86%

ZINC 47.4 tok/s

llama 55.2 tok/s

Prompt+decode. Cold latency: ZINC 25316 ms · llama.cpp 2779 ms

Gemma 4 31B Q4K M 346 tok prompt · 128 tok output

Compared

Prefill 155%

ZINC 40.2 tok/s

llama 25.9 tok/s

Decode 71%

ZINC 37.5 tok/s

llama 53.0 tok/s

Overall 131%

ZINC 39.4 tok/s

llama 30.0 tok/s

Prompt+decode. Cold latency: ZINC 29083 ms · llama.cpp 3971 ms

Qwen 3.5 9B Q4K M 322 tok prompt · 128 tok output

Compared

Prefill 144%

ZINC 89.2 tok/s

llama 62.0 tok/s

Decode 77%

ZINC 88.8 tok/s

llama 115.9 tok/s

Overall 125%

ZINC 89.1 tok/s

llama 71.5 tok/s

Prompt+decode. Cold latency: ZINC 14840 ms · llama.cpp 2554 ms

Qwen 3.6 27B Dense Q4K M 322 tok prompt · 128 tok output

Compared

Prefill 159%

ZINC 48.3 tok/s

llama 30.3 tok/s

Decode 79%

ZINC 45.5 tok/s

llama 57.8 tok/s

Overall 135%

ZINC 47.5 tok/s

llama 35.1 tok/s

Prompt+decode. Cold latency: ZINC 24795 ms · llama.cpp 3748 ms

Qwen 3.6 35B A3B UD Q4K XL 322 tok prompt · 128 tok output

Compared

Prefill 90%

ZINC 42.0 tok/s

llama 46.7 tok/s

Decode 33%

ZINC 43.1 tok/s

llama 130.2 tok/s

Overall 74%

ZINC 42.3 tok/s

llama 57.1 tok/s

Prompt+decode. Cold latency: ZINC 29929 ms · llama.cpp 2372 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 346 tok prompt · 128 tok output · 3 runs · Compared	Z 48.8 tok/s 7092.8 ms L 46.8 tok/s 7372.8 ms 104% +2.0 tok/s	Z 43.9 tok/s 22.8 ms/tok L 107.4 tok/s 9.3 ms/tok 41% -63.5 tok/s	Z 47.4 tok/s prompt+decode L 55.2 tok/s prompt+decode 86% -7.8 tok/s 22537 ms slower
Gemma 4 31B Q4K M 346 tok prompt · 128 tok output · 3 runs · Compared	Z 40.2 tok/s 8615.6 ms L 25.9 tok/s 13330.5 ms 155% +14.3 tok/s	Z 37.5 tok/s 26.7 ms/tok L 53.0 tok/s 18.9 ms/tok 71% -15.5 tok/s	Z 39.4 tok/s prompt+decode L 30.0 tok/s prompt+decode 131% +9.4 tok/s 25112 ms slower
Qwen 3.5 9B Q4K M 322 tok prompt · 128 tok output · 3 runs · Compared	Z 89.2 tok/s 3610.7 ms L 62.0 tok/s 5080.1 ms 144% +27.2 tok/s	Z 88.8 tok/s 11.3 ms/tok L 115.9 tok/s 8.6 ms/tok 77% -27.1 tok/s	Z 89.1 tok/s prompt+decode L 71.5 tok/s prompt+decode 125% +17.6 tok/s 12286 ms slower
Qwen 3.6 27B Dense Q4K M 322 tok prompt · 128 tok output · 3 runs · Compared	Z 48.3 tok/s 6669.0 ms L 30.3 tok/s 10384.4 ms 159% +17.9 tok/s	Z 45.5 tok/s 22.0 ms/tok L 57.8 tok/s 17.3 ms/tok 79% -12.3 tok/s	Z 47.5 tok/s prompt+decode L 35.1 tok/s prompt+decode 135% +12.4 tok/s 21047 ms slower
Qwen 3.6 35B A3B UD Q4K XL 322 tok prompt · 128 tok output · 3 runs · Compared	Z 42.0 tok/s 7672.1 ms L 46.7 tok/s 6749.4 ms 90% -4.7 tok/s	Z 43.1 tok/s 23.2 ms/tok L 130.2 tok/s 7.7 ms/tok 33% -87.2 tok/s	Z 42.3 tok/s prompt+decode L 57.1 tok/s prompt+decode 74% -14.8 tok/s 27557 ms slower

Decode 53.0 tok/s 62% decode ratio

Prefill 50.5 tok/s 78% prefill ratio · 66 prompt tok

Fastest Qwen 3.5 9B Q4K M 93.3 tok/s

Overall 65% best row: Qwen 3.6 27B Dense Q4K M

Models

ZINC llama.cpp

Gemma 4 26B-A4B MoE Q4K M 70p · 256out Compared

Prefill 8%

ZINC 42.1 tok/s

llama 516.3 tok/s

Decode 42%

ZINC 45.9 tok/s

llama 109.0 tok/s

34% overall

Gemma 4 31B Q4K M 70p · 256out Compared

Prefill 9%

ZINC 38.0 tok/s

llama 435.3 tok/s

Decode 75%

ZINC 39.4 tok/s

llama 52.1 tok/s

61% overall

Qwen 3.5 9B Q4K M 64p · 256out Compared

Prefill 148%

ZINC 91.7 tok/s

llama 61.8 tok/s

Decode 81%

ZINC 93.3 tok/s

llama 115.1 tok/s

95% overall

Qwen 3.6 27B Dense Q4K M 64p · 256out Compared

Prefill 146%

ZINC 42.9 tok/s

llama 29.3 tok/s

Decode 82%

ZINC 46.7 tok/s

llama 56.9 tok/s

96% overall

Qwen 3.6 35B A3B UD Q4K XL 64p · 256out Compared

Prefill 78%

ZINC 37.6 tok/s

llama 48.0 tok/s

Decode 31%

ZINC 39.5 tok/s

llama 127.3 tok/s

41% overall

Details

Gemma 4 26B-A4B MoE Q4K M 70 tok prompt · 256 tok output

Compared

Prefill 8%

ZINC 42.1 tok/s

llama 516.3 tok/s

Decode 42%

ZINC 45.9 tok/s

llama 109.0 tok/s

Overall 34%

ZINC 45.0 tok/s

llama 131.2 tok/s

Prompt+decode. Cold latency: ZINC 22351 ms · llama.cpp 3841 ms

Gemma 4 31B Q4K M 70 tok prompt · 256 tok output

Compared

Prefill 9%

ZINC 38.0 tok/s

llama 435.3 tok/s

Decode 75%

ZINC 39.4 tok/s

llama 52.1 tok/s

Overall 61%

ZINC 39.1 tok/s

llama 64.3 tok/s

Prompt+decode. Cold latency: ZINC 25310 ms · llama.cpp 6695 ms

Qwen 3.5 9B Q4K M 64 tok prompt · 256 tok output

Compared

Prefill 148%

ZINC 91.7 tok/s

llama 61.8 tok/s

Decode 81%

ZINC 93.3 tok/s

llama 115.1 tok/s

Overall 95%

ZINC 92.9 tok/s

llama 98.2 tok/s

Prompt+decode. Cold latency: ZINC 12892 ms · llama.cpp 3794 ms

Qwen 3.6 27B Dense Q4K M 64 tok prompt · 256 tok output

Compared

Prefill 146%

ZINC 42.9 tok/s

llama 29.3 tok/s

Decode 82%

ZINC 46.7 tok/s

llama 56.9 tok/s

Overall 96%

ZINC 45.9 tok/s

llama 47.9 tok/s

Prompt+decode. Cold latency: ZINC 22046 ms · llama.cpp 6326 ms

Qwen 3.6 35B A3B UD Q4K XL 64 tok prompt · 256 tok output

Compared

Prefill 78%

ZINC 37.6 tok/s

llama 48.0 tok/s

Decode 31%

ZINC 39.5 tok/s

llama 127.3 tok/s

Overall 41%

ZINC 39.1 tok/s

llama 95.7 tok/s

Prompt+decode. Cold latency: ZINC 27139 ms · llama.cpp 3469 ms

Raw data

Model	Prefill	Decode	Overall
Gemma 4 26B-A4B MoE Q4K M 70 tok prompt · 256 tok output · 3 runs · Compared	Z 42.1 tok/s 1660.7 ms L 516.3 tok/s 141.4 ms 8% -474.2 tok/s	Z 45.9 tok/s 21.8 ms/tok L 109.0 tok/s 9.2 ms/tok 42% -63.1 tok/s	Z 45.0 tok/s prompt+decode L 131.2 tok/s prompt+decode 34% -86.2 tok/s 18511 ms slower
Gemma 4 31B Q4K M 70 tok prompt · 256 tok output · 3 runs · Compared	Z 38.0 tok/s 1840.4 ms L 435.3 tok/s 167.7 ms 9% -397.2 tok/s	Z 39.4 tok/s 25.4 ms/tok L 52.1 tok/s 19.2 ms/tok 75% -12.8 tok/s	Z 39.1 tok/s prompt+decode L 64.3 tok/s prompt+decode 61% -25.2 tok/s 18615 ms slower
Qwen 3.5 9B Q4K M 64 tok prompt · 256 tok output · 3 runs · Compared	Z 91.7 tok/s 698.3 ms L 61.8 tok/s 1018.6 ms 148% +29.8 tok/s	Z 93.3 tok/s 10.7 ms/tok L 115.1 tok/s 8.7 ms/tok 81% -21.8 tok/s	Z 92.9 tok/s prompt+decode L 98.2 tok/s prompt+decode 95% -5.3 tok/s 9098 ms slower
Qwen 3.6 27B Dense Q4K M 64 tok prompt · 256 tok output · 3 runs · Compared	Z 42.9 tok/s 1491.7 ms L 29.3 tok/s 2149.9 ms 146% +13.6 tok/s	Z 46.7 tok/s 21.4 ms/tok L 56.9 tok/s 17.6 ms/tok 82% -10.1 tok/s	Z 45.9 tok/s prompt+decode L 47.9 tok/s prompt+decode 96% -1.9 tok/s 15720 ms slower
Qwen 3.6 35B A3B UD Q4K XL 64 tok prompt · 256 tok output · 3 runs · Compared	Z 37.6 tok/s 1701.9 ms L 48.0 tok/s 1312.2 ms 78% -10.4 tok/s	Z 39.5 tok/s 25.3 ms/tok L 127.3 tok/s 7.9 ms/tok 31% -87.7 tok/s	Z 39.1 tok/s prompt+decode L 95.7 tok/s prompt+decode 41% -56.6 tok/s 23670 ms slower

Methodology ZINC CLI vs llama.cpp on the same CUDA node · Four-scenario matrix with same-file baselines

3 measured · 1 warmup

Benchmark runner Exact prompts and token caps Prompt matrix test Published JSON

Hardware: one NVIDIA GeForce RTX 5090 (32 GB VRAM, 1792 GB/s) benchmark node. ZINC and llama.cpp run on the same host and use the same GGUF file for each model.
ZINC path: sync the current source tree to the CUDA node, build with zig build -Doptimize=ReleaseFast -Dbackend=cuda, then measure generation through the ZINC CLI. CUDA kernels are NVRTC-compiled at runtime for the visible GPU.
GPU selection: the target GPU is pinned by UUID via CUDA_VISIBLE_DEVICES for both engines, because nvidia-smi indices are unreliable on the WSL2 passthrough.
Baseline path: launch llama.cpp against the same model file, preferring one reusable llama-server per model across the full scenario matrix and falling back to llama-cli when the server path is unavailable.
Scenarios: Quick Chat, Coding Review, Incident Context, and Long Coding Draft. The prompts are real-world chat, code-review, support-context, and coding-plan workloads instead of synthetic factual completions.
Statistics: one warmup pass is discarded, then three measured runs are collected. Published prefill, decode, end-to-end throughput, and latency values are medians.
Execution order: the harness records the full ZINC scenario matrix before starting the llama.cpp baseline phase, so only one inference engine owns the GPU during measurement.
Run hygiene: published CUDA runs assume a clean node with no competing ZINC, llama.cpp, or other GPU workloads.

Benchmarks

Quick Chat

Coding Review

Incident Context

Long Coding Draft

Quick Chat

Coding Review

Incident Context

Long Coding Draft

Quick Chat

Coding Review

Incident Context

Long Coding Draft

Quick Chat

Coding Review

Incident Context

Long Coding Draft