What worked, what did not, and what the measurements actually said while building ZINC. The newest entries are shown in full; the rest are grouped into a compact archive.
In July this blog argued a Q8 KV cache is close to free on an RDNA4 card. On August 17 we turned it off by default for Muse Glimmer on Apple Silicon, because it was costing 15 to 22 percent of decode. Both results are correct. Q8_0 stores a KV element in 1.06 bytes instead of 4, so the quantized path reads 3.8 times fewer bytes and still loses, which means the flash-attention kernel at ordinary context lengths is not bandwidth-bound at all. KV quantization is a capacity decision that a decode kernel charges you for in time.
A commit on August 17 made Muse Glimmer decode about six percent faster on an M4 Max without adding a kernel. The faster Q5_K matvec was already compiled, already loaded, and never dispatched, because its threadgroup input cache was declared for K<=4096 and Muse's lm-head is K=6656. The lm-head is 925 MB of the roughly 15 GiB the decoder reads per token, and it was running on a translated shader that reads weights a byte at a time. The interesting part is not the fix, it is why the kernel was marked intentionally unused: a correct measurement about dispatch fusion got generalized into a wrong decision about a kernel that was never about dispatch count.
Qwen 3.8 27B landed on Hugging Face on August 14 and ran through ZINC the next day. Two weeks later, the tuned ROCm path beats llama.cpp ROCm across all four published workloads. This is what architecture reuse bought us, what the new fused RMS-norm and Q8 path changed, and why Metal decode still needs work.
Four posts in this series reached for the same lever: quantize the KV cache to q8, halve every byte, double the headroom. That lever is real, and at 8 bits it is close to free. The trouble starts when the swarm wants more and reaches below 8 bits, because the KV cache is not one number to round off. Keys carry large per-channel outliers and values do not, so a single uniform 4-bit or 2-bit knob falls off a quality cliff that the papers only climb back up with per-channel key quantization and explicit outlier isolation. KIVI keeps 2-bit quality only by quantizing keys per-channel and values per-token; KVQuant holds 3-bit to under 0.1 perplexity loss only with pre-RoPE per-channel keys and dense-and-sparse outlier handling. On a single 16 GB RX 9070 XT that means the free win ends at 8 bits, and everything below it is an engineering project with a per-model accuracy budget, not a config flag.
Yesterday's post found the swarm had about 2 GiB of VRAM headroom on a 16 GB RX 9070 XT, and treated that as scarce. It is scarcer than it needs to be, because a good chunk of it is spent storing the same bytes eight times. Every agent in a coding swarm carries an identical prefix, the system prompt plus the tool schemas, and the naive engine caches a separate copy of that prefix's KV for each of the eight decode slots. A 2,000-token shared prefix is 0.30 GiB per copy at fp16, so eight copies cost 2.44 GiB while one shared copy costs 0.30 GiB. Storing it once reclaims 2.14 GiB, which is roughly the entire headroom the swarm was fighting for. This is the prefix-sharing case an earlier post said a single-user engine could mostly skip, and a swarm cannot.
Yesterday's post argued that preempting a decode slot on an RX 9070 XT is nearly free because the paused turn's KV cache never leaves VRAM. That is true, and it hides a ceiling. A paused turn keeps every byte of its KV resident, so under sustained oversubscription the paused turns stack up in the same 16 GB the running turns are using. The arithmetic is blunt: eight running turns at a 6k-token average already sit at 13.5 GiB, which leaves room for about two average paused turns before the card is full and you are forced back into the swap or recompute you were avoiding. And the turns you actually preempt are the long ones, so a single paused 16k-token refactor can overflow the budget on its own.
On one RX 9070 XT with eight decode slots, the foreground agent usually gets a slot in under a second. The problem is the tail. When it lands behind a long background turn, a slow refactor holding its slot for twenty-plus seconds, it waits the whole thing out: p99 is 6.2 seconds and the worst case is half a minute. That is priority inversion, the same failure that reset Mars Pathfinder, and the fix is not a smarter queue. It is preempting the decode slot at a token boundary, which on RDNA4 costs almost nothing because the paused turn's KV cache never leaves the 16 GB of VRAM.
Once a single RX 9070 XT is past its twelve-agent knee, the card is already delivering every token it can, about 0.88 turns per second. Every agent you add after that buys zero throughput and only lengthens the wait before a turn starts. A discrete-event simulation shows the unbounded queue's p99 wait climbing from 9 to 27 seconds between twelve and twenty-four agents while goodput never moves, which means the fix is not a bigger buffer. It is admission control, and the only real question is whether you shed by count or by delay.