Last updated: 2026-06-12
Inference Runtime
Bench Hot Decode
Hot-path decode kernel microbenchmarks.
Measures per-dispatch GPU latency, memory bandwidth, and VRAM utilisation for individual compute kernels (DMMV, SSM delta-net) in isolation. Run via `zig build hot-bench -Doptimize=ReleaseFast`.
1 exports shown
function
main
pub fn main() !void Run the hot-decode microbenchmark suite against the selected model and kernels.