Forward Zinc Rt

pub const m0_max_decode_tokens_default: u32 = 8

Decode-token budget used by the no-layer smoke tail (`generateNoLayer`) and by benchmarks that want a short, bounded run on the scalar reference path.

The full M1 forward paths (`generateScalarHybrid`, `generateScalarDense`) pass the caller-supplied `max_tokens` through unchanged and never consult this constant. Override via ZINC_RT_MAX_DECODE_TOKENS.

pub const Model = struct

Loaded GGUF model plus all the load-time scratch derived from it.

Owns the mmap'd file, the resolved per-layer tensor table, the cached F32 dequants of small norm/SSM-side tensors, and the re-quantized weight blobs the decode path streams in place of the larger source-format weights. A `Model` is built once with `load` and consumed by `generate` / `generateWithOptions`; the GPU rings get the same handle so they can read the underlying bytes too.

pub fn load(path: []const u8, allocator: std.mem.Allocator) !Model

pub fn deinit(self: *Model) void

pub fn validateDecodeGraph(self: *const Model, allocator: std.mem.Allocator) !DecodeGraphSummary

pub const DecodeGraphSummary = struct

Counts produced by `Model.validateDecodeGraph` — how many IR nodes were emitted, and how the per-layer mix (full attention, SSM, MoE) breaks down.

Useful for asserting the lowered graph matches the model's expected shape.

pub const DirectComputeKind = enum

Tag identifying which direct-compute shortcut the active tier executed for the current decode step.

`none` means the scalar host path retired the token; the other variants name the kernel the GPU ring actually ran (a first-element RMSNorm, an argmax, an argmax composed with that RMSNorm, or a row-range dequantized matvec).

pub const BenchmarkShortcutFlags = struct

Flags marking which benchmark-only fast-paths influenced the run.

The performance suite consults these to decide whether a number is comparable to the reference scalar path or whether a measurement-only shortcut was in effect (top-k forced to zero, LM-head row count capped, decode budget clamped).

pub fn any(self: BenchmarkShortcutFlags) bool

pub const GenerateResult = struct

Output of a `generate` / `generateWithOptions` call: the produced token stream, prefill / decode wall-clock splits, the originally-requested and effective decode budgets, and a set of direct-tier instrumentation counters that report how much of the per-token work was actually retired by the GPU ring versus by the scalar fallback.

The token slice is allocator-owned and must be released with `deinit`.

pub fn deinit(self: *GenerateResult, allocator: std.mem.Allocator) void

pub const GenerateOptions = struct

Knobs threaded through to `generateWithOptions`.

Lets callers opt out of the per-token direct-tier admission validation when they only want the scalar reference numbers.

pub fn generate( model: *const Model, prompt_tokens: []const u32, max_tokens: u32, eos_token_id: u32, allocator: std.mem.Allocator, ) !GenerateResult

Run the full ZINC_RT forward pass against `model` with default `GenerateOptions`: prefill the prompt, validate the per-token decode IR, and decode at most `max_tokens` tokens (stopping early on `eos_token_id`).

Tries the scalar hybrid path, then the scalar dense path, then falls back to a no-layer smoke tail. Equivalent to `generateWithOptions(..., .{})`.

pub fn generateWithOptions( model: *const Model, prompt_tokens: []const u32, max_tokens: u32, eos_token_id: u32, allocator: std.mem.Allocator, options: GenerateOptions, ) !GenerateResult

Full ZINC_RT forward pass with caller-supplied `GenerateOptions`.

Logs the validated decode-graph summary, then picks among three paths: the scalar hybrid MoE+SSM path (for Qwen 3.6-shaped models whose tensors fully resolve via `canRunScalarHybrid`), the scalar dense attention path (for Gemma 4-shaped models via `canRunScalarDense`), and the no-layer smoke tail (everything else). The hybrid and dense paths consume the GPU ring's direct-compute results; the smoke tail is CPU-only. throwaway decode IR.

pub fn initTokenizer(model: *const Model, allocator: std.mem.Allocator) !Tokenizer

Build a `Tokenizer` from `model`'s GGUF vocab metadata.

Convenience wrapper around `Tokenizer.init` so callers don't have to reach into the `Model`'s parse state.

pub const Tokenizer = struct

Minimal longest-match BPE-ish tokenizer that reads the vocab and EOS id straight out of a GGUF file.

Encodes prompts via the GPT-2 byte-to-unicode mapping and falls back to byte 0 on misses so the scalar M1 forward path always sees a well-formed token stream.

pub fn init(gf: *const gguf.GGUFFile, allocator: std.mem.Allocator) !Tokenizer

pub fn deinit(self: *Tokenizer) void

pub fn eosId(self: *const Tokenizer) u32

pub fn encodeGemmaChat(self: *const Tokenizer, user_text: []const u8, allocator: std.mem.Allocator) !?[]u32

pub fn encodePrompt(self: *const Tokenizer, text: []const u8, allocator: std.mem.Allocator) ![]u32

pub fn decodeToken(self: *const Tokenizer, token_id: u32, buf: []u8) []const u8