Forward Cuda

Compile kernels, allocate every device buffer, upload inv_freq + sinks, zero the KV cache and SSM state.

Allocate per-sequence slot state for `n_slots` concurrent sequences, each with up to `slot_ctx` positions.

Attn layers get slot KV; ssm layers get per-slot conv + recurrent state (zero-initialised). Additive — production single-sequence buffers are untouched. Re-callable (frees first).

Byte offset of slot `slot`'s K/V for position `pos` in attn layer `L`'s slot KV: (slot*slot_ctx + pos)*kv_dim*sizeof(f32).

The future per-seq kv-write + slot attention kernels (4b/4c) will use this exact indexing.

Byte offset of slot `slot`'s conv ring (ssm layer): slot*conv_state_len*f4.

Byte offset of slot `slot`'s recurrent state (ssm layer): slot*ssm_state_len*f4.

Effort 28 increment 4 (qwen serving): zero a REUSED slot's SSM conv ring + recurrent state so a new request prefilling from pos=0 into this slot starts from clean state.

The serving engine reuses slots across requests (nslots < concurrent clients) and qwen's SSM state is ACCUMULATED (not position-indexed), so without this a reused slot inherits the prior request's recurrent state → corruption. The attention slot KV needs NO reset (it is position-indexed and overwritten on prefill; reads never reach beyond the current sequence's positions — same reason gemma's slot KV reuses cleanly). No-op if slot state is unallocated. Call on ADMIT, before per-token prefill.

Sub-step 4a plumbing smoke: prove the slot-offset arithmetic round-trips and that distinct slots map to NON-overlapping device regions for BOTH the attention KV and the SSM conv + recurrent state.

Requires allocSlotState with n_slots >= 2 (so slot 0 and slot n_slots-1 differ). Returns true on success.

Reset the production single-sequence recurrent state to a fresh-process start: zero every ssm layer's conv ring + recurrent state and clear the per-layer conv offset.

The attention KV cache needs no reset — it is position-indexed and each sequence overwrites from pos 0, reading only what it wrote. Used by the batch harness to make the per-sequence SERIAL reference truly single-sequence (without this, the unindexed SSM recurrent state leaks from one reference sequence into the next). Additive — never called on the real decode/prefill path; that path runs one sequence/process.

Run one greedy decode step for `token` at sequence position `pos`, returning the argmax token id.

v0: embed → final rms_norm → LM head → argmax (layers are gated by `run_layers`).

Prefill helper: run every layer (build the KV cache) but SKIP the tail rms_norm + LM head + argmax.

A prompt-internal token's logits are never used — only the final prompt token's argmax seeds generation — so the (vocab x n_embd) head matvec is pure waste on T-1 of the T prompt tokens. It is a large share for the MoE models, whose active per-token forward is small next to the full-vocab head. The async layer ops are drained here (waitPending) so the ring is freed each token; bit-identical generation.

Batched prefill that writes KV cache + SSM state into the given slot's per-sequence region (instead of the single-seq scratch).

Used by the serve engine so server-mode prefill gets the 5× batched-GEMM speedup instead of falling back to per-token decodeBatch (B=1).

Download the current hidden state (for sanity checks).

Download the top of the logits buffer (for top-k reporting).